Computing chip, data transmission method and related equipment

Abstract

According to the computing chip, the data transmission method and the related equipment provided by the embodiment of the invention, the computing chip comprises a tensor computing unit and a plurality of execution units, and a hardware direct connection path independent of a shared memory is configured between the tensor computing unit and a storage unit of each execution unit. The hardware direct connection path is used for carrying out data transmission between the tensor calculation unit and the corresponding execution unit, hardware-level direct transmission of data between heterogeneous calculation units is achieved, and therefore the process that data must be transferred through a shared memory in a traditional hybrid architecture is effectively avoided. According to the method and the device, the additional memory access delay and explicit synchronization overhead generated by the write-back-synchronization-add process are eliminated, so that the data transmission path is remarkably shortened while the instruction sending process is simplified, and then the pipeline parallel execution efficiency and the overall data throughput performance of a computing chip are improved when the computing chip executes a large-scale hybrid computing task.

Description

Technical Field

[0001] This application relates to the field of computational data processing technology, and in particular to computing chips, data transmission methods and related equipment. Background Technology

[0002] With the explosive growth of deep learning and large-model inference applications, the computing mode of general-purpose graphics processing units (GPGPUs) has gradually evolved into a hybrid computing architecture. In order to accelerate high-density general-purpose matrix multiplication (GEMM) operations, this hybrid computing architecture typically integrates dedicated tensor cores to perform macroscopic matrix calculations, and works in conjunction with execution units (SIMT units) to perform subsequent element-wise microscopic processing such as layer normalization and activation functions. This mode of coexistence of "macroscopic computing" and "microscopic computing" has become the mainstream computing process of AI chips.

[0003] However, existing hybrid computing architectures rely on on-chip shared memory as an intermediary when interacting with data between units of different granularities (such as tensor computation units and execution units). Specifically, the tensor computation unit must first write a large block of the result matrix to shared memory and insert synchronization instructions, and then the execution unit reads the data into the associated registers using explicit load instructions. This "write-back-synchronization-load" process inevitably introduces shared memory access latency and synchronization overhead, thus limiting the execution efficiency between heterogeneous computing units within the computing chip. Summary of the Invention

[0004] This application provides a computing chip, a data transmission method, and related equipment, which can improve the execution efficiency between heterogeneous computing units in a computing chip.

[0005] To achieve the above objectives, a first aspect of this application provides a computing chip, including: a tensor computing unit and a plurality of execution units, wherein the tensor computing unit and the storage unit of each execution unit are configured with a hardware direct connection independent of shared memory, and the hardware direct connection is used for data transmission between the tensor computing unit and the corresponding execution unit.

[0006] To achieve the above objectives, a second aspect of this application provides a data transmission method for a computing chip, the method comprising:

[0007] Obtain the result data generated by the tensor computation unit executing computation macro instructions;

[0008] Obtain the mapping relationship between the result data and each execution unit, and determine the transmission data corresponding to each execution unit based on the mapping relationship;

[0009] Each piece of data is directly transmitted to the corresponding execution unit via the hardware direct connection.

[0010] In some embodiments, when the result data is matrix data, obtaining the mapping relationship between the result data and each execution unit includes:

[0011] The matrix dimension of the matrix data is obtained based on the tensor descriptor associated with the macro instruction;

[0012] Based on the matrix dimension and hardware sharding granularity, the number of data sub-blocks of the matrix data in the logical space is calculated, and a set of logical coordinates corresponding to each data sub-block is generated.

[0013] The mapping relationship is generated based on the number of fragments and the set of logical coordinates.

[0014] In some embodiments, generating the mapping relationship based on the number of fragments and the set of logical coordinates includes:

[0015] Obtain the unit identifiers of the multiple execution units currently assigned to execute tasks in parallel;

[0016] Based on the topology mapping rules and the number of fragments, each logical coordinate in the logical coordinate set is associated with each unit identifier to generate the mapping relationship indicating the transmission target of each data sub-block.

[0017] In some embodiments, when the computational macro instruction includes a parsing instruction package, the parsing instruction package begins with an execution unit activation instruction, and obtaining the mapping relationship between the result data and each execution unit includes:

[0018] Based on the parameter storage instructions in the parsing instruction package, the shared parameter information and the target register indices corresponding to the multiple execution units are determined.

[0019] The shared parameters are read from the result data based on the shared parameter information;

[0020] The mapping relationship is generated based on the shared parameters and the multiple target register indices.

[0021] In some embodiments, the parsing instruction packet further includes an end instruction, and the method further includes:

[0022] When the end instruction is parsed, the transmitted data in the storage unit corresponding to each target register index is confirmed and detected.

[0023] After confirming that the transmission data corresponding to each execution unit has been written to the corresponding storage unit, the start execution instructions of multiple execution units are triggered.

[0024] In some embodiments, the instruction to trigger the start execution of the plurality of execution units includes:

[0025] The preceding data processing of each execution unit is completed and verified.

[0026] If any of the execution units fails to complete the preceding data transmission processing, a pause signal is generated, and the data transmission of the tensor calculation unit is paused based on the pause signal until all the execution units complete the preceding data transmission processing.

[0027] In some embodiments, the step of directly transmitting each piece of data to the corresponding execution unit via the hardware direct connection includes:

[0028] Get the buffer index for the current transmission cycle;

[0029] For each piece of transmitted data, the dynamic address offset of the transmitted data in the storage unit associated with the corresponding execution unit is determined according to the buffer index;

[0030] The transmitted data is directly written into the storage unit through the hardware direct connection, within the storage space area determined by the dynamic address offset.

[0031] To achieve the above objectives, a third aspect of this application provides a data transmission device for a computing chip, comprising:

[0032] The result data acquisition module is used to acquire the result data generated by the tensor calculation unit executing calculation macro instructions;

[0033] A data transmission determination module is used to obtain the mapping relationship between the result data and each execution unit, and to determine the data transmission corresponding to each execution unit based on the mapping relationship;

[0034] The data transmission module is used to directly transmit each piece of data to the corresponding execution unit through the hardware direct connection path.

[0035] To achieve the above objectives, a fourth aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the data transmission method of the computing chip as described in the second aspect.

[0036] To achieve the above objectives, a fifth aspect of the present application provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data transmission method of the computing chip described in the second aspect above.

[0037] The computing chip, data transmission method, and related devices proposed in this application include a computing chip comprising a tensor computing unit and multiple execution units. A hardware direct connection, independent of shared memory, is configured between the tensor computing unit and the storage unit of each execution unit. By configuring a hardware direct connection, independent of shared memory, between the tensor computing unit and the storage unit of the execution units, this application achieves direct hardware-level data transmission between heterogeneous computing units. This effectively avoids the traditional hybrid architecture where data must be transferred through shared memory, eliminating the additional memory access latency and explicit synchronization overhead caused by the "write-back-synchronization-load" process. This simplifies the instruction sending process while significantly shortening the data transmission path, thereby greatly improving the pipelined parallel execution efficiency and overall data throughput performance of the processor when performing large-scale hybrid computing tasks.

[0038] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the overall hardware architecture of a computing chip provided in one embodiment of this application.

[0040] Figure 2 This is a schematic diagram of an architecture for transmitting shared parameters between workgroup-level instructions and thread bundle-level instructions in a related art, provided in another embodiment of this application.

[0041] Figure 3 This is a schematic diagram of a shared parameter transmission architecture in a related technology provided in another embodiment of this application.

[0042] Figure 4 This is a schematic diagram of the structure of a computing chip provided in another embodiment of this application.

[0043] Figure 5 This is a flowchart of a data transmission method for a computing chip provided in another embodiment of this application.

[0044] Figure 6 yes Figure 5 The flowchart for step 502.

[0045] Figure 7 yes Figure 6 The flowchart for step 603.

[0046] Figure 8 yes Figure 5 Another flowchart for step 502.

[0047] Figure 9 yes Figure 5 The flowchart for step 503.

[0048] Figure 10 This is a schematic diagram of an architecture for direct transmission of matrix data streams based on a hardware direct connection path, provided in another embodiment of this application.

[0049] Figure 11 This is a schematic diagram of the architecture and process for implementing direct hardware transmission of shared parameters based on instruction packet parsing, provided in another embodiment of this application.

[0050] Figure 12 This is a flowchart of a data transmission confirmation and detection process provided in another embodiment of this application.

[0051] Figure 13 This is a flowchart of a pre-transmission data confirmation process provided in another embodiment of this application.

[0052] Figure 14 This is a schematic diagram of the data transmission device of a computing chip provided in another embodiment of this application.

[0053] Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0057] With the explosive growth of deep learning and large-scale model inference applications, the computing model of general-purpose graphics processing units (GPGPUs) has gradually evolved into a hybrid computing architecture. To accelerate high-density general matrix multiplication (GEMM) operations, this hybrid computing architecture typically integrates dedicated tensor cores to perform macroscopic matrix calculations, and coordinates with execution units (such as Single Instruction Multiple Threads, SIMT) to perform subsequent element-wise microscopic processing such as layer normalization and activation functions. This coexistence of "macroscopic computing" and "microscopic computing" has become the mainstream computing process for AI chips.

[0058] Reference Figure 1 This is a schematic diagram of the overall hardware architecture of the computing chip provided in an embodiment of this application. The computing chip is, for example, a general-purpose graphics processing unit (GPGPU). Figure 1 As shown, this computing chip architecture macroscopically includes a host and multiple chips that establish data connections with the host (such as...). Figure 1 (Chips 1 to 3 shown). The host acts as the control terminal, responsible for issuing task data packets; while each chip acts as a processing terminal, integrating a command processor and multiple block processing clusters connected to it. The command processor acts as a chip-level task distribution hub, and can be configured with task data packet caching, parsing modules, and a synchronization engine to deconstruct and route tasks issued by the host to the corresponding block processing clusters.

[0059] At the intermediate level, each block processing cluster contains a scheduling unit and multiple connected computing units (such as...). Figure 1 (See computing units 1 and 2 shown). The scheduling unit plays a crucial role within the cluster, not only receiving operation data packets forwarded by the command processor but also balancing the load and scheduling tasks among its subordinate computing units. This nested cluster design enables the graphics processor to efficiently manage large-scale concurrent tasks, breaking down complex computational loads level by level to the underlying processing cores.

[0060] At the micro level, such as Figure 1As shown in the lower right corner, each computing unit integrates a task scheduling module, a tensor computation unit, and multiple execution units (such as execution units 1 to 3). The task scheduling module is responsible for finer-grained instruction distribution and scheduling within the computing unit; the execution units are general-purpose cores for handling general-purpose parallel computing tasks (such as SIMT instructions); and the tensor computation units are dedicated hardware accelerators for accelerating specific operations such as matrix multiplication and accumulation (MMA).

[0061] Reference Figure 2 This is a schematic diagram illustrating the architecture for transmitting shared parameters between workgroup-level instructions and thread bundle-level instructions in a related art, provided in an embodiment of this application. For example... Figure 2 As shown, after the computing chip performs general matrix multiplication (GEMM) using the tensor computing unit, the calculation results generated by the tensor computing unit cannot be directly sent to the execution unit. Instead, the data needs to be written back to shared memory (SMEM) through the Load Store Unit (LSU) interface. Subsequently, the execution units (such as EU0-EU3) execute the explicit load instruction (Load.smem) to read the data from the shared memory back to their respective register files (RF), so that the subsequent execution units can use the data in the various register files to perform parallel calculations.

[0062] Reference Figure 3 This is a schematic diagram of a shared parameter transmission architecture provided in an embodiment of this application. For example... Figure 3 As shown, when the instruction issuing / scheduling unit (Work group scheduler, WGS) executes work group-level (WG level) macro instructions to prepare shared parameters, it first needs to perform a storage operation (such as...). Figure 3 The operation corresponding to the St.spm / St.gmem instructions shown in the figure writes the shared data to the scratchpad memory (SPM) or global memory (GMEM). Subsequently, before starting or executing warp-level instructions, the execution unit (SIMT) needs to access the scratchpad memory or global memory to perform an explicit load operation to read the shared data and fill it into its internal scalar general-purpose registers, thereby completing the data transfer of shared parameters.

[0063] like Figures 1 to 3During data transmission in the computing chip shown, this hybrid computing architecture relies on on-chip shared memory as an intermediary when interacting with data between units of different granularities (such as tensor computation units and execution units). Specifically, the tensor computation unit must first write a large block of result matrix to shared memory and insert synchronization instructions, and then the execution unit reads the data into the associated register using explicit load instructions. This "write-back-synchronization-load" process inevitably introduces shared memory access latency and synchronization overhead, thus limiting the execution efficiency between heterogeneous computing units within the computing chip.

[0064] To improve the execution efficiency between heterogeneous computing units in a computing chip, this application embodiment configures a hardware direct connection path independent of shared memory between the storage units of the tensor computing unit and the execution unit. This enables direct hardware-level data transmission between heterogeneous computing units, effectively avoiding the process in traditional hybrid architectures where data must be transferred through shared memory. It eliminates the additional memory access latency and explicit synchronization overhead caused by the "write-back-synchronization-load" process, simplifying the instruction sending process while significantly shortening the data transmission path. This, in turn, greatly improves the pipelined parallel execution efficiency and overall data throughput performance of the processor when performing large-scale hybrid computing tasks.

[0065] The computing chip, data transmission method, and related devices provided in the embodiments of this application will be further described below. First, the computing chip will be described.

[0066] Reference Figure 4 This is a schematic diagram of the structure of a computing chip provided in an embodiment of this application. Figure 4 As shown, this computing chip can be a general-purpose graphics processing unit (GPGPU) designed for general-purpose computing or artificial intelligence acceleration. Its internal architecture integrates heterogeneous computing resources, specifically including tensor computation units and multiple execution units. The tensor computation unit refers to a macroscopic instruction execution module specifically designed to perform high-density matrix operations (such as general matrix multiplication GEMM), typically handling large-granularity computational tasks at the workgroup level (WG level). The execution unit refers to the execution core designed based on a single-instruction multithreading (SIMT) model (e.g.,...). Figure 4 The execution units EU0, EU1, EU2, and EU3 are designed to handle fine-grained logical operations or element-wise operations at the warp level.

[0067] To optimize the data interaction efficiency between these two heterogeneous units, the computing chip provided in this application has a hardware direct connection path independent of shared memory between the tensor computing unit and the storage unit of each execution unit. This hardware direct connection path is used for data transmission between the tensor computing unit and the corresponding execution unit. Here, the storage unit refers to a local register resource (e.g., a vector general purpose register (VGPR) or scalar general purpose register (SGPR)) tightly coupled to the execution unit, used to temporarily store operands or instruction parameters to be processed. The hardware direct connection path refers to a dedicated physical data bus or on-chip interconnect network that directly connects the output port of the tensor computing unit to the input port of the execution unit in the hardware structure (e.g., in chip layout design).

[0068] In the computing chip provided in this application, the hardware direct connection is designed to be "independent of shared memory," which means that the data transmission path does not pass through the on-chip shared memory module or the general-purpose load storage unit (LSU) in terms of physical connection and logical control. Under this configuration, the data stream or parameter stream generated by the tensor computation unit does not need to go through the memory access loop of "write to shared memory - synchronization - read from shared memory," but can bypass the common storage level and directly use the hardware direct connection between the tensor computation unit and each execution unit to transmit data directly point-to-point to the private storage space of the target execution unit.

[0069] Understandably, this hardware direct connection can be a hardwiring structure between modules at the physical level, directly interconnecting the tensor computing unit and the parallel computing unit through a dedicated hardware interface. This hardware interface constructs a private physical channel independent of the on-chip general-purpose bus architecture, enabling the tensor computing unit to send high-bandwidth data streams (such as matrix operation results) or control commands (such as configuration parameters) directly to the parallel computing unit without having to temporarily store data through shared memory. This achieves point-to-point direct transmission and communication between modules at the circuit level.

[0070] The computing chip provided in this application eliminates the additional memory access latency and explicit synchronization overhead caused by relying on shared memory in hybrid computing architectures by constructing hardware direct-pass circuits independent of shared memory. This "direct-pass" mechanism not only reduces the occupation of on-chip memory bandwidth, but also greatly simplifies the dependency relationship of instruction pipeline, thereby significantly improving the parallel efficiency and data throughput performance of computing chips when performing hybrid computing tasks such as large-scale deep learning inference.

[0071] Based on the computing chip provided above, the data transmission method applied to this computing chip will be further described below. (Refer to...) Figure 5 This is an optional flowchart of a data transmission method for a computing chip provided in an embodiment of this application. Figure 5 The method may include, but is not limited to, steps 501 to 503. It is also understood that this embodiment... Figure 5 The order of steps 501 to 503 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0072] Step 501: Obtain the result data generated by the tensor calculation unit executing the calculation macro instruction.

[0073] Step 501 will be described in detail below.

[0074] In some embodiments, when the computing chip responds to the execution of computational macro instructions, it first performs the operation of acquiring the result data generated by the tensor computation unit executing the computational macro instructions. Specifically, the tensor computation unit is a core component in the processor specifically designed for handling high-density matrix operations. In response to received computational macro instructions (such as workgroup-level large-scale matrix multiplication and accumulation instructions or instruction packets), it initiates the corresponding hardware logic to perform the operation. The result data generated in this process can refer to general matrix operation output values ​​(such as matrix data after matrix multiplication) or shared parameter information extracted after parsing the instruction packet. This step constitutes the data source of the entire data transmission process, ensuring that subsequent transmission operations are based on valid data generated at the macroscopic computational level.

[0075] Step 502: Obtain the mapping relationship between the result data and each execution unit, and determine the transmission data corresponding to each execution unit based on the mapping relationship.

[0076] Step 502 is described in detail below.

[0077] In some embodiments, the computing chip further executes the mapping relationship between the acquired result data and each execution unit, and determines the operation of the transmission data corresponding to each execution unit based on the mapping relationship. Since the data granularity generated by the tensor computing unit is relatively large, while the data granularity processed by the execution unit is relatively small, a mapping relationship (e.g., topological coordinate mapping based on matrix dimensions or broadcast rules) is needed to establish the correspondence between the transmission data of the two. This facilitates the computing chip's subsequent logical segmentation or assignment of the overall result data based on this mapping relationship, thereby accurately parsing out the specific portion of data or shared data (i.e., transmission data) that each execution unit should receive.

[0078] The following section will further describe in detail how to determine the mapping relationship between the result data and each execution unit in two specific scenarios.

[0079] In the first case, when the result data corresponding to the macro instruction executing the tensor computation unit to process the representation matrix data is matrix data, refer to Figure 6 The process of obtaining the mapping relationship between the result data and each execution unit includes the following steps 601 to 603.

[0080] Step 601: Obtain the matrix dimension of the matrix data based on the tensor descriptor associated with the macro instruction.

[0081] Step 602: Based on the matrix dimension and hardware sharding granularity, calculate the number of data sub-blocks in the logical space of the matrix data, and generate a set of logical coordinates corresponding to each data sub-block.

[0082] Step 603: Generate a mapping relationship based on the number of shards and the set of logical coordinates.

[0083] Steps 601 to 603 are described in detail below.

[0084] In the first case, when the tensor computation unit executes a macro instruction representing the processing of matrix data, the macro instruction can be as shown in the following example code.

[0085] smovi x5, 0x00008000; / / A matrix desc

[0086] smovi x7, 0x00201080; / / B matrix desc

[0087] smov x4, x3; / / C matrix desc

[0088] mma.r.16x32x2048.f16.s8.s8.acc1.out1 x5, x7, x4, <1> / / r representsdst is vgpr

[0089] The process shown in the example code above first loads the tensor descriptors describing the memory layout and format of matrices A, B, and C into registers x5, x7, and x4 respectively using the smovi and smov instructions. Then, the core mma.r matrix multiplication and accumulation instruction is executed. The key feature mark .r explicitly indicates that the output target of this operation is the Vector General Purpose Register (VGPR). This means that after the tensor computation unit completes the matrix operation of the specified size (16x32x2048) and precision (f16 / s8), it will use hardware direct connection to directly distribute and write the result data into the private registers of the execution unit without having to transfer it through shared memory.

[0090] In this case, the resulting data is matrix data. The computing chip first executes operations based on the tensor descriptor associated with the macro instruction to obtain the matrix dimensions of the matrix data. It can be understood that a tensor descriptor is a metadata structure embedded in the macro instruction code or stored in a specific register, which defines the geometry and format information of the results output by the tensor computation unit.

[0091] By parsing the tensor descriptor, the computing chip can determine the number of rows and columns of the matrix data generated by the current operation in the mathematical dimension (i.e., matrix dimension, such as M×N), as well as the bit width precision occupied by each element (i.e., data type, such as FP16 or FP32). This provides the necessary total data volume and geometric boundary basis for subsequent data fragmentation, ensuring that the chip can correctly identify the characteristics of the data to be transmitted.

[0092] Furthermore, the computing chip performs operations based on matrix dimensions and hardware fragmentation granularity, calculating the number of data sub-blocks of the matrix data in the logical space, and generating a set of logical coordinates corresponding to each data sub-block. It can be understood that hardware fragmentation granularity refers to the specification of the smallest data unit (e.g., a 16×16 matrix block) that a single execution unit (such as a Warp) can receive or process in one transmission cycle.

[0093] The computing chip uses this hardware granularity as a divisor to perform grid-based segmentation calculations on the total matrix dimension of the acquired matrix data, thereby determining how many small blocks the entire large matrix has been divided into (i.e., the number of slices). Simultaneously, the system assigns a unique index location (e.g., [Block X, Block Y]) in a logical two-dimensional space to each segmented data sub-block. These index locations collectively constitute a set of logical coordinates, thus completing the structured discretization of large-scale matrix data at the logical level.

[0094] In one example, the output data of the tensor computation unit is a 64-row × 64-column half-precision floating-point (FP16) matrix. The processor first reads the tensor descriptor associated with the current computation macro instruction and parses the matrix dimension information. Simultaneously, the system obtains the current hardware fragmentation granularity, assuming it is preset to 16×16, which is the smallest data block size that a single execution unit (e.g., a warp thread bundle) can receive or process in a single transfer operation.

[0095] Based on the above parameters, the hardware logic in the computing chip automatically calculates the number of slices. The computing chip divides the total number of rows of the matrix data, 64, by the number of rows at the hardware slice granularity, 16, to obtain a slice count of 4 in the logical row direction; similarly, it divides the total number of columns, 64, by the number of columns at the hardware slice granularity, 16, to obtain a slice count of 4 in the logical column direction. Thus, the computing chip determines that the macroscopic matrix is ​​discretized into a total of 16 (4×4) independent data sub-blocks in the logical space.

[0096] Subsequently, the system generates a set of corresponding logical coordinates for these 16 data sub-blocks. Each data sub-block is assigned a unique two-dimensional logical index, ranging from the starting coordinate (0, 0) at the top left corner to the ending coordinate (3, 3) at the bottom right corner. For example, the data sub-block located in the third column of the first row is marked with logical coordinates (0, 2). These generated logical coordinates not only identify the relative geometric position of each data sub-block in the original large matrix, but also provide a unique addressing basis for accurately routing these data blocks to specific physical units in subsequent steps (e.g., assigning coordinates (0, 2) to the execution unit with ID 2 according to mapping rules).

[0097] Next, the computing chip performs an operation to generate a mapping relationship based on the number of fragments and the set of logical coordinates. At this point, the computing chip, according to preset topology rules, associates the logical coordinates of each generated data sub-block with the hardware identifiers (IDs) of the multiple physically available execution units. This association establishes the routing rules for "where each data sub-block should go," i.e., the mapping relationship. This relationship table or logic circuit not only indicates the direction of data flow but also implicitly contains the parallelism allocation strategy, ensuring that subsequent hardware direct connections can accurately push the correct data blocks to the corresponding computing cores based on this relationship.

[0098] The following section will describe in more detail how to generate a mapping relationship based on the number of shards and the set of logical coordinates.

[0099] Reference Figure 7 Based on the number of fragments and the set of logical coordinates, a mapping relationship is generated, including the following steps 701 to 702.

[0100] Step 701: Obtain the unit identifiers of the multiple execution units currently assigned to parallel execution tasks.

[0101] Step 702: Based on the topology mapping rules and the number of fragments, associate each logical coordinate in the logical coordinate set with each unit identifier to generate a mapping relationship indicating the transmission target of each data sub-block.

[0102] Steps 701 to 702 are described in detail below.

[0103] In some embodiments, the computing chip first performs an operation to obtain the unit identifiers of multiple execution units currently assigned to parallel execution tasks. Specifically, before preparing for data transmission, the computing chip first needs to determine which hardware resources (i.e., which execution units) are currently active and participating in a specific workgroup task. It is understood that a unit identifier (Unit ID) is a unique physical address or logical index value assigned to each execution unit (such as a SIMT Core or EU (Execution Unit)). By reading the task scheduler or hardware status register, the computing chip can identify the identity information of the group of execution units currently assigned computing tasks, thereby determining the receiving end in the data transmission process and ensuring that subsequent data distribution is only targeted at valid and active execution units.

[0104] Then, the computing chip performs an operation based on topology mapping rules and the number of fragments, associating each logical coordinate in the logical coordinate set with the unit identifier corresponding to the execution unit at each receiving end, generating a mapping relationship indicating the transmission target of each data sub-block. In this process, the topology mapping rule refers to a predefined hardware routing strategy (such as row-first filling, zigzag filling, or modulo hash algorithm) that specifies how to map logical data blocks to physical computing arrays.

[0105] As can be understood, row-first filling is a linear sequential mapping rule that allocates consecutive logical data sub-blocks to execution units with adjacent physical IDs according to the row index priority in the logical coordinate set. This method ensures that data in the same row in the logical space can be mapped to a set of computational cores with consecutive physical IDs, thereby maximizing the continuity of memory access when processing matrix operations with row dependencies.

[0106] Z-shaped fill is a non-linear mapping rule based on a space-filling curve. It traverses a two-dimensional logical space according to a recursive "Z"-shaped trajectory, interweaving two-dimensional logical coordinates into one-dimensional physical unit identifiers. This rule can significantly maintain the spatial locality of data in two-dimensional space, making logically adjacent matrix blocks (whether row-adjacent or column-adjacent) more likely to be assigned to computationally nearby units, thereby optimizing on-chip cache hit rate.

[0107] Modulo hashing is a load balancing mapping rule based on discretization. It calculates and assigns a unique target unit identifier by performing a specific hash operation on the logical coordinates of data sub-blocks and taking the modulo of the total number of execution units. This rule can discretize and evenly distribute continuous logical data streams across all physical computing units, effectively avoiding local hardware hotspots or uneven load caused by excessively dense data in specific areas.

[0108] According to this rule, the computing chip pairs and binds the logical coordinates representing the location of the data sub-block with the confirmed unit identifier (i.e., the unit identifier). This association process can be achieved by establishing a routing lookup table to clarify which specific physical computing unit each logical data block should be sent to, thus forming the final mapping relationship that guides data transmission.

[0109] In one example, suppose the current workgroup task is assigned to 16 execution units (e.g., thread bundles labeled Warp0 to Warp15) for joint execution. The computing chip obtains the unit identifiers (i.e., ID sequence: 0, 1, ..., 15) of these 16 active execution units by reading the task scheduler's status information, thereby establishing the list of receiver addresses for the physical transport layer.

[0110] Subsequently, the computing chip performs mapping calculations from logical space to physical space. Assume the preset topology mapping rule is "Row-Major Mapping," and the number of slices is known to be 4 (i.e., 4 data blocks per row). The hardware logic, according to this rule, performs a one-to-one association operation between each generated logical coordinate and the acquired cell identifier. For example, for a data sub-block with logical coordinates (0, 0), the system calculates the mapping index 0 × 4 + 0 = 0, associating it with Warp 0; for a data sub-block with logical coordinates (1, 2) (i.e., the block in the 2nd row and 3rd column), the system calculates the mapping index 1 × 4 + 2 = 6, associating it with Warp 6.

[0111] A complete mapping table is generated by the computing chip (e.g., {(0,0)→Warp 0; ...; (1,2)→Warp 6; ...; (3,3)→Warp 15}). This mapping will serve as the routing instructions for subsequent hardware direct connections, ensuring that each segmented data sub-block can be accurately pushed to the designated execution unit for processing.

[0112] Through steps 701 to 702 above, by combining real-time unit identifier acquisition with dynamic association based on topology rules, it can flexibly adapt to parallel computing arrays of different sizes and realize automated directional routing from data sub-blocks to physical computing units. This not only avoids the hard-coded dependency of the software layer on the underlying hardware address, but also ensures that the data load can be evenly distributed to each execution unit according to the expected topology, thereby improving the utilization rate of hardware resources and the efficiency of parallel computing in the computing chip.

[0113] Through steps 601 to 603 above, by introducing tensor descriptor parsing and logical coordinate sharding technology, automatic adaptation and load balancing from macroscopic matrix data to microscopic parallel computing resources are achieved. This allows the large-scale result data generated by the tensor computing unit to be automatically divided into sub-blocks adapted to the processing capabilities of the execution unit without the need for cumbersome loop addressing or manual intervention at the software level, and accurately routed to the target execution unit. This greatly improves the parallelism and bus bandwidth utilization of large-scale matrix data transmission under the heterogeneous computing architecture in the computing chip.

[0114] In the second case, when the computation macro instruction includes a parsing instruction packet, the parsing instruction packet begins with the execution unit activation instruction, referring to... Figure 8 The process of obtaining the mapping relationship between the result data and each execution unit includes the following steps 801 to 803.

[0115] Step 801: Based on the parameter storage instructions in the parsed instruction packet, determine the shared parameter information and the target register indices corresponding to multiple execution units.

[0116] Step 802: Read the shared parameters from the result data based on the shared parameter information.

[0117] Step 803: Generate a mapping relationship based on shared parameters and multiple target register indices.

[0118] Steps 801 to 803 are described in detail below.

[0119] In the first case, when the macro instructions executed by the tensor computation unit include a parsing instruction package, the parsing instruction package starts with the execution unit activation instruction (call.eu) and ends with the termination instruction (stsgpr), and includes parameter storage instructions. The parsing instruction package can be shown in the following example code.

[0120] call.eu 0, 0, <2> ; / / function call for SIMT codes

[0121] stsgpr s11, x5; / / the shared data needs to write warp sgpr s11. ...

[0123] stsgpr.end s12, x6; / / the end of instruction packet of call-eu

[0124] Understandably, this parsing instruction package begins logically with the `call.eu` instruction (i.e., the unit activation instruction) to activate the SIMT program of the execution unit; this is followed by a series of `stsgpr` parameter storage instructions (such as `stsgpr s11,x5`) to instruct the hardware to directly write the shared data in the workgroup-level registers (such as `x5`) into the scalar general-purpose registers (such as `s11`) of the target execution unit; finally, the `stsgpr.end` instruction (such as `stsgpr.end s12, x6`, i.e., the end instruction) serves as the end marker of the package, announcing the end of the parameter configuration sequence to the hardware, thereby achieving atomic encapsulation of computation initiation and parameter configuration at the instruction flow level.

[0125] In this case, the resulting data is the shared parameter. The computing chip first executes the parameter storage instruction based on the parsed instruction packet to determine the shared parameter information and the target register indices corresponding to multiple execution units. Specifically, when the computing chip's instruction scheduling unit recognizes that the parsed instruction packet begins with an execution unit activation instruction (e.g., Call.EU, used to start a SIMT thread), it continues to decode the subsequent instruction sequence (including the parameter storage instruction).

[0126] The parameter storage instruction (e.g., stsgpr) is a control instruction specifically used to define data transmission behavior. Its encoded field explicitly carries the characteristic description of the data source (i.e., shared parameter information, such as the register address of the source operand) and the storage location identifier of the receiving end on the execution unit side (i.e., the target register index, such as the number of the scalar register SGPR). Thus, by parsing the parameter storage instruction, the shared parameter information of the shared parameters to be transmitted and the target register indexes corresponding to multiple execution units can be determined, making the hardware decoding of the two key control elements of "what to transmit" and "where to store" clear.

[0127] Then, the computing chip performs the operation of reading shared parameters from the result data based on shared parameter information. Understandably, in this case, the result data refers to the values ​​stored in the macro registers (WGGPR) associated with the tensor computation unit or workgroup engine. Based on the parsed shared parameter information (such as the source address of the shared parameter), the computing chip directly extracts the actual configuration values ​​as shared parameters from these macro registers, such as tensor descriptors, memory base addresses, or access steps.

[0128] Finally, the computing chip performs an operation to generate a mapping relationship based on shared parameters and multiple target register indices. Since parameter configuration typically has a "one-to-many" broadcast characteristic, the computing chip logically binds the read shared parameters to the determined target register indices, generating a broadcast control signal, i.e., a mapping relationship. This mapping relationship explicitly indicates the hardware path: this set of shared parameters should be concurrently written to all active execution units within the current workgroup, and the specific write location is the specific scalar register pointed to by the target register index.

[0129] In one example, when the instruction scheduling unit parses an instruction packet starting with `call.eu`, the hardware decoding logic further processes the parameter storage instructions following the packet, such as `stsgpr s11, x5`. Based on the instruction's encoding format, the processor determines that the shared parameter information originates from the workgroup-level register `x5`, and simultaneously determines the target register indices as the scalar general-purpose register `s11` in the first execution unit (Warp 0) and the scalar general-purpose register `s12` in the second execution unit (Warp 1).

[0130] Then, the source register x5 specified in the hardware logic access instruction of the control processor is used to read the actual stored value, i.e., the shared parameter, such as a tensor descriptor address 0x00008000, or the base address pointer of the video memory.

[0131] Finally, the computing chip uses the read value and the target index to construct an internal mapping signal, which logically indicates: "Write the value 0x00008000 simultaneously into {Warp 0's11 register and Warp 1's12 register}". This mapping is then sent to the SGPR access interface, causing the driver hardware circuitry to concurrently fill 0x00008000 into s11 and s12 scalar registers.

[0132] Through steps 801 to 803 above, by directly extracting parameters and generating broadcast mappings during the instruction packet parsing stage, the parameter passing process between heterogeneous units is internalized as part of the instruction pipeline. This not only avoids the redundant operation of "store-load" transfer that requires external shared memory or global memory in traditional architectures, thus significantly reducing the memory access latency of parameter configuration, but also greatly simplifies the hybrid computing programming model and reduces synchronization overhead through instruction packet encapsulation similar to function calls.

[0133] Step 503: Transmit each data item directly to the corresponding execution unit via a hardware direct connection.

[0134] Step 503 will be described in detail below.

[0135] In some embodiments, the computing chip ultimately performs the operation of directly transmitting each data transmission to the corresponding execution unit via a hardware pass-through path. Here, a hardware pass-through path refers to... Figure 4 As shown, the physical layout design is independent of on-chip shared memory, directly connecting to the high-bandwidth data bus or dedicated interface of heterogeneous computing units. Through this hardware direct connection, the computing chip can bypass traditional storage-level relay links and write the predetermined transmission data point-to-point into the local storage resources (such as register files) of the target execution unit, thereby realizing the physical "direct delivery" of transmission data and completing the final distribution from macroscopic calculation results to microscopic execution units.

[0136] The physical transmission process will be described further below.

[0137] Reference Figure 9 Each data transmission is directly transmitted to the corresponding execution unit through a hardware direct connection, including the following steps 901 to 903.

[0138] Step 901: Obtain the buffer index for the current transmission cycle.

[0139] Step 902: For each piece of transmitted data, determine the dynamic address offset of the transmitted data in the memory unit associated with the corresponding execution unit based on the buffer index.

[0140] Step 903: Through a hardware direct connection, the transmitted data is directly written into the storage unit, the storage space area determined by the dynamic address offset.

[0141] Steps 901 to 903 are described in detail below.

[0142] In some embodiments, the computing chip first performs an operation to obtain the buffer index of the current transfer cycle. It is understood that, to support pipelined parallel processing, the computing chip in this embodiment employs a multi-buffering mechanism (such as a double-buffered Ping-Pong Buffer) to manage the data read / write rhythm. Here, the current transfer cycle refers to a specific time window in which the tensor computation unit is sending data to the execution unit. During this current transfer cycle, the computing chip's hardware logic maintains a status flag, i.e., a buffer index (e.g., 0 or 1), to indicate which buffer region is currently open for write operations, while another region may be being read by the execution unit, thus clarifying the timing context of the current data transfer and preventing read / write conflicts.

[0143] Then, the computing chip performs an operation for each transmitted data, determining the dynamic address offset of the transmitted data in the memory unit associated with the corresponding execution unit based on the buffer index. It can be understood that the memory unit refers to the Vector Register File (VGPR) within the execution unit. Because register resources are logically divided into multiple buffer blocks, the computing chip may not be able to write using static absolute addresses. Therefore, the computing chip's hardware logic needs to calculate a displacement value relative to the base address, i.e., the dynamic address offset, based on the obtained buffer index.

[0144] Understandably, the calculation of this dynamic address offset can be achieved through address generation logic or dedicated hardware circuitry within the processor. Its working principle involves using the current buffer index (e.g., a 0 or 1 flag indicating the Ping-Pong buffer state) as a dynamic factor, and then multiplying or performing a hardware left shift operation on the dynamic factor with a pre-defined single-block buffer capacity (i.e., the storage stride). Through this real-time calculation, the system can derive a numerical increment relative to the register file base address, i.e., the dynamic address offset. This allows the write pointer of the hardware direct-access path to automatically jump to the starting position of a specific memory block currently in a "free" or "writable" state, ensuring that data transfer and parallel computation can occur simultaneously without conflict in different memory partitions.

[0145] Finally, the computing chip utilizes a hardware direct connection independent of shared memory to directly push the data stream output from the tensor computation unit to the specific physical register location (i.e., the storage space region) in the register port of the target execution unit, based on a pre-calculated dynamic address offset, to complete the data writing. This process is entirely driven by hardware signals, requiring no software instruction intervention in address calculation, thus achieving high-speed data landing.

[0146] Through steps 901 to 903 above, by introducing the combination of buffer index and dynamic address offset, this application achieves deep overlap between computation and transmission. This allows the execution unit to process the previous frame of data (e.g., located in buffer A) while the tensor computation unit can write the next frame of data directly into the currently free buffer (e.g., located in buffer B) without conflict. This effectively masks the data transmission delay and greatly improves the continuous computation capability and overall throughput of the computing chip.

[0147] Reference Figure 10 This is a schematic diagram of an architecture for direct transmission of matrix data streams based on hardware direct-connection paths, provided in an embodiment of this application. For example... Figure 10 The diagram illustrates the process of matrix data block transmission based on the computing chip provided above, involving a tensor computation unit and multiple execution units (identified as execution units EU0 to EUN). During this process, the tensor computation unit executes workgroup-level matrix multiply-accumulate (MMA) macro instructions (e.g., instruction code mma.r.16x32x2048.f16.s8.s8.acc1.out1 x5, x7, x4, ...). <1> The process involves obtaining matrix data, then using the chip's hardware control logic to parse the tensor descriptors in the macro instructions (such as the matrix descriptor A / B / C matrix desc preset by the smovi instruction) and establishing the data layout of the matrix data.

[0148] Subsequently, the hardware control logic of the computing chip logically divides the resulting matrix data generated by the tensor computation unit according to a predefined matrix layout. Then, through the hardware direct connection between the tensor computation unit and each execution unit, it writes the divided data sub-blocks separately and in parallel into the registers (VGPRs) associated with different execution units (or different warps). This achieves direct data transfer from the Tensor Core to EUs, compared to existing technologies (i.e., such as...). Figure 2 The scheme shown eliminates read / write latency and synchronization wait in shared memory, significantly improving data throughput efficiency between heterogeneous computing units.

[0149] Reference Figure 11This is a schematic diagram of an architecture and process for direct hardware transmission of shared parameters based on instruction packet parsing, provided in an embodiment of this application. Figure 11 The diagram illustrates the broadcast transmission process of shared parameters based on the parsed instruction packets provided above for the computing chip. When the computing chip's workgroup engine executes the `call.eu` instruction (i.e., the unit activation instruction), the hardware logic recognizes it as the start marker of the instruction packet and treats the subsequent series of `stsgpr` instructions (i.e., parameter storage instructions) up to the `stsgpr.end` end instruction as a complete atomic operation packet. During this process, the computing chip's instruction issue / schedule unit does not write these parameters to external memory. Instead, it uses hardware direct connections to write the parsed shared parameters in parallel to the register files of each thread bundle in the execution unit through the corresponding scalar general-purpose register access interface. When the `stsgpr.end` instruction (i.e., the end instruction) is parsed, it signifies that the shared parameter configuration is complete, and the execution unit can then start executing thread-level (Warp-level) SIMT instructions. At this point, each thread bundle does not need to execute additional load instructions and can directly read the pre-initialized shared data from its local SGPR, thereby greatly simplifying the parameter passing programming model and reducing memory access latency.

[0150] In addition, the parsed instruction packet also includes an end instruction, see reference. Figure 12 The data transmission method in the computing chip also includes the following steps 1201 to 1202.

[0151] Step 1201: When the end instruction is parsed, the transmitted data in the memory unit corresponding to each target register index is checked for confirmation.

[0152] Step 1202: After confirming that the transmission data corresponding to each execution unit has been written to the corresponding storage unit, trigger the start execution instructions of multiple execution units.

[0153] Steps 1201 to 1202 are described in detail below.

[0154] In some embodiments, when the computing chip parses an end instruction, it performs an acknowledgment check on the transmitted data in the memory cell corresponding to each target register index. The end instruction (e.g., the aforementioned Stsgpr.end) is a logical terminator of the instruction packet, which announces the completion of the current parameter configuration or data transmission task to the hardware control logic.

[0155] At this point, in response to the termination instruction, the computing chip activates the hardware-level scoreboard mechanism or state machine logic to perform an acknowledgment check to verify the atomicity and integrity of data transmission. This involves checking whether each physical register (i.e., memory unit) indicated by the target register index of each execution unit has completed the physical write operation of the transmitted data, and ensuring that shared parameters (such as configuration information) and data sub-blocks (such as matrix fragments) are in a stable and valid state, thereby preventing subsequent instructions from reading outdated data.

[0156] Then, after confirming that the transmission data corresponding to each execution unit has been written to the corresponding memory unit, the computing chip triggers the start execution instructions of multiple execution units. When the detection logic returns a signal indicating that all transmission data has been successfully written to the registers of the corresponding execution units, the computing chip will release the blocking state of the execution units or issue a start signal (Kick-off), thereby directly activating the program counter of the execution units and enabling them to start executing start execution instructions (such as warp-level operation code), ensuring that the execution units will only enter the running state if the input data is absolutely complete.

[0157] The following section will further describe the necessary checks required when executing the trigger start-up command.

[0158] Reference Figure 13 This triggers the start execution instructions of multiple execution units, including the following steps 1301 to 1302.

[0159] Step 1301: Perform a verification of the completion of the preceding data transmission for each execution unit.

[0160] Step 1302: When any execution unit has not completed the preceding data transmission processing, a pause signal is generated, and the data transmission of the tensor calculation unit is paused based on the pause signal until all execution units have completed the preceding data transmission processing.

[0161] Steps 1301 to 1302 are described in detail below.

[0162] In some embodiments, after confirming that the transmission data corresponding to each execution unit has been written to the corresponding storage unit, the computing chip performs a verification operation on the processing completion of the preceding transmission data for each execution unit. The preceding transmission data refers to the data load that has been written to the execution unit's storage unit before the current transmission cycle (e.g., the previous pipeline cycle). Specifically, since the tensor computation unit (as a producer) and the execution unit (as a consumer) may operate at different frequencies or loads, in order to prevent new data from overwriting old data that has not yet been consumed, the computing chip's hardware logic needs to maintain a backpressure detection mechanism. At this time, the computing chip determines whether each execution unit has completed reading or processing this part of the old data by polling the status register of each execution unit or checking the "ready / busy" flag, thereby establishing the security of the current transmission channel.

[0163] Finally, when any execution unit has not completed its preceding data transfer processing, the computing chip generates a pause signal and suspends data transfer in the tensor computing unit based on this signal until all execution units have completed their preceding data transfer processing. It is understandable that the pause signal (such as a Stall Signal or Flow Control Signal) is a high-priority hardware control signal. Once the detection logic detects that at least one execution unit is busy, the system triggers this signal to freeze the output pipeline of the tensor computing unit, thus implementing the "slowest node synchronization principle," meaning that the pause will only be lifted and data transfer will resume when all target units report an idle state.

[0164] Through steps 1301 to 1302 above, and by implementing a rigorous completion check and global pause strategy, the common "production-consumption" rate mismatch problem in heterogeneous computing architectures is effectively solved. This not only fundamentally eliminates data overflow or overwrite errors caused by untimely processing at the receiving end, ensuring the integrity and correctness of data transmission, but also enables the hardware to adaptively adjust the working rhythm between different computing units, realizing automatic pipeline alignment and collaborative work.

[0165] In one example, suppose the instruction scheduling unit of the computing chip is parsing an instruction packet containing parameter configurations and eventually reads the end instruction stsgpr.end. At this point, the computing chip's hardware logic immediately activates the "transmission completion confirmation" mechanism to poll the register file status of the target execution unit (e.g., Warp 0 to Warp 15).

[0166] At this point, the hardware scoring board of the computing chip detects that the memory cells (SGPRs) from Warp 0 to Warp 14 have all returned "write complete" signals, but Warp 15 has not yet completed the write due to internal bus arbitration delay. The confirmation detection logic remains in a waiting state until Warp 15 also returns a completion signal, confirming that all transmitted data (i.e., configuration parameters) has been stably written to the corresponding memory cells. Subsequently, the computing chip immediately sends start execution instructions simultaneously to the program counters (PCs) of these 16 execution units, switching them from idle to running states, and initiating the computation task using the newly written data.

[0167] Meanwhile, to ensure the security of data transmission within the computing chip, the computing chip continuously executes detection logic in the data transmission pipeline. That is, before the tensor computation unit prepares to send a new round of matrix fragment data to the execution unit, the hardware flow control unit of the computing chip will check the processing completion of the previous transmission data (i.e., the data sent in the previous round but not yet consumed) for each execution unit.

[0168] Suppose that during the detection process, the computation chip's execution processing discovers that although most execution units are idle, execution unit EU3, due to executing a complex activation function, still has unprocessed preceding data remaining in its input buffer (i.e., execution unit EU3 is in a "busy" state). At this point, the computation chip's hardware flow control logic immediately generates a high-level pause signal and, based on this signal, forcibly pauses the data transmission port of the tensor computation unit, freezing the entire transmission pipeline. This pause continues until execution unit EU3 completes the reading and computation of the preceding data and releases the input buffer. Only then is the pause signal removed, allowing the tensor computation unit to continue transmitting new data, ensuring that no overwrite conflicts occur between old and new data.

[0169] Through steps 1201 to 1202 above, by introducing mandatory confirmation checks and condition triggers when parsing the end instruction, this application strictly guarantees the "write first, execute later" timing logic at the hardware level. This not only effectively avoids the "read-after-write" data hazards and race conditions common in heterogeneous computing, but also eliminates the need to manually insert synchronization wait instructions (Barrier) at the software level. Thus, while ensuring the correctness of the calculation, it further simplifies the instruction flow and reduces synchronization overhead.

[0170] Through steps 501 to 503 above, the data source is first established, then the data transmission direction is logically determined through mapping relationships, and finally the data transmission is physically performed using hardware direct connection paths. This avoids the cumbersome process of "write-back-synchronization-load" that must be performed through shared memory in traditional architectures, significantly reduces the data interaction latency between heterogeneous computing units, reduces pauses in the instruction pipeline, and thus greatly improves the overall parallel computing efficiency and data throughput of the processor.

[0171] This application also provides a data transmission device for a computing chip, which can implement the data transmission method of the computing chip described above, referring to... Figure 14 The device 1400 includes:

[0172] The result data acquisition module 1410 is used to acquire the result data generated by the execution of the computation macro instruction by the tensor computation unit.

[0173] The data transmission determination module 1420 is used to obtain the mapping relationship between the result data and each execution unit, and to determine the data transmission corresponding to each execution unit based on the mapping relationship;

[0174] The data transmission module 1430 is used to directly transmit each piece of data to the corresponding execution unit via a hardware direct connection path.

[0175] In some embodiments, the data transmission determination module 1420 is further configured to:

[0176] The matrix dimension of the matrix data is obtained based on the tensor descriptor associated with the macro instruction;

[0177] Based on the matrix dimension and hardware sharding granularity, the number of data sub-blocks in the logical space of the matrix data is calculated, and a set of logical coordinates corresponding to each data sub-block is generated.

[0178] A mapping relationship is generated based on the number of fragments and the set of logical coordinates.

[0179] In some embodiments, the data transmission determination module 1420 is further configured to:

[0180] Get the unit identifiers of the multiple execution units currently assigned to execute tasks in parallel;

[0181] Based on the topology mapping rules and the number of fragments, each logical coordinate in the logical coordinate set is associated with each unit identifier to generate a mapping relationship indicating the transmission target of each data sub-block.

[0182] In some embodiments, the data transmission determination module 1420 is further configured to:

[0183] Based on the parameter storage instructions in the parsing instruction packet, the shared parameter information and the target register indices corresponding to multiple execution units are determined.

[0184] Shared parameters are read from the result data based on shared parameter information;

[0185] A mapping relationship is generated based on shared parameters and multiple target register indices.

[0186] In some embodiments, the data transmission module 1430 is further configured to:

[0187] When the end instruction is parsed, the transmitted data in the memory unit corresponding to each target register index is checked for confirmation.

[0188] After confirming that the transmission data corresponding to each execution unit has been written to the corresponding storage unit, the start execution instructions of multiple execution units are triggered.

[0189] In some embodiments, the data transmission module 1430 is further configured to:

[0190] The preceding data transmission is processed and verified for each execution unit;

[0191] If any execution unit fails to complete the preceding data transmission processing, a pause signal is generated, and the data transmission of the tensor calculation unit is paused based on the pause signal until all execution units have completed the preceding data transmission processing.

[0192] In some embodiments, the data transmission module 1430 is further configured to:

[0193] Get the buffer index for the current transmission cycle;

[0194] For each piece of transmitted data, the dynamic address offset of the transmitted data in the memory unit associated with the corresponding execution unit is determined based on the buffer index.

[0195] Through a hardware direct connection, the transmitted data is written directly into the storage unit, and the storage space area is determined by the dynamic address offset.

[0196] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, the specific implementation of the data transmission device of the computing chip is basically the same as the specific implementation of the data transmission method of the computing chip, and will not be repeated here.

[0197] This application also provides an electronic device, including:

[0198] At least one memory;

[0199] At least one processor;

[0200] At least one program;

[0201] The program is stored in memory, and the processor executes at least one program to implement the data transmission method of the computing chip described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0202] Please see Figure 15 , Figure 15 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0203] The processor 1501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0204] The memory 1502 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1502 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1502 and is called and executed by the processor 1501 to execute the data transmission method of the computing chip in the embodiments of this application.

[0205] The input / output interface 1503 is used to implement information input and output;

[0206] The communication interface 1504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0207] Bus 1505 transmits information between various components of the device (e.g., processor 1501, memory 1502, input / output interface 1503, and communication interface 1504);

[0208] The processor 1501, memory 1502, input / output interface 1503 and communication interface 1504 are connected to each other within the device via bus 1505.

[0209] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the data transmission method of the computing chip described above.

[0210] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0211] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0212] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0213] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0214] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0215] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0216] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, or indirect coupling or communication connection between the apparatus or units, and may be electrical, mechanical, or other forms.

[0218] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0219] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0220] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0221] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A computing chip, characterized in that, include: The system comprises a tensor computation unit and multiple execution units. Each execution unit has a hardware direct connection to its storage unit, which is independent of shared memory. This hardware direct connection is used for data transmission between the tensor computation unit and the corresponding execution unit.

2. A data transmission method for a computing chip, characterized in that, include: Obtain the result data generated by the tensor computation unit executing computation macro instructions; Obtain the mapping relationship between the result data and each execution unit, and determine the transmission data corresponding to each execution unit based on the mapping relationship; Each piece of data is directly transmitted to the corresponding execution unit via the hardware direct connection.

3. The data transmission method according to claim 2, characterized in that, When the result data is matrix data, obtaining the mapping relationship between the result data and each execution unit includes: The matrix dimension of the matrix data is obtained based on the tensor descriptor associated with the macro instruction; Based on the matrix dimension and hardware sharding granularity, the number of data sub-blocks of the matrix data in the logical space is calculated, and a set of logical coordinates corresponding to each data sub-block is generated. The mapping relationship is generated based on the number of fragments and the set of logical coordinates.

4. The data transmission method according to claim 3, characterized in that, The process of generating the mapping relationship based on the number of fragments and the set of logical coordinates includes: Obtain the unit identifiers of the multiple execution units currently assigned to execute tasks in parallel; Based on the topology mapping rules and the number of fragments, each logical coordinate in the logical coordinate set is associated with each unit identifier to generate the mapping relationship indicating the transmission target of each data sub-block.

5. The data transmission method according to claim 2, characterized in that, When the computational macro instruction includes a parsing instruction package, the parsing instruction package begins with an execution unit activation instruction, and obtaining the mapping relationship between the result data and each execution unit includes: Based on the parameter storage instructions in the parsing instruction package, the shared parameter information and the target register indices corresponding to the multiple execution units are determined. The shared parameters are read from the result data based on the shared parameter information; The mapping relationship is generated based on the shared parameters and the multiple target register indices.

6. The data transmission method according to claim 5, characterized in that, The parsing instruction packet also includes an end instruction, and the method further includes: When the end instruction is parsed, the transmitted data in the storage unit corresponding to each target register index is confirmed and detected. After confirming that the transmission data corresponding to each execution unit has been written to the corresponding storage unit, the start execution instructions of multiple execution units are triggered.

7. The data transmission method according to claim 6, characterized in that, The instructions that trigger the start execution of the plurality of execution units include: The preceding data processing of each execution unit is completed and verified. If any of the execution units fails to complete the preceding data transmission processing, a pause signal is generated, and the data transmission of the tensor calculation unit is paused based on the pause signal until all the execution units complete the preceding data transmission processing.

8. The data transmission method according to claim 2, characterized in that, The step of directly transmitting each piece of data to the corresponding execution unit via the hardware direct connection includes: Get the buffer index for the current transmission cycle; For each piece of transmitted data, the dynamic address offset of the transmitted data in the storage unit associated with the corresponding execution unit is determined according to the buffer index; The transmitted data is directly written into the storage unit through the hardware direct connection, within the storage space area determined by the dynamic address offset.

9. A data transmission device for a computing chip, characterized in that, include: The result data acquisition module is used to acquire the result data generated by the tensor calculation unit executing calculation macro instructions; A data transmission determination module is used to obtain the mapping relationship between the result data and each execution unit, and to determine the data transmission corresponding to each execution unit based on the mapping relationship; The data transmission module is used to directly transmit each piece of data to the corresponding execution unit through the hardware direct connection path.

10. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the data transmission method of the computing chip according to any one of claims 2 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data transmission method of the computing chip according to any one of claims 2 to 8.

Images