Data transmission method, system and program product for ai computing chip

Abstract

This application discloses a data transmission method, system, and program product for AI computing chips. The method includes: receiving and storing context configuration information for multiple tasks across multiple data transmission channels; determining an authorized channel according to preset arbitration rules in each clock cycle and obtaining context configuration information associated with the current task; generating read and write transfer requests to a target bus based on the context configuration information; reading data from the target bus according to the read transfer request and storing the data in a buffer corresponding to the channel based on the channel identifier; retrieving data from the buffer corresponding to the authorized channel; performing byte shifting and merging processing on the data when the write transfer request is an unaligned transfer or a narrow bit-width transfer; and performing write transfer of the write data through the target bus. This method solves the problems of inefficient scheduling, unaligned performance collapse, and bus bandwidth mismatch in traditional DMA under multi-core high-concurrency scenarios.

Description

Technical Field

[0001] This disclosure generally relates to the field of artificial intelligence (AI), and more specifically to a data transmission method, system, and program product for AI computing chips. Background Technology

[0002] With the development of artificial intelligence, the bandwidth and flexibility of data transfer have become key bottlenecks restricting real-time AI inference. When AI computing chips (such as GPUs, NPUs, etc.) execute large-scale AI model inference tasks, it is necessary to frequently transfer data such as feature maps and weight parameters between external memory and internal computing units. Such data transfer tasks are usually completed by Direct Memory Access (DMA) controllers.

[0003] However, existing DMA technologies face some challenges in multi-core, high-concurrency scenarios. Therefore, a DMA data transfer scheme that can address issues such as bus bandwidth mismatch, inefficient multi-task scheduling, and performance bottlenecks associated with unaligned access is needed. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this application proposes an improved data transmission method for AI computing chips, as well as systems, computer program products, and non-transitory computer-readable media associated with the data transmission method.

[0005] According to a first aspect of this disclosure, a data transmission method for an AI computing chip is provided. The method includes: receiving and storing context configuration information for multiple tasks across multiple data transmission channels; in each clock cycle, determining an authorized channel from all channels initiating transmission requests according to a preset arbitration rule, and obtaining context configuration information associated with the current task corresponding to the authorized channel; generating read transmission requests and write transmission requests to a target bus based on the context configuration information; reading data from the target bus according to the read transmission request, and storing the read data in a buffer corresponding to the channel according to the identifier of the channel to which the read data belongs; retrieving data from the buffer corresponding to the authorized channel, and in response to the write transmission request being an unaligned transmission or a narrow bit-width transmission, performing byte shifting and merging processing on the retrieved data based on the start address alignment requirement of the write transmission request to generate aligned write data conforming to the target bus bit width; and in response to the write channel of the target bus being ready, performing write transmission of the generated write data through the target bus.

[0006] According to a second aspect of this disclosure, a data transmission system for an AI computing chip is provided. The system includes: a configuration module for receiving and storing context configuration information for multiple tasks across multiple data transmission channels; a scheduling module for determining an authorized channel from all channels initiating transmission requests according to a preset arbitration rule in each clock cycle, and obtaining context configuration information associated with the current task corresponding to the authorized channel; a channel control module for generating read and write transmission requests to the target bus based on the context configuration information; a read data processing module for reading data from the target bus according to the read transmission request, and storing the read data in a buffer corresponding to the channel according to the identifier of the channel to which the read data belongs; and a write data processing module for retrieving data from the buffer corresponding to the authorized channel, and, in response to the write transmission request being an unaligned transmission or narrow bit-width transmission, performing byte shifting and merging processing on the retrieved data based on the start address alignment requirement of the write transmission request to generate aligned write data conforming to the target bus bit width. The write data processing module is also used to transmit the generated write data through the target bus in response to the write channel being ready.

[0007] In other aspects of this disclosure, a non-transitory computer-readable medium storing instructions and a computer program product including the instructions are provided. These instructions, when executed by one or more processors, cause the processors to perform the methods described in this disclosure. Attached Figure Description

[0008] The operation and function of these and other features disclosed herein, as well as related structural elements and combinations thereof, will become more apparent upon consideration of the following description and appended claims with reference to the accompanying drawings (all of which form a part of this specification, wherein like reference numerals denote corresponding portions in the drawings). However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and not for limiting purposes.

[0009] Figure 1 A logical structure diagram of a data transmission system for an AI computing chip according to an embodiment of the present disclosure is shown.

[0010] Figure 2 A schematic diagram of the data flow within a data transmission system according to an embodiment of the present disclosure is shown.

[0011] Figure 3 A flowchart of a data transmission method for an AI computing chip according to an embodiment of the present disclosure is shown.

[0012] Figure 4 A schematic diagram of an example computing device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0013] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components and arrangements are described below to simplify this disclosure. Of course, these are merely examples and not limiting.

[0014] In the field of data transmission technology for AI computing chips, existing DMA technology solutions suffer from problems such as bus bandwidth and bit width mismatch, inefficient dynamic scheduling of multiple tasks, and performance collapse due to unaligned access. These problems manifest themselves in the following ways.

[0015] (1) Low bus bandwidth utilization: Traditional advanced eXtensible interfaces (AXI) may not be able to match the data throughput requirements of the computing units inside AI computing chips (e.g., GPUs, NPUs, etc.). For example, an NPU computing unit processes data with a 1024-bit width, while traditional DMA only has a 128-bit AXI interface. A single 1024-bit transfer needs to be split into at least 8 sub-transfers. In addition, the protocol overhead such as address handshaking for each sub-transfer results in an effective bandwidth utilization of less than 40%.

[0016] (2) Excessive task switching latency: When a fixed-channel DMA needs to switch data transmission tasks, the software must reconfigure all registers, which cannot meet the needs of real-time AI inference. For example, in autonomous driving scenarios, the NPU needs to frequently switch between camera feature map transfer tasks and LiDAR point cloud transfer tasks. Each switch requires saving all register configurations of the current task and rewriting them into the new task configuration. This process takes more than 500 nanoseconds, causing the computing unit to be in a waiting state during this period.

[0017] (3) Unaligned access leads to redundant transmission: Existing DMA controllers do not support hardware-level unaligned address access. When AI data (e.g., convolutional feature maps) is stored at unaligned addresses, software is required to perform padding and alignment. For example, if a feature map sub-block is stored at the starting address 0x1003 (not 16-byte aligned), the DMA cannot directly initiate a burst transmission starting from this address. The software needs to allocate an additional alignment buffer, copy and align the data, and then have the DMA move it, resulting in approximately 30% redundant data transmission and wasted memory space.

[0018] These limitations result in low utilization of computing resources, increased system power consumption, and insufficient support for real-time scenarios in practical applications of computing chips.

[0019] To address the aforementioned issues, this application proposes an improved high-bandwidth direct memory access (DMI) data transfer scheme for AI computing chips. According to embodiments of the invention, context configuration information for multiple tasks can be stored in each data transfer channel. Furthermore, in each clock cycle, an authorized channel is determined from all channels initiating transfer requests based on preset arbitration rules, and the context configuration information associated with the current task is obtained. This allows the transfer contexts of multiple tasks to reside simultaneously in the hardware, and authorized channels can switch seamlessly in each clock cycle, significantly reducing task switching latency. In the write data transfer path, in response to unaligned or narrow-width write transfer requests, byte shifting and merging processing is performed on the retrieved data based on the start address alignment requirement to generate aligned write data conforming to the target bus width. Data splicing and alignment can be automatically completed by hardware without software padding and alignment operations, eliminating redundant data transfer and memory space waste in traditional schemes, thereby avoiding performance degradation due to unaligned access. Furthermore, this hardware splicing process ensures that each write transmission is sent in an aligned manner, avoiding the protocol overhead caused by being forced to split into multiple narrow transmissions due to misalignment. Combined with the diversion mechanism that stores the read data from each channel into independent buffers according to the channel identifier and the time-division multiplexing implemented by cycle scheduling, the effective bandwidth utilization can be significantly improved (e.g., improved to near full bandwidth).

[0020] In some implementations of the above technical solutions, before generating read and write transfer requests, the original data transfer instruction can be received and split into multiple sub-transfer instructions based on the segmented transfer length and address jump step size, thereby efficiently supporting the multi-dimensional data block access patterns common in AI computing. Optionally, the address jump step size is greater than the segmented transfer length to support non-contiguous storage scenarios where there are address gaps between data blocks. In some implementations, when a read or write transfer request crosses a preset address boundary, it can be automatically split into multiple sub-transfer requests that do not cross that boundary to ensure that the transfer request conforms to the target bus protocol specification. In some optional implementations, the preset arbitration rule adopts a round-robin scheduling rule, which sequentially polls all channels that initiate transfer requests to determine the authorized channel in each clock cycle. In some implementations, a configuration access request including the target address and read / write control information can be received through an external configuration bus, and the corresponding internal register can be determined after address decoding and read / write operations can be performed, thereby achieving a transparent conversion from external configuration to internal registers. In some implementations, each channel can be configured with four or more register groups to support the simultaneous resident context of more tasks. In some implementations, the write address and the generated write data can be stored in a write address buffer and a write data buffer, respectively, and then retrieved and sent in response to a slave device ready signal on the target bus, thereby accurately matching the receiving capability of the downstream slave device through a backpressure mechanism.

[0021] Figure 1 A logic structure diagram of a data transmission system 100 for an AI computing chip according to an embodiment of the present disclosure is shown. Figure 1 As shown, the data transmission system 100 may include a configuration module 110, a scheduling module 120, a channel control module 130, a read data processing module 140, and a write data processing module 150. In some embodiments, the data transmission system 100 is configured to receive configuration access requests from an external configuration bus 101 and perform actual data read and write transfers via a target bus 102.

[0022] In the embodiments of this disclosure, the external configuration bus 101 can be any bus protocol that supports read and write access to internal registers via address mapping. In the embodiments described below, the external configuration bus 101 can be illustrated using an Advanced High-performance Bus (AHB). However, the technical solutions of this disclosure are not limited to this and can also be applied to other configuration buses with similar addressing capabilities, such as AXI-Lite, Advanced Peripheral Bus (APB), or PCIe. As long as the external configuration bus can perform addressing operations on internal registers via a target address and read / write control information, the configuration interface unit 111 can similarly translate external bus access into read and write operations on DMA internal registers.

[0023] In the embodiments of this disclosure, the target bus 102 can be any bus protocol that supports burst transmission, defines a data width, and has unaligned transfer and / or narrow transfer mechanisms. In the embodiments described below, the target bus 102 is illustrated using the Advanced eXtensible Interface (AXI) bus as an example. However, the technical solutions of this disclosure are not limited to this and can also be applied to other buses with similar mechanisms, such as OCP, Wishbone, or TileLink. As long as the target bus defines a data width, supports byte lane addressing based on the start address, and allows unaligned or narrow transfer, the byte shift merging, address boundary splitting, and scheduling schemes described in this disclosure can be applied similarly.

[0024] In embodiments of this disclosure, configuration module 110 is used to receive and store context configuration information for multiple tasks across multiple data transmission channels. In some embodiments, configuration module 110 may further include configuration interface unit 111 and register unit 112. Configuration interface unit 111 may be configured to receive configuration access requests from an external configuration bus 101 (e.g., an AHB bus), which may include a target address and read / write control information. Configuration interface unit 111 can convert transfers on the external configuration bus 101 into operations on DMA internal registers, including write and read operations. Specifically, configuration interface unit 111 can determine the internal register targeted by the configuration access request based on the target address; for example, it can determine the target channel based on the high-order field of the target address and the specific register within that channel based on the low-order field of the target address. Then, based on the read / write control information, it performs write or read operations on the determined internal register to store or retrieve the corresponding context configuration information.

[0025] Register unit 112 can be configured to accept configuration operations from configuration interface unit 111, enabling read and write operations on registers at corresponding addresses. In some embodiments, register unit 112 further includes channel register unit 113, which can be configured to maintain multiple register groups for each data transmission channel, storing context configuration information for multiple different tasks associated with that channel. In some embodiments, each channel can be configured with four register groups, meaning each channel can simultaneously store the transmission context of four different tasks. In other embodiments, each channel can be configured with more or fewer register groups, depending on the context transmission requirements of the actual application.

[0026] In embodiments of this disclosure, the scheduling module 120 is configured to determine an authorized channel from all channels initiating transmission requests in each clock cycle according to a preset arbitration rule, and obtain context configuration information associated with the current task corresponding to the authorized channel. The preset arbitration rule is used to indicate how to select a unique channel as the authorized channel from all channels initiating transmission requests in each clock cycle. In some embodiments, the preset arbitration rule can be based on channel priority, for example, different priority weights can be pre-assigned to each channel, and the channel with the highest priority among the currently requesting channels can be selected in each clock cycle. In some embodiments, the preset arbitration rule is a round-robin (RR) scheduling rule. When the round-robin scheduling rule is used, the scheduling module 120 determines a channel as the authorized channel from the currently requesting channels in each clock cycle in a preset cyclical order; the next channel after the currently authorized channel becomes the starting polling position for the next clock cycle. Since the authorization decision is made again in each clock cycle, the scheduling module 120 can achieve seamless authorization switching between channels, and the authorization of a single channel will not continue into subsequent clock cycles, thereby achieving low-latency multi-task concurrent scheduling. Specifically, the scheduling module 120 obtains scheduling requests for all channels from the front end (i.e., the channel control module 130 corresponding to each channel), and according to the polling scheduling rules, outputs a permitted channel identifier for each clock cycle of a channel that initiates a transmission request.

[0027] In embodiments of this disclosure, the channel control module 130 is configured to generate read transfer requests and write transfer requests to the target bus 102 based on context configuration information. In some embodiments, each data transmission channel is equipped with an independent channel control module 130. The channel control module 130 may further include a read address control submodule 131 and a write address control submodule 132.

[0028] The read address control submodule 131 is configured to obtain the channel configuration from the channel register unit 113 of the corresponding channel, parse the register configuration into a read transfer request for the target bus (e.g., AXI bus), and send the read scheduling request to the scheduling module 120. The scheduling module 120 obtains the read scheduling requests for all channels from the read address control submodules 131 of each channel, selects an authorized channel in each clock cycle according to the aforementioned arbitration rules, and returns an authorization signal to the read address control submodule 131 corresponding to that authorized channel. The authorized read address control submodule 131 sends the generated read transfer request to the read data processing module 140.

[0029] Similarly, the write address control submodule 132 is configured to obtain the channel configuration from the channel register unit 113 of the corresponding channel, parse the register configuration into a write transfer request to the target bus (e.g., AXI bus), and send the write scheduling request to the scheduling module 120. The scheduling module 120 obtains the write scheduling requests for all channels from the write address control submodules 132 of each channel, selects an authorized channel in each clock cycle according to the arbitration rules described above, and returns an authorization signal to the write address control submodule 132 corresponding to the authorized channel. The authorized write address control submodule 132 sends the generated write transfer request to the write data processing module 150.

[0030] In some embodiments, the read address control submodule 131 is further configured to: when the data transmission requested by the read transfer request crosses a preset address boundary (e.g., the 4KB address boundary specified by the AXI protocol—in the AXI protocol, to prevent addressing errors caused by a single burst transfer crossing the address spaces of different slave devices, a single transfer is prohibited from crossing the 4KB boundary), split the read transfer request into multiple sub-read transfer requests that do not cross the preset address boundary. Similarly, the write address control submodule 132 is further configured to: when the data transmission requested by the write transfer request crosses a preset address boundary, split the write transfer request into multiple sub-write transfer requests that do not cross the preset address boundary. Through the above splitting process, the generated read transfer requests and write transfer requests conform to the protocol specifications of the target bus, avoiding transmission anomalies caused by a single burst transfer crossing the address boundary.

[0031] In embodiments of this disclosure, the read data processing module 140 reads data from the target bus 102 according to a read transfer request, and stores the read data in a buffer corresponding to the channel according to the identifier of the channel to which the read data belongs. In some embodiments, the buffer can be a first-in-first-out (FIFO) buffer, or other buffer structures that can provide data in the order of writing. In some embodiments, the read data processing module 140 maintains an independent buffer for each channel. Specifically, the read data processing module 140 generates a read request for the target bus (e.g., an AXI bus) according to the received read transfer request, that is, generates information in the read address channel on the target bus, including the read address, burst type, and burst length, etc.; then receives the read data returned from the target bus 102; next, the received read data is stored in the buffer corresponding to the channel according to its different channels, for use during write data transmission. By maintaining an independent buffer for each channel, the read data from different channels is isolated from each other and does not interfere with each other, providing an ordered data source for subsequent write data transmission.

[0032] In embodiments of this disclosure, the write data processing module 150 is used to retrieve data from the buffer corresponding to the currently authorized channel, and in response to whether the current write transfer request is an unaligned transfer or a narrow transfer, performs byte shifting and merging processing on the retrieved data based on the start address alignment requirement of the write transfer request to generate aligned write data that conforms to the target bus width.

[0033] As explained above, unaligned transmission refers to a write transmission whose starting address is not aligned with the data width. For example, on a target bus with a data width of 128 bits (16 bytes), a transmission with a starting address of 0x1003 (not a multiple of 16 bytes) is an unaligned transmission. Narrow-width transmission refers to a single transmission where the data width is smaller than the target bus data width. For example, on a 128-bit bus, only 32 bits of valid data are transmitted. The starting address alignment requirement is used to determine which byte channels on the target bus carry the valid data for this transmission. In the case of an unaligned or narrow transmission, the write data processing module 150 performs byte shifting on the retrieved data according to the starting address alignment requirement, moving the valid data bytes to the correct byte channel position. When multiple narrow transmissions need to be merged into a single bus-width transmission, the module also performs merging processing on each data segment, concatenating them byte-by-byte in order to form aligned write data that conforms to the target bus width. Thus, the hardware automatically completes the data concatenation and alignment without requiring software padding and alignment operations.

[0034] In embodiments of this disclosure, the write data processing module 150 is further configured to transmit the generated write data via the target bus 102.

[0035] In a specific example, the write data processing module 150 generates a write request to the target bus (e.g., an AXI bus) based on the received write transfer request, i.e., it generates information in the write address channels on the target bus, including the write address, burst type, and burst length. In the case of an unaligned or narrow-width write transfer request, the write data processing module 150 performs byte shifting and merging processing on the retrieved data. For example, on a 128-bit (16-byte) wide target bus 102, if the write transfer start address is 0x1003 (not 16-byte aligned) and the valid data to be written is 8 bytes, then byte channels 0 to 2 of the target bus correspond to invalid bytes, the 0th byte of valid data is shifted to byte channel 3, the 1st byte to byte channel 4, ..., the 7th byte to byte channel 10, and byte channels 11 to 15 correspond to invalid bytes. The write data processing module 150 identifies the validity of each byte channel through a write strobe signal, thereby generating complete 16-byte aligned write data. When narrow-width transmissions occur, for example, multiple 32-bit (4-byte) narrow-width transmission requests are generated sequentially on a 128-bit bus. The write data processing module 150 can shift the valid data bytes of these narrow-width transmissions to their corresponding byte channel positions and merge them until they are assembled into a single-aligned transmission data unit that conforms to the target bus data width. The processed write address and the generated write data are stored in the write address buffer and write data buffer, respectively. Subsequently, the write data processing module 150 waits for a ready signal from the slave device on the target bus 102; in response to the ready signal, the write data processing module 150 retrieves the write address and write data from the write address buffer and write data buffer, respectively, and completes the write transmission through the target bus 102. This ensures that the write data transmission is precisely matched to the receiving capability of the downstream slave device, avoiding data loss or overflow.

[0036] In some embodiments, the data transmission system 100 may optionally include an instruction slicing module 160. The instruction slicing module 160 is configured to receive an original data transmission instruction, which includes a start address, a total transmission length, segmented transmission lengths, and an address jump step. The instruction slicing module 160 automatically splits the original data transmission instruction into multiple sub-transmission instructions, wherein the transmission length of each sub-transmission instruction is equal to the segmented transmission length, and in two adjacent sub-transmission instructions, the start address of the latter sub-transmission instruction is increased by the address jump step compared to the start address of the former sub-transmission instruction. Then, the channel control module 130 sequentially generates read transmission requests and write transmission requests to the target bus 102 based on each sub-transmission instruction.

[0037] In a specific example, the original data transfer instruction starts at address 0x0, has a total transfer length of 16KB, a segmented transfer length of 4KB, and an address jump step of 16KB. The instruction slicing module 160 is configured to split this original instruction into four sub-transfer instructions: the first sub-transfer instruction starts at address 0x0 and has a transfer length of 4KB; the second sub-transfer instruction starts at address 16KB and has a transfer length of 4KB; the third sub-transfer instruction starts at address 32KB and has a transfer length of 4KB; and the fourth sub-transfer instruction starts at address 48KB and has a transfer length of 4KB. In this example, the address jump step (16KB) is greater than the segmented transfer length (4KB), indicating that there are address gaps between the data segments. This is a typical case of non-contiguous storage of multidimensional feature map data in AI computing. Through this hardware-level slicing mechanism, the software only needs to issue one original data transfer instruction to complete the full access to the multidimensional data block, simplifying the software programming model and improving data transfer efficiency. For example, during convolutional neural network inference, when the NPU needs to access data sub-blocks that are not stored contiguously by row or by channel in a multidimensional feature map, the above-mentioned slicing mechanism can automatically calculate the address jump step size based on the width, height and channel dimension of the feature map, thereby achieving efficient handling of multidimensional data blocks driven by a single instruction.

[0038] Figure 2 A schematic diagram of the data flow within a data transmission system according to an embodiment of the present disclosure is shown. The following is in conjunction with... Figure 2 Describe the data interaction relationships between the various modules within the system.

[0039] like Figure 2 As shown, the data flow unfolds horizontally from left to right. The left side of the diagram represents the configuration path entry, and the right side represents the target bus interface. The overall data flow is divided into two paths: the upper path is the read transfer path, and the lower path is the write transfer path, both sharing the front-end configuration and register modules.

[0040] On the left side of the diagram, external configuration access requests are made through the configuration interface module (dma_cfg_if, which can be used to implement...). Figure 1 The configuration interface unit 111 in the DMA module enters the system. The configuration interface module can translate configuration transfers on the external bus into operations on the DMA's internal registers, including write and read operations. The configuration information is then passed to the DMA register module (dma_reg, which can be used to implement...) Figure 1 Register unit 112 in the module contains channel register submodules (dma_ch_reg, which can be used to implement...) corresponding to each channel. Figure 1The channel register unit 113 is used for each channel. Each channel register submodule is configured to store registers for a single channel, and each channel is equipped with four sets of registers (marked "×4" in the figure) to simultaneously store the context configuration information of four different tasks associated with that channel. The number of channel register submodules corresponds to the number of channels supported by the system (marked "× CH_NUM" in the figure). The register module provides the stored context configuration information to both the read and write transfer paths.

[0041] In the read transmission path, the data stream passes through the following modules sequentially: the read step size slicing module (RD STRIDE, which can be used to implement...) Figure 1 The instruction slicing module 160 (read side) and the read channel address control module (dma_chnl_raddr_ctrl) can be used to implement... Figure 1 The read address control submodule 131 and the read scheduling arbitration module (dma_rd_arbiter, which can be used to implement...) are mentioned. Figure 1 The scheduling module 120 (read-side part) and the read data processing module (rd_data, which can be used to implement...) Figure 1 The read data processing module 140 in the middle is ultimately connected to the target bus through the AXI read interface (AXI RD IF). Specifically, the read step size slicing module (RD STRIDE) receives the original data transmission command and splits it into multiple sub-transmission commands according to the segment transmission length and address jump step size. Each channel is equipped with an independent read channel address control module (dma_chnl_raddr_ctrl). This module obtains the channel configuration from the corresponding channel register submodule (dma_ch_reg), parses the configuration into a read transmission request to the target bus, and decomposes read transmission requests that cross preset address boundaries (e.g., 4K byte address boundaries). Then, it sends the read transmission request to the read scheduling arbitration module (dma_rd_arbiter). The read scheduling arbitration module (dma_rd_arbiter) obtains the read scheduling requests of all channels from the read channel address control modules of each channel and selects an authorized channel to be granted permission in each clock cycle according to the round-robin scheduling rules. The read data processing module (rd_data) issues a read request (i.e. generates information in the read address channel on the target bus) through the AXI read interface (AXI RD IF) according to the authorized read transfer request, receives the returned read data, and stores the read data into the first-in-first-out buffer corresponding to each channel according to the identifier of the channel, so that it can be retrieved and used when writing data is transferred.

[0042] In the write transmission path, the data stream passes through the following modules in sequence: the write stride module (WR STRIDE, which can be used to implement...). Figure 1The instruction slicing module 160 (write side portion) and the write channel address control module (dma_chnl_waddr_ctrl, which can be used to implement...) Figure 1 The write address control submodule 132 and the write scheduling arbitration module (dma_wr_arbiter, which can be used to implement...) Figure 1 The scheduling module 120 (write side part) and the write data processing module (wr_data, which can be used to implement...) Figure 1 The write data processing module 150 is ultimately connected to the target bus via the AXI write interface (AXI WR IF). Specifically, the write step-size slicing module (WR STRIDE) corresponds to the read step-size slicing module, used to split the original data transmission instructions into sub-transmission instructions. Each channel is equipped with an independent write channel address control module (dma_chnl_waddr_ctrl). This module obtains the channel configuration from the corresponding channel register submodule (dma_ch_reg), parses the configuration into a write transmission request to the target bus, and decomposes write transmission requests that cross preset address boundaries. Then, it sends the write transmission request to the write scheduling arbitration module (dma_wr_arbiter). The write scheduling arbitration module (dma_wr_arbiter) obtains the write scheduling requests of all channels from the write channel address control modules of each channel, and selects an authorized channel to be granted permission in each clock cycle according to the round-robin scheduling rules. The write data processing module (wr_data) retrieves the data previously stored by the read data processing module (rd_data) from the FIFO buffer corresponding to the authorized channel, generating information within the write data channel. In the case of unaligned or narrow transmission requests, the write address and write data are concatenated. The processed write address and write data are stored separately in the FIFO buffers inside the write data processing module (wr_data). After waiting for the slave device to send a ready signal on the AXI write interface (AXI WR IF), the write data processing module (wr_data) retrieves the write address and write data from its internal FIFO buffers and completes the write transmission through the AXI write interface (AXI WR IF).

[0043] In the above architecture, the read step-size slicing module and the write step-size slicing module are optional modules. When the system does not enable the step-size slicing function, the configuration information output by the register module is directly passed to the corresponding channel address control module without undergoing step-size slicing processing.

[0044] Figure 3 A flowchart of a data transmission method 300 for an AI computing chip according to an embodiment of the present disclosure is shown. The following is in conjunction with... Figure 3 The method embodiments of this disclosure are described.

[0045] refer to Figure 3 Data transmission method 300 begins with operation S301.

[0046] In operation S301, context configuration information for multiple tasks across multiple data transmission channels is received and stored. In some embodiments, operation S301 further includes: receiving a configuration access request from an external configuration bus (e.g., an AHB bus), the configuration access request including a target address and read / write control information; determining the internal register targeted by the configuration access request based on the target address; and performing a write or read operation on the determined internal register based on the read / write control information to store or retrieve the context configuration information for multiple tasks. In some embodiments, each channel may be configured with four register groups for storing the context configuration information for multiple tasks associated with that channel.

[0047] In operation S302, at each clock cycle, an authorized channel is determined from all channels that have initiated transmission requests according to a preset arbitration rule, and the context configuration information associated with the current task corresponding to the authorized channel is obtained. In some embodiments, the preset arbitration rule is a round-robin scheduling rule, in which the data transmission system sequentially and round-robinally determines one channel as the authorized channel from all channels that have initiated transmission requests at each clock cycle.

[0048] Optionally, before operation S302, when the system enables the instruction slicing function, the data transmission method 300 may further include: receiving an original data transmission instruction, which includes a starting address, a total transmission length, a segmented transmission length, and an address jump step; splitting the original data transmission instruction into multiple sub-transmission instructions, wherein the transmission length of each sub-transmission instruction is equal to the segmented transmission length, and in two adjacent sub-transmission instructions, the starting address of the latter sub-transmission instruction is increased by the address jump step compared to the starting address of the former sub-transmission instruction. In a typical application scenario, the address jump step is greater than the segmented transmission length to support multi-dimensional data access with address gaps between data blocks.

[0049] In operation S303, read transfer requests and write transfer requests to the target bus are generated based on the context configuration information. When the instruction slicing function is enabled, this operation specifically involves generating read transfer requests and write transfer requests to the target bus sequentially based on each sub-transfer instruction. In some embodiments, operation S303 further includes: in response to the data transfer requested by the read transfer request or write transfer request crossing a preset address boundary (e.g., a 4KB boundary), decomposing the read transfer request or write transfer request into multiple sub-transfer requests that do not cross the preset address boundary, to ensure that the generated transfer requests conform to the protocol specifications of the target bus (e.g., an AXI bus).

[0050] In operation S304, data is read from the target bus according to the read transfer request, and the read data is stored in the buffer corresponding to the channel according to the identifier of the channel to which the read data belongs. In some embodiments, operation S304 further includes: generating a target bus read request (e.g., reading information in the address channel on the AXI bus), receiving the returned read data, and storing the read data into the independent buffer corresponding to each channel according to the channel identifier, for retrieval during subsequent write transfers.

[0051] In operation S305, data is retrieved from the buffer corresponding to the authorized channel. In response to a write transfer request that is either an unaligned transfer or a narrow-width transfer, the retrieved data undergoes byte shifting and merging based on the start address alignment requirement of the write transfer request to generate aligned write data that conforms to the target bus width. Specifically, in operation S305, the byte shifting and merging process may include: shifting valid data bytes to the correct byte lane corresponding to the target address, and merging multiple narrow-width transfers of data into a data unit of the bus data width.

[0052] In operation S306, the generated write data is written and transferred via the target bus. In some embodiments, operation S306 further includes: storing the write address and the generated write data in a write address buffer and a write data buffer, respectively; and retrieving the write address and write data from the write address buffer and the write data buffer, respectively, in response to a slave ready signal on the target bus, for writing and transfer via the target bus.

[0053] This disclosure provides a data transmission method and system for AI computing chips, aiming to solve the problems of low bandwidth utilization, poor scheduling efficiency and redundant transmission caused by bus width mismatch, excessive task switching latency and unaligned access performance collapse in traditional DMA technology.

[0054] This disclosure proposes a high-bandwidth direct memory access data transmission scheme for AI computing chips. This scheme stores multi-task context configuration information for each channel, polls to determine the authorized channel and obtains the current task context in each clock cycle, achieving zero-gap task switching to reduce switching latency. In the write data path, unaligned or narrowly transmitted data is automatically shifted and merged by hardware to generate aligned write data, eliminating the need for software padding. Combined with channel-based buffering of read-back data and cycle-by-cycle time-division multiplexing, the effective bandwidth utilization is increased to near full bandwidth. In some implementations, instruction slicing can support multi-dimensional data block access, automatic address boundary decomposition ensures protocol compliance, external configuration bus address decoding enables transparent register conversion, and write address and write data are separated and buffered, with a backpressure mechanism responding to device readiness to match downstream receiving capabilities.

[0055] Figure 4 A schematic diagram of an example computing device 400 according to an embodiment of the present disclosure is shown. This computing device 400 can be used to implement... Figure 1 The data transmission system 100 shown is an example. Figure 4 As shown, computing device 400 may include bus 402 or other communication mechanism for transmitting information, and one or more hardware processors 404 coupled to bus 402 for processing information. The one or more hardware processors 404 may include, for example, one or more general-purpose microprocessors.

[0056] like Figure 4 As shown, in some embodiments, computing device 400 may further include main memory 406 coupled to bus 402. Main memory 406 is used to store information and instructions executed by one or more processors 404, such as random access memory (RAM), cache, and / or other dynamic storage devices. Main memory 406 may also be used to store temporary variables or other intermediate information during the execution of instructions executed by one or more processors 404. When these instructions are stored in storage media accessible to one or more processors 404, they can cause computing device 400 to become a dedicated machine customized to perform the operations specified in the instructions. Storage device 408 may include non-volatile and / or volatile storage media. Non-volatile storage media may include, for example, optical discs or magnetic disks. Volatile storage media may include dynamic memory. Common forms of storage media may include, for example, floppy disks, hard disks, solid-state drives, magnetic tape, or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a perforated pattern, RAM, DRAM, PROM, and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cartridge, or networking versions thereof.

[0057] like Figure 4 As shown, in some embodiments, computing device 400 may further include one or more communication interfaces or network interfaces 410 coupled to bus 402. Network interface 410 may provide bidirectional data communication coupling to one or more network links connected to one or more networks.

[0058] The execution of certain operations can be distributed across processors rather than residing within a single machine, but rather deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0059] Each of the processes, methods, and algorithms described in the preceding sections may be embodied in code modules executed by one or more computer systems or computer processors including computer hardware, and may be fully or partially automated by these code modules. The processes and algorithms may be implemented, partially or fully, in dedicated circuit systems.

[0060] When the functions disclosed herein are implemented as software functional units and sold or used as stand-alone products, they may be stored in a processor-executable, non-volatile, computer-readable storage medium. Specific technical solutions (all or part) disclosed herein, or aspects contributing to the prior art, may be embodied in the form of a software product. The software product may be stored in a storage medium and includes several instructions that cause a computing device (which may be a personal computer, server, network device, etc.) to perform all or some steps of the methods of the embodiments of this application. The storage medium may include a flash drive, portable hard disk drive, ROM, RAM, magnetic disk, optical disk, other media operable to store program code, or any combination thereof.

[0061] Specific embodiments further provide a system including a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor to cause the system to perform operations corresponding to steps in any method of the embodiments disclosed above. Specific embodiments further provide a non-transitory computer-readable storage medium storing instructions executable by one or more processors to cause the one or more processors to perform operations corresponding to steps in any method of the embodiments disclosed above.

[0062] The embodiments disclosed herein can be implemented via a cloud platform, server, or server cluster (collectively referred to below as the "Service System") that interacts with a client. The client can be a terminal device or a client registered by a user at the platform, wherein the terminal device can be a mobile terminal, a personal computer (PC), or any device that can have the platform application installed.

[0063] The various features and processes described above can be used independently of each other or combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Additionally, certain method or process blocks may be omitted in some embodiments. The methods and processes described herein are not limited to any particular order, and their associated blocks or states may be executed in other suitable orders. For example, described blocks or states may be executed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. Example blocks or states may be executed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The exemplary systems and components described herein may be configured differently than described. For example, components may be added to, removed from, or rearranged compared to the disclosed example embodiments.

[0064] The various operations of the exemplary methods described herein can be performed at least in part by an algorithm. The algorithm may be included in program code or instructions stored in memory (e.g., the aforementioned non-transitory computer-readable storage medium). This algorithm may include a machine learning algorithm. In some embodiments, the machine learning algorithm may not explicitly program the computer to perform the function but may learn from training data to produce a predictive model of the function.

[0065] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, these processors can constitute an engine of processor implementation that operates to perform one or more of the operations or functions described herein.

[0066] Similarly, the methods described herein may be implemented at least in part by a processor, wherein one or more specific processors are instances of hardware. For example, at least some operations of the methods may be performed by one or more processors or an engine implemented by a processor. Furthermore, one or more processors may also be operable to support the execution of relevant operations in a “cloud computing” environment or as the execution of relevant operations in a “Software as a Service” (SaaS) context. For example, at least some operations may be performed by a group of computers (as an example of a machine containing processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces (APIs)).

[0067] The execution of certain operations can be distributed across processors rather than residing within a single machine, and can be deployed across multiple machines. In some example embodiments, the processor or processor-implemented engine may reside in a single geographic location (e.g., in a home environment, office environment, or server farm). In other example embodiments, the processor or processor-implemented engine may be distributed across multiple geographic locations.

[0068] Throughout this specification, multiple instances may be implemented as components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of these individual operations may be performed simultaneously, and not necessarily in the order illustrated. Structures and functions presented as separate components in the example configuration may be implemented as composite structures or components. Similarly, structures and functions presented as single components may be implemented as single components. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0069] As used herein, "or" is inclusive rather than exclusive unless explicitly indicated by the context. Therefore, in this document, "A, B, or C" means "A, B, A and B, A and C, B and C, or A, B, and C" unless explicitly indicated by the context. Furthermore, "and" is combined and separate unless explicitly indicated by the context. Therefore, in this document, "A and B" means "A and B, combined or separate" unless explicitly indicated by the context. Additionally, multiple instances of resources, operations, or structures described herein may be provided as a single instance. Furthermore, the boundaries between various resources, operations, engines, and data storage devices are somewhat arbitrary and specific operations are illustrated within the context of a particular illustrative configuration. Other functional assignments are foreseeable and fall within the scope of various embodiments of this disclosure. Generally, structures and functions presented as individual resources in example configurations may be implemented as combined structures or resources. Similarly, structures and functions presented as single resources may be implemented as single resources. These and other changes, modifications, additions, and improvements fall within the scope of the embodiments of this disclosure as expressed in the appended claims. Therefore, this specification and drawings should be considered illustrative rather than restrictive.

[0070] The terms “comprising” or “including” are used to indicate the presence of a subsequently claimed feature, but do not preclude the addition of other features. Unless otherwise specifically stated or otherwise understood in the context in which they are used, conditional language such as “may,” “can,” “may,” and “can” is generally intended to convey that certain embodiments include certain features, components, and / or steps that are not included in other embodiments. Therefore, this conditional language is generally not intended to imply that one or more embodiments require features, components, and / or steps in any way, or that one or more embodiments must include logic for determining whether such features, components, and / or steps are included in or performed in any particular embodiment, with or without user input or prompts.

[0071] Although the general outline of the subject matter has been described with reference to specific exemplary embodiments, various modifications and changes can be made to these embodiments without departing from the broad scope of embodiments of this application. Where more than one embodiment is disclosed, these embodiments of the subject matter may be referred to individually or collectively herein as the term "invention," this is for convenience only and is not intended to automatically limit the scope of this application to any single disclosure or concept.

[0072] The embodiments illustrated herein are described in detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this application. Therefore, "implementation" is not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims and their equivalents in their full scope.

Claims

1. A data transmission method for an AI computing chip, characterized in that, include: Receive and store context configuration information for multiple tasks across multiple data transmission channels; In each clock cycle, an authorized channel is determined from all channels that initiate transmission requests according to preset arbitration rules, and the context configuration information associated with the current task corresponding to the authorized channel is obtained. Based on the context configuration information, generate read transfer requests and write transfer requests to the target bus; Data is read from the target bus according to the read transfer request, and the read data is stored in a buffer corresponding to the channel according to the identifier of the channel to which the read data belongs; Data is retrieved from the buffer corresponding to the authorized channel, and in response to the write transfer request being an unaligned transfer or a narrow bit-width transfer, the retrieved data is byte-shifted and merged based on the start address alignment requirement of the write transfer request to generate aligned write data that conforms to the target bus bit width. as well as The generated write data is transmitted via the target bus.

2. The data transmission method according to claim 1, characterized in that, Also includes: Receive the original data transmission instruction, which includes: starting address, total transmission length, segmented transmission length, and address jump step size; The original data transmission instruction is split into multiple sub-transmission instructions, wherein the transmission length of each sub-transmission instruction is equal to the segment transmission length, and in two adjacent sub-transmission instructions, the starting address of the latter sub-transmission instruction is increased by the address jump step size compared to the starting address of the former sub-transmission instruction. The process of generating the read transfer request and write transfer request for the target bus includes: generating the read transfer request and write transfer request for the target bus in sequence according to each of the sub-transfer instructions.

3. The data transmission method according to claim 2, characterized in that, The address jump step size is greater than the segment transmission length.

4. The data transmission method according to claim 1, characterized in that, Generating the read transfer request and write transfer request to the target bus further includes: In response to the data transmission requested by the read transfer request or the write transfer request crossing a preset address boundary, the read transfer request or the write transfer request is decomposed into multiple sub-transfer requests that do not cross the preset address boundary.

5. The data transmission method according to claim 1, characterized in that, The preset arbitration rules include a polling scheduling rule, which is configured to sequentially determine one channel as the authorized channel from all channels that have initiated transmission requests in each clock cycle.

6. The data transmission method according to claim 1, characterized in that, Receiving and storing context configuration information for multiple tasks on the multiple data transmission channels includes: Receive a configuration access request from an external configuration bus, the configuration access request including a target address and read / write control information; The internal register targeted by the configuration access request is determined based on the target address; Based on the read / write control information, write or read operations are performed on the determined internal registers to store or retrieve the context configuration information of the multiple tasks.

7. The data transmission method according to claim 6, characterized in that, The external configuration bus is the Advanced High Performance Bus (AHB).

8. The data transmission method according to claim 1, characterized in that, The target bus is the Advanced Extensible Interface (AXI) bus.

9. The data transmission method according to claim 1, characterized in that, Each channel is configured with four register groups to store context configuration information for the multiple tasks associated with that channel.

10. The data transmission method according to claim 1, characterized in that, The process of writing and transferring the generated write data through the target bus includes: The write address and the generated write data are stored in corresponding caches, and In response to a slave device ready signal on the target bus, write address and write data are retrieved from the corresponding buffers respectively for write transmission via the target bus.

11. A data transmission system for an AI computing chip, characterized in that, include: The configuration module is used to receive and store context configuration information for multiple tasks across multiple data transmission channels; The scheduling module is used to determine an authorized channel from all channels that have initiated transmission requests in each clock cycle according to preset arbitration rules, and to obtain the context configuration information associated with the current task corresponding to the authorized channel. The channel control module is used to generate read transfer requests and write transfer requests to the target bus based on the context configuration information. The read data processing module is used to read data from the target bus according to the read transmission request, and store the read data in a buffer corresponding to the channel according to the identifier of the channel to which the read data belongs; as well as The write data processing module is used to retrieve data from the buffer corresponding to the authorized channel, and in response to the write transfer request being an unaligned transfer or a narrow bit-width transfer, perform byte shifting and merging processing on the retrieved data based on the start address alignment requirement of the write transfer request to generate aligned write data that conforms to the target bus bit width. The write data processing module is also used to perform write transfer of the generated write data through the target bus.

12. A computer program product, characterized in that... Includes instructions that, when executed by a processor, perform the method according to any one of claims 1-10.

Images