Data loading method, processor, electronic device and medium

Abstract

This disclosure provides a data loading method, processor, electronic device, and medium. The method includes: determining a loading dimension and a preset size of a data block based on a loading instruction of an acquired tensor to be processed, and the starting coordinates of each of a plurality of data loading engines, wherein the preset size of the data block is spaced between the first starting coordinates of a first data loading engine and the second starting coordinates of a second data loading engine; loading a data block of the preset size from the tensor to be processed along the loading dimension direction using the first starting coordinates and the second starting coordinates as the current coordinates of the starting points, respectively; in response to the completion of loading a data block of the preset size, updating the current coordinates of the starting point in the first dimension by adding a step size, so as to continue loading another data block of the preset size based on the updated current coordinates of the starting point.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and specifically to a data loading method, apparatus, processor, electronic device, and storage medium. Background Technology

[0002] In recent years, fields such as artificial intelligence, deep learning, and high-performance computing have achieved rapid development. In these applications, core algorithms such as Generalized Matrix Multiplication (GEMM) and multidimensional convolution operations typically require frequent processing of massive amounts of tensor data. A tensor can be understood as a multidimensional array, encompassing features such as batch size, depth, height, width, and channel count. With the continuous improvement of computing power in modern processors (such as the arithmetic logic unit (ALU) or dedicated tensor computing cores), the data paths of these computing units are becoming increasingly wider, leading to a sharp increase in the demand for data input bandwidth. Therefore, memory access bandwidth and data transfer efficiency are gradually becoming bottlenecks restricting the overall performance of the processor. Summary of the Invention

[0003] This disclosure provides a data loading method, apparatus, processor, electronic device, and storage medium for multiple data loading engines to load tensor data into a cache in parallel, thereby improving the efficiency of processor data loading and enhancing hardware computing utilization.

[0004] According to one aspect of this disclosure, a data loading method is provided, applied to a processor including multiple data loading engines. The method includes: obtaining a loading instruction for a tensor to be processed; determining a loading dimension and a preset size of a data block, and the starting coordinates of each of the multiple data loading engines, based on the loading instruction, wherein a preset size of a data block is spaced between the first starting coordinates of a first data loading engine and the second starting coordinates of a second data loading engine; loading a data block of the preset size from the tensor to be processed along the loading dimension direction, with the first data loading engine and the second data loading engine respectively using the first starting coordinates and the second starting coordinates as the current coordinates of the starting points, wherein the data block includes a preset number of data elements continuously arranged in the first dimension of the tensor to be processed; in response to the completion of loading a data block of the preset size, the first data loading engine updates the current coordinates of the starting point in the first dimension by adding a step size, so as to continue loading another data block of the preset size continuously based on the updated current coordinates of the starting point, wherein the step size is determined based on the number of multiple data loading engines and a preset number; wherein the loading dimension is the same as the first dimension, or the loading dimension is an adjacent dimension higher than the first dimension.

[0005] In some embodiments, the data of the tensor to be processed is stored contiguously in the first dimension, and the loading dimension is the same as the first dimension. Loading a data block with a preset size from the tensor to be processed along the loading dimension direction includes: the data loading engine generating a loading request based on the current coordinates and the preset size to obtain the pixel of the tensor to be processed at the corresponding position of the current coordinates; in response to the generation of the loading request, incrementing the coordinate value of the current coordinates in the loading dimension by a unit step and incrementing the consecutive loading count by one to generate the next loading request; in response to the consecutive loading count reaching a preset number, determining that the data block has been loaded and resetting the consecutive loading count.

[0006] In some embodiments, the data of the tensor to be processed is stored non-contiguously in the first dimension, and the loading dimension is an adjacent dimension higher than the first dimension. Loading a data block with a preset size from the tensor to be processed along the loading dimension direction includes: the data loading engine generating a loading request based on the current coordinates and the preset size to obtain a preset number of pixels in a row at the position corresponding to the current coordinates of the tensor to be processed; in response to the generation of the loading request, incrementing the coordinate value of the current coordinates in the loading dimension by a unit step to generate the next loading request; and in response to the current coordinates exceeding the size of the data block in the loading dimension, determining that the data block has been loaded.

[0007] In some embodiments, the first dimension is the width dimension of the tensor to be processed, and the loading dimension is the height dimension of the tensor to be processed.

[0008] In some embodiments, the method further includes: splitting the loading request into multiple loading sub-requests based on the relative relationship between the current coordinates and the effective data boundary of the tensor to be processed and the preset size of the data block, so that the data requested to be loaded by each loading sub-request is either entirely within the effective data range of the tensor to be processed, or none of them are within the effective data range of the tensor to be processed.

[0009] In some embodiments, based on the relative relationship between the current coordinates and the effective data boundary of the tensor to be processed, and the preset size of the data block, splitting the loading request into multiple loading sub-requests includes: generating a loading sub-request to obtain padding data for the tensor to be processed when the data of the preset size corresponding to the current coordinates is completely outside the left boundary of the effective data of the tensor to be processed; generating two loading sub-requests when the data of the preset size corresponding to the current coordinates crosses the left boundary of the effective data, one loading sub-request being used to obtain padding data and the other loading sub-request being used to obtain data from within the effective data range; generating a loading sub-request to obtain data from within the effective data range when the data of the preset size corresponding to the current coordinates is completely between the left and right boundaries of the effective data; generating two loading sub-requests when the data of the preset size corresponding to the current coordinates crosses the right boundary of the effective data, one loading sub-request being used to obtain data from within the effective data range and the other loading sub-request being used to obtain padding data; and generating a loading sub-request to obtain padding data when the data of the preset size corresponding to the current coordinates is completely outside the right boundary of the effective data.

[0010] In some embodiments, in response to the data requested to be loaded by the load sub-request not being within the valid data range of the tensor to be processed, the data loading engine writes a preset constant value to the processor's target cache as fill data.

[0011] In some embodiments, a hardware counter is used to record the number of consecutive loads.

[0012] In some embodiments, multiple data loading engines begin loading data from the tensor to be processed in parallel at the same clock cycle.

[0013] In some embodiments, multiple data loading engines asynchronously load data from tensors to be processed, starting at different clock cycles.

[0014] In some embodiments, the processor further includes a computing unit, and the number of multiple data loading engines is determined based on the data channel bandwidth of the computing unit and the preset size of the data block, so that the data acquired in parallel by the multiple data loading engines is assembled to meet the input bandwidth requirements of the computing unit.

[0015] According to one aspect of this disclosure, a data loading apparatus is provided, comprising: an instruction acquisition module configured to acquire a loading instruction for a tensor to be processed; an instruction parsing module configured to determine, based on the loading instruction, a loading dimension and a preset size of a data block, and the starting coordinates of each of a plurality of data loading engines, wherein the first starting coordinate of a first data loading engine and the second starting coordinate of a second data loading engine are spaced apart by the preset size of the data block; a data loading module configured to include a plurality of data loading engines, the plurality of data loading engines being configured to load data blocks of a preset size from the tensor to be processed along the loading dimension direction, respectively using the first starting coordinate and the second starting coordinate as the current coordinates of the starting points, wherein the data blocks include a preset number of data elements continuously arranged in the first dimension of the tensor to be processed; and a coordinate update module configured to, in response to the completion of loading a data block of the preset size, control the corresponding data loading engine to perform a step update on the current coordinates of the starting point in the first dimension by adding a step size, so as to continue loading another data block of the preset size continuously based on the updated current coordinates of the starting point, wherein the step size is determined based on the number of the plurality of data loading engines and the preset number, wherein the loading dimension is the same as the first dimension, or the loading dimension is an adjacent dimension higher than the first dimension.

[0016] According to one aspect of this disclosure, a processor is provided, comprising: an instruction parsing unit configured to receive and parse a data loading instruction, wherein the data loading instruction instructs execution to load a tensor to be processed from memory into a cache; and an execution unit configured to execute the data loading instruction, wherein the execution unit executes the data loading instruction, including implementing the above-described data loading method to load data of the tensor to be processed from memory through a plurality of data loading engines.

[0017] According to one aspect of this disclosure, an electronic device is provided, including: a processor according to an embodiment of this disclosure; and a memory connected to the processor, wherein the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the data loading method described above.

[0018] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein when the computer-executable instructions are executed by a processor, the above-described data loading method is implemented. Attached Figure Description

[0019] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 A schematic diagram of the structure of a general-purpose graphics processor according to an embodiment of the present disclosure is shown.

[0021] Figure 2 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown.

[0022] Figure 3A Exemplary starting coordinates for several data loading engines according to embodiments of this disclosure are shown.

[0023] Figure 3B A schematic diagram of data loading is shown when the loading dimension is the width dimension according to an embodiment of the present disclosure.

[0024] Figure 3C A schematic diagram of data loading is shown when the loading dimension is the height dimension according to an embodiment of the present disclosure.

[0025] Figure 4 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown when the loading dimension and the first dimension are the same.

[0026] Figure 5 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown when the loading dimension and the first dimension are different.

[0027] Figure 6 A schematic diagram illustrating boundary processing and coordinate jump when the loading dimension is the width dimension according to an embodiment of the present disclosure is shown.

[0028] Figure 7 An exemplary flowchart of a boundary handling method when the loading dimension is the height dimension, according to an embodiment of the present disclosure, is shown.

[0029] Figure 8 A schematic diagram illustrating the mapping between state machine states and data regions when the loading dimension is the height dimension according to an embodiment of this disclosure is shown.

[0030] Figure 9 A schematic diagram is shown showing a loading request crossing the right boundary of valid data when the loading dimension is the height dimension according to an embodiment of the present disclosure.

[0031] Figure 10 An exemplary block diagram of a data loading apparatus according to an embodiment of the present disclosure is shown.

[0032] Figure 11 An exemplary block diagram of a processor according to an embodiment of the present disclosure is shown.

[0033] Figure 12 An exemplary block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0034] Figure 13 A block diagram of an example computer device according to an embodiment of the present disclosure is shown.

[0035] Figure 14 An exemplary block diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0036] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0037] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0038] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0039] To alleviate the bottleneck of memory access bandwidth and data transfer efficiency becoming increasingly restrictive to overall processor performance, modern processor architectures typically introduce dedicated data loading engines. These engines offload heavy workloads such as address calculation, boundary checks, and data transfer from the core computing threads. In this architecture, the computing unit only needs to issue an asynchronous memory access instruction containing a tensor descriptor and starting coordinates. The dedicated data loading engine then automatically calculates the multidimensional address and asynchronously transfers the multidimensional data from global memory to shared memory or a cache closer to the computing unit, thereby achieving asynchronous and efficient processing of computation and memory access.

[0040] However, as the throughput requirements of the underlying computing units continue to increase, the bandwidth of a single data loading engine often has a physical limit, making it difficult to meet the instantaneous input bandwidth demands of large computing units. Although the bandwidth shortage can be compensated for by increasing the amount of data in a single request, the size of a continuous data block loaded in a single operation is usually limited due to the bus width and hardware structure constraints of the underlying storage system.

[0041] To effectively expand the input bandwidth to the computing unit, one solution is to configure multiple data loading engines to work in parallel. However, the storage layout of multidimensional tensor data in physical memory is often complex and diverse (e.g., there are differences such as row-first layout and channel-first layout). This means that logically adjacent tensor elements may be contiguous in physical memory addresses, or they may become discontinuous due to being separated by data from other dimensions.

[0042] If multiple data loading engines are allowed to initiate requests independently, or if tensor data is loaded continuously in a single dimension, it is not only difficult to efficiently coordinate and assemble the complete continuous data block required by the computing unit, but also leads to complex hardware state machine design and huge address calculation overhead when dealing with coordinate jumps in multi-dimensional space and padding of tensor out-of-bounds.

[0043] Therefore, existing processor architectures lack a general data loading mechanism that can efficiently organize multiple data loading engines to work in parallel and collaboratively, and can flexibly adapt to the different dimensions of tensor continuity using unified underlying logic.

[0044] To address the aforementioned issues, embodiments of this disclosure provide a fast and efficient data loading method that enables parallel loading of tensor data by multiple data loading engines for different storage layouts. This not only significantly increases the input bandwidth of the computing unit but also allows for flexible adaptation to the continuity characteristics of tensors in different dimensions using a unified underlying state machine logic. This avoids complex address calculations and greatly improves data handling efficiency and the overall performance of the processor.

[0045] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0046] Figure 1 A schematic diagram of the structure of a general-purpose graphics processor according to an embodiment of the present disclosure is shown.

[0047] like Figure 1 As shown, a general-purpose graphics processing unit (GPU) is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can include a streaming processor cluster, such as... Figure 1The diagram shows streaming processor clusters 1 to M, where M is a positive integer. In a general-purpose graphics processor, one streaming processor cluster can handle one computational task, or multiple streaming processor clusters can work together to handle one computational task. Multiple streaming processor clusters can share data through global storage spaces such as global caches or High Bandwidth Memory (HBM).

[0048] Taking streaming processor cluster 1 as an example, each streaming processor cluster can include multiple computing units, such as Figure 1 The computation unit consists of computation units 1, 2, and N, where N is a positive integer. Each computation unit is used to perform specific arithmetic and logical operations, such as accumulation, reduction, regular addition, subtraction, multiplication, division, and matrix multiplication and addition operations for multidimensional tensors.

[0049] A computing unit can include multiple cores, each of which may include an arithmetic logic unit (ALU), a floating-point unit, etc., to execute specific computational tasks. In addition, a computing unit may include a register file and shared memory for hierarchical storage of source and destination data related to the computational task. Cores within a computing unit can share data and communicate with each other through this shared memory. Furthermore, a streaming processor cluster may include a buffer, which can be used to share data between different computing units within the cluster, or as a low-latency relay for loading external data (such as tensor data in high-bandwidth memory) into the computing unit.

[0050] In parallel computing, computational operations in fields such as artificial intelligence (e.g., convolution operations, generalized matrix multiplication (GEMM), etc.) typically require processing massive amounts of multidimensional tensor data. This data is usually stored in external high-bandwidth memory or global cache. Computational cores (especially ALUs with ultra-wide data paths) have extremely high requirements for data throughput and access bandwidth when performing high-concurrency computations. If different memory layouts lead to untimely data supply, the computational core will be in a waiting state, severely reducing the hardware's computational utilization.

[0051] To address the aforementioned technical problems of limited memory access bandwidth and the inability of computing unit data paths to fully utilize input data, such as... Figure 1 As shown, the general-purpose graphics processor provided in this embodiment of the disclosure includes multiple data loading engines (e.g., Figure 1The data loading engines shown are 1, 2, and P (where P is a positive integer greater than 1). These data loading engines are configured to specifically handle moving data (e.g., tensor data) from high-bandwidth memory or global cache to caches or shared memory within a streaming processor cluster, or moving data from within a streaming processor cluster to high-bandwidth memory or global cache. Furthermore, the data loading engines can also perform data transfer between streaming processor clusters.

[0052] Specifically, to further expand the bandwidth of a single data load and meet the enormous input bandwidth requirements of the computing unit, the multiple data loading engines in the embodiments of this invention are configured to work collaboratively and load data in parallel. By having multiple data loading engines work in parallel, the general-purpose graphics processor can achieve step-by-step, skip-step data loading (for example, after controlling the first data loading engine to load several consecutive pixels, a certain interval is skipped, and the second data loading engine loads the data within that interval). This parallel memory access arrangement improves the efficiency of data loading, multiplies the bus bandwidth, and thus enables faster improvement in the computational utilization of the computing unit hardware and faster release of cache space.

[0053] Those skilled in the art should understand that Figure 1 The embodiments shown are merely exemplary and do not constitute a limitation on the scope of this disclosure. In actual hardware architectures, the coupling of modules or units in the various figures is not limited to the one illustrated. For example, the coupling method between the data loading engine and the streaming processor cluster, as well as the data provision method between the data loading engine and the streaming processor cluster and computing units, can have a variety of flexible configurations. For example, one data loading engine may correspond to one streaming processor cluster or computing unit; one data loading engine may serve multiple streaming processor clusters or computing units; multiple data loading engines may collaborate to provide concurrent loading services for a streaming processor cluster or computing unit with high bandwidth requirements, for example, by merging and assembling data through multiple data loading engines to meet the needs of a single high-bandwidth computing unit; or multiple data loading engines and multiple streaming processor clusters or computing units may form a cross-matrix service. Furthermore, Figure 1 The number of each module (M, N, P) can be flexibly set according to specific computing power requirements and hardware resources, and this disclosure does not impose specific restrictions on this.

[0054] Figure 2 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown.

[0055] In embodiments of this disclosure, for example, having Figure 1The device or electronic device shown is a general-purpose graphics processor hardware architecture that includes multiple data loading engines, used to perform... Figure 2 The data loading method 200 is shown in the figure.

[0056] Figure 2 Exemplary steps of method 200 are shown.

[0057] In step S202, the loading instruction for the tensor to be processed is obtained.

[0058] In step S204, based on the loading instructions, the loading dimension and the preset size of the data block, as well as the starting coordinates of each data loading engine among the multiple data loading engines, are determined.

[0059] In step S206, the first data loading engine and the second data loading engine use the first starting coordinate and the second starting coordinate as the current coordinates of the starting point, respectively, to load a data block with a preset size from the tensor to be processed along the loading dimension direction.

[0060] In step S208, in response to the completion of loading a data block of a preset size, the first data loading engine updates the current coordinates of the starting point by adding a step size in the first dimension, so as to continue loading another data block of a preset size based on the updated current coordinates of the starting point.

[0061] In the fields of artificial intelligence or deep learning, tensors are typically used to perform generalized matrix multiplication (GEMM), convolution operations, or other tensor operations. Figure 1 The architecture of the general-purpose graphics processor shown allows these tensors to be processed (e.g., input feature map data, weight parameters, etc.) to be initially stored in a large-capacity memory outside the chip or on-chip (e.g., Figure 1 This involves retrieving tensors from large-capacity memory and loading them into high-speed temporary storage (e.g., high-bandwidth memory or global cache) closer to the computational cores for high-concurrency processing. Figure 1 It is located in the cache or shared memory within the streaming processor cluster for use by the arithmetic logic units within the computing core.

[0062] In some embodiments, the tensor to be processed can be stored in the NDHWC data storage format. Here, N represents the batch dimension, D represents the depth dimension, H represents the height dimension, W represents the width dimension, and C represents the channel dimension. In the NDHWC format, the physical address arrangement of data in memory is with dimension C as the fastest changing inner dimension, followed by W, H, D, and N dimensions in sequence. That is, it follows a purely linear pattern of continuous arrangement, starting with all channel elements at the first spatial location determined by the height and width dimensions, and then storing all channel elements at the next adjacent spatial location.

[0063] In some embodiments, the tensor to be processed can also be stored in the N(C / x)DHW(xC) data storage format, where x is a positive integer greater than 1. This format is often referred to as interleave mode. Similar to NDHWC, but with the key difference that the number of channels is divided into multiple groups of x channels (e.g., x can be a multiple of 2 such as 8, 16, or 32), and each group of x channels is locally arranged according to the NDHWC format.

[0064] It is understandable that, regardless of whether the tensor to be processed is stored in the aforementioned NDHWC or N(C / x)DHW(xC) data storage format, from the perspective of the user or upper-level software, they can both be regarded as a linear layout storage of a multidimensional array, that is, a large contiguous storage structure that can be described by a single instruction containing the starting address and the dimensions of each dimension. Different memory layouts only result in different strides of each dimension in the physical memory address. Similarly, it is understandable that, without deviating from the principle of linear layout storage, the order of tensor dimensions can be adjusted according to the actual situation. For example, W can be used as the inner dimension that changes the fastest.

[0065] However, when faced with the enormous data throughput demands of computing units, the limited bandwidth of a single data loading engine means that simply performing multiple sequential loading operations without intervals using a single engine to compensate for the bandwidth deficiency will severely impact computational efficiency. To double the bandwidth, embodiments of this disclosure break down a continuous loading instruction into several smaller instructions based on data blocks (e.g., 1×8 pixel blocks) and utilize multiple data loading engines to collaboratively load data.

[0066] To facilitate understanding of the parallel loading logic of multidimensional tensors in the underlying hardware of this disclosure, and considering the continuity of tensors across dimensions, the following embodiments primarily illustrate the split loading (i.e., per w split or per h split) of the tensor to be processed in the W (width) or H (height) dimension. Specifically, when the tensor is read discontinuously in an inner dimension (such as the C dimension), the hardware state machine will split the loading in the adjacent higher-level dimension (such as the W dimension); similarly, when it is discontinuous in the W dimension, it will split the loading in the H dimension. In the embodiments of this disclosure, regardless of the dimension in which the split is performed, multiple data loading engines are used to perform jump-step loading in units of continuous one-dimensional data blocks. Without departing from the principles of this disclosure, the embodiments of this disclosure can also be applied to scenarios where the split is performed in any other dimension.

[0067] Specific examples of this disclosure will be described in detail below.

[0068] In step S202, the loading instruction for the tensor to be processed is obtained.

[0069] In some embodiments, it can be achieved through Figure 1 The instruction dispatch module or instruction parsing unit in the general-purpose graphics processor (e.g., the instruction scheduler at the front end of a streaming processor cluster) obtains instructions to load the tensor to be processed from mass storage to high-speed memory. In some examples, this could be from... Figure 1 High-bandwidth memory, global cache, or system main memory are used to load tensors to be processed, for example... Figure 1 The data loading direction can be defined by the cache within a streaming processor cluster, the shared memory within a computing unit, or the register file directly connected to the arithmetic logic unit. Those skilled in the art can determine the specific definition of the data loading direction based on the actual situation.

[0070] It is understood that the "Load" instruction mentioned in the embodiments of this disclosure can also be broadly interpreted as a data transfer instruction. For example, the instruction to load the tensor to be processed can also be to perform a reverse operation (Store operation), that is, to move or write the tensor to be processed from high-speed memory to a large-capacity memory; or, the instruction can also be an instruction to control the data migration of the tensor to be processed between two memories with the same read / write speed (e.g., different global cache levels). In other words, the method of this disclosure can be applied to any scenario involving the transfer of tensor data. The physical specifications, capacity, and access rate type of the original storage space and the target storage space of the tensor to be processed do not limit or affect the state machine operation and specific implementation of the data loading method of this disclosure.

[0071] In step S204, based on the loading instructions, the loading dimension and the preset size of the data block, as well as the starting coordinates of each data loading engine among the multiple data loading engines, are determined.

[0072] The loading instruction for the tensor to be processed can include parameter information of the tensor to determine its shape, size, coordinates, and storage method in its memory space. As mentioned above, the storage method of the tensor to be processed can be a multidimensional linear arrangement format such as NDHWC or N(C / x)DHW(xC). In some examples, when a multidimensional tensor is mapped to a one-dimensional physical memory address, the physical addresses of continuous data in some dimensions will be discontinuous. In other examples, slicing the tensor will cause the physical addresses of continuous data in some dimensions to be discontinuous. Therefore, by determining which innermost continuous dimension (or the fastest changing dimension) of the tensor to be processed is discontinuous or caused by slicing, the loading dimension for this multi-engine parallel operation can be dynamically determined.

[0073] In some embodiments, the loading dimension can be the width dimension (W dimension) of the tensor to be processed. Specifically, in response to the fact that the data of the tensor to be processed is discontinuous in the innermost dimension (e.g., the C dimension, the channel dimension) (e.g., when the requested data block spans a non-contiguous group of channels in memory, or when the C dimension is skipped due to step size settings), since it is not possible to provide a large block of contiguous data in the C dimension to satisfy the single read of a single data loading engine, the hardware logic will determine the width dimension (W dimension) adjacent to the C dimension as the loading dimension to utilize the local address continuity in the W dimension for splitting concurrent requests.

[0074] In some embodiments, the loading dimension can be the height dimension (H dimension) of the tensor to be processed. Specifically, in response to the fact that the data in the inner C dimension and W dimension of the tensor to be processed are discontinuous (for example, when the loaded data is a very narrow feature map boundary region, causing the effective pixels in the W dimension to be insufficient to fill the single read requirement of a single data loading engine), since the frequency of jumping to the H dimension during W dimension loading is too high or the continuous data volume in the W dimension is insufficient to support splitting, continuous loading in the W dimension is not conducive to parallel and efficient loading. Therefore, the hardware logic will further determine the height dimension (H dimension) adjacent to the W dimension as the loading dimension and split the loading requests of multiple data loading engines in the H dimension.

[0075] In some embodiments, each of the multiple data loading engines loads a data block of the same size in parallel each clock cycle. The size of this data block can be a hardware-defined preset size (i.e., a preset data block size), which typically depends on the physical bus width or maximum single memory access capability limit of the underlying individual data loading engine. In other embodiments, the loading operations of the various data loading engines may not occur in the same clock cycle, but the data blocks loaded by each data loading engine in different clock cycles can be used for subsequent parallel computation by the corresponding computing units.

[0076] Furthermore, the number of data loading engines invoked in a single load instruction is determined by a combination of the internal data path bandwidth requirements of the computing unit (e.g., ALU) and the aforementioned predefined data block preset size.

[0077] For example, if a computing unit needs to consume the equivalent of 16 pixels in a single clock cycle to maintain full-load computing, but due to physical hardware limitations, the maximum memory access bandwidth of a single data loading engine can only support loading 8 pixels (i.e., the default data block size is set to 8 pixels), then a single engine cannot meet the throughput requirements of the computing unit. Therefore, hardware logic or instructions will configure two data loading engines (2 × 8 pixels = 16 pixels) to work collaboratively. In this way, by piecing together data blocks acquired collaboratively by different data loading engines, the doubled input bandwidth requirement of the computing unit can be met, preventing the computing unit from becoming idle due to insufficient data bandwidth.

[0078] When the original tensor to be processed is stored in a linear layout, in order to facilitate multiple data loading engines to initiate memory access requests in parallel and efficiently, the preset size of the data block can be determined as a one-dimensional data block. In some embodiments, each data block corresponds to a continuous segment of data in the width dimension (W dimension) of the tensor to be processed (e.g., 8 consecutive pixels).

[0079] In other embodiments, each data block corresponds to a continuous segment of data in the height dimension (H dimension) of the tensor to be processed (e.g., 8 consecutive columns of pixels).

[0080] It is understood that the actual number of bytes occupied by a data block in physical memory depends on the data type of the elements in the tensor to be processed (e.g., data bit width such as INT8, FP16, or FP32) and the maximum length that the underlying storage bus can support for extraction at a time. Therefore, those skilled in the art should understand that the number of pixels contained in a single one-dimensional data block (i.e., the preset size) can be dynamically adjusted or pre-configured according to the specific hardware scenario and tensor data bit width, and this disclosure does not impose strict limitations on this.

[0081] For ease of understanding, the embodiments and illustrations below will be mainly based on an embodiment where the preset size of the data block is the width dimension (W dimension) and contains 8 pixels (i.e., a data block of 8 consecutive pixels in the width dimension of 1×8).

[0082] As mentioned above, the data loading engine can be configured with dedicated address mapping hardware logic. Specifically, the streaming processor cluster in a general-purpose graphics processor can assign starting coordinates to multiple data loading engines configured for the current task by executing instructions. Subsequently, the data loading engine obtains the parameter information of the tensor to be processed (such as the dimensions of each dimension of the tensor, step size, etc.) from the loading instructions, and uses its internal address mapping logic to map the multidimensional coordinates of the tensor to be processed in its multidimensional coordinate system and convert them into a one-dimensional physical address in physical memory, thereby realizing the reading of the underlying data without the main computing core (such as a computing unit) consuming computing power to calculate the physical address.

[0083] In some embodiments, to avoid data overlap or conflict when multiple data loading engines read data in parallel, the first starting coordinate of the first data loading engine and the second starting coordinate of the second data loading engine are separated by a "data block preset size" in the arrangement dimension of the data blocks.

[0084] Typically, regardless of whether the subsequent loading dimension is determined to be the width or height dimension, the starting coordinates assigned to multiple data loading engines are expanded along the arrangement dimension of the data block itself (i.e., the first dimension, such as the width dimension W). Specifically, when the data block is arranged along the width dimension (e.g., a 1×8 pixel block), the starting coordinates between two consecutive data loading engines are exactly separated by one data block preset size (i.e., 8 pixels in the width dimension).

[0085] Figure 3A Exemplary starting coordinate assignments for several data loading engines according to embodiments of this disclosure are shown.

[0086] like Figure 3A As shown, when two data loading engines are configured to work together, the coordinate representation of the tensor to be processed is set as (height, width). If the starting coordinates of the first data loading engine are assigned to (0, 0), then the starting coordinates of the second data loading engine are assigned to (0, 8). Those skilled in the art should understand that the specific height and width of the tensor to be processed in the figure are not limited, and if more data loading engines are configured (e.g., 4), the starting coordinates of the third and fourth engines can be assigned to (0, 16) and (0, 24) respectively.

[0087] In step S206, the first data loading engine and the second data loading engine use the first starting coordinate and the second starting coordinate as the current coordinates of the starting point, respectively, to load a data block with a preset size from the tensor to be processed along the loading dimension direction.

[0088] In some embodiments, a data block includes a predetermined number (e.g., 8) of data elements (e.g., pixels) arranged consecutively along a first dimension (e.g., the width dimension W) of the tensor to be processed.

[0089] It should be noted that, depending on the loading dimension, the preset number of pixels obtained by a single data loading engine when generating a single loading request and the hardware execution logic for determining that the data block has been loaded differ, which depends on the physical continuity of the underlying data in different dimensions.

[0090] The following combination Figure 3B , Figure 3C , Figure 4 and Figure 5 The document describes different implementations when the loading dimension is the width dimension (the first dimension) and when the loading dimension is the height dimension (an adjacent dimension higher than the first dimension).

[0091] Figure 4 An exemplary flowchart of a data loading method 400 according to an embodiment of this disclosure is shown when the loading dimension is the same as the first dimension (i.e., per w split). Figure 4 As shown, method 400 specifically includes steps S402, S404 and S406.

[0092] In step S402, the data loading engine can generate a loading request based on the current coordinates and preset size to obtain the pixel of the tensor to be processed at the corresponding position of the current coordinates.

[0093] In step S404, in response to the generation of a loading request, the coordinate value of the current coordinate in the loading dimension can be incremented by a unit step, and the cumulative number of consecutive loadings can be incremented by one to generate the next loading request.

[0094] In step S406, in response to the number of consecutive loading attempts reaching a preset number, it can be determined that the data block has been loaded and the number of consecutive loading attempts is reset.

[0095] Specifically, when the loading dimension is determined to be the width dimension (W dimension), since the data blocks themselves are arranged along the W dimension, the loading direction of the data loading engine needs to be along the W dimension (i.e., Figure 3BThe tensor moves horizontally (within the horizontal direction). Because the tensor is discontinuous in the inner C dimension and split in the W dimension, each step (1 pixel) in the W dimension may cause its physical memory address to span discontinuous regions. Therefore, the preset quantity of a single load request from a single data loading engine is 1 pixel (i.e., a unit step size of 1). For each data loading engine, it initiates a load request corresponding to 1 pixel for each coordinate movement in the width dimension, and completes the loading of the 1×8 data block by initiating 8 consecutive requests.

[0096] Figure 3B A schematic diagram of data loading is shown when the loading dimension is the width dimension according to an embodiment of the present disclosure.

[0097] like Figure 3B As shown, the first data loading engine and the second data loading engine start from the current coordinates (0, 0) and (0, 8) respectively, and load data in units of 1 pixel, moving to the right along the width dimension (i.e., the direction of the horizontal arrow in the figure).

[0098] In some embodiments, multiple data loading engines can begin loading data from the tensor to be processed in parallel within the same clock cycle. In other embodiments, multiple data loading engines can also begin loading data from the tensor to be processed asynchronously within different clock cycles. Those skilled in the art should understand that the "parallel loading" mentioned in this disclosure does not limit the multiple data loading engines to performing loading actions in the same clock cycle or in a lockstep manner. Each data loading engine has an independent state machine and queue; they are processing the splitting task of the same tensor, but can independently and asynchronously advance their respective data loading progress based on their respective memory access response latency.

[0099] exist Figure 3B In the example shown, due to the asynchronous execution characteristic, when the first data loading engine finishes loading the data at coordinate (0,2), its current coordinate is about to move to (0,3) (as shown). Figure 3B As shown in the first solid box in the first row (position 301), the second data loading engine may have already finished loading the data at coordinates (0, 14), and its current coordinates will move to (0, 15) (as shown in the image). Figure 3B As shown in position 303 within the second solid line box in the first row of the middle text.

[0100] In some embodiments, a hardware counter can be used to record the number of consecutive loads. For example, in response to the generation of a single load request, a hardware timer increments by one. When the counter has incremented from zero to a preset size of the data block (e.g., 8), it is determined that the data block has been loaded. In other embodiments, the hardware counter can also be used to record the number of loads by decrementing the preset size value one by one to zero. Since the preset size of the data block is typically a fixed and small value (e.g., 8 or 16), maintaining a hardware counter with a very small bit width (e.g., 3-bit or 4-bit) is sufficient, resulting in very low overhead in terms of hardware area and power consumption.

[0101] In response to the completion of data block loading (e.g., a counter reaches a preset number), the current coordinates of the data loading engine will trigger a step update (as shown by arrow 307, jumping to...). Figure 3B The new starting point is located at position 305 within the first dashed box in the first row, and the counter is reset to prepare for loading the next data block.

[0102] Figure 5 An exemplary flowchart of a data loading method 500 according to an embodiment of this disclosure is shown when the loading dimension is higher than the first dimension (i.e., per h split). Figure 5 As shown, method 500 specifically includes steps S502, S504 and S506.

[0103] In step S502, the data loading engine can generate a loading request based on the current coordinates and preset size to obtain a preset number of pixels in a row at the position corresponding to the tensor to be processed at the current coordinates.

[0104] In step S504, in response to the generation of a loading request, the coordinate value of the current coordinate in the loading dimension can be incremented by a unit step to generate the next loading request.

[0105] In step S506, in response to the current coordinates exceeding the size of the data block in the loading dimension, it can be determined that the data block has been loaded.

[0106] Specifically, when the loading dimension is determined to be the height dimension (H dimension), it means that the data of the tensor to be processed is not stored contiguously in the first dimension (W dimension), and the engine needs to move along the H dimension (i.e., Figure 3C The data moves vertically within the data block. At this point, for a single request, the corresponding row of W-dimensional data (e.g., 8 consecutive pixels) in the underlying physical memory can be retrieved in a single operation. Therefore, the preset number of data blocks retrieved by a single data loading engine in a single loading request is the preset size of the entire data block (e.g., 8 pixels).

[0107] Figure 3CThis example illustrates data loading with height as the loading dimension. Figure 3C As shown, the first and second data loading engines start from the initial coordinates (0, 0) and (0, 8) respectively, and load data line by line along the height dimension (i.e., in the direction of the downward arrow in the figure), with each request consisting of a 1×8 pixel block. Between two consecutive loading requests, the height dimension coordinate value corresponding to the second loading request is one unit higher than that corresponding to the first loading request (i.e., the step size along the H dimension is 1).

[0108] It should be noted that the hardware implementation for determining the completion of data block loading in the height dimension (H dimension) has been optimized in this disclosure. In some embodiments, for cases where the loading dimension is the height dimension, the aforementioned counter can also be used to record the number of rows issued in consecutive loading requests to determine the timing of coordinate step updates.

[0109] In other embodiments, the height (H dimension) of the image or feature map may be large and dynamically variable (e.g., the height may be thousands or even tens of thousands). If only a counter is used to record the loading progress of the H dimension, a large-bit-width (e.g., 16-bit or 32-bit) dynamic counter will need to be instantiated in the hardware. In a multi-engine concurrent architecture, equipping each engine with a large-bit-width counter will inevitably lead to a significant increase in hardware area and wasted power, placing a heavy burden on the underlying hardware.

[0110] Therefore, in step S506, this disclosure also provides a method that uses coordinate comparison to replace the counter. Since the data loading engine needs to compare the current coordinates with the valid data boundary of the tensor during data loading to determine if an out-of-bounds (OOB) error has occurred and perform padding, this embodiment directly reuses the existing boundary comparator in the hardware to compare the currently loaded H-dimensional coordinates with the size of the tensor to be processed in the H-dimensional dimension (i.e., the bottom boundary). If the current H-coordinates corresponding to the current loading request have reached or exceeded the bottom boundary of the tensor (e.g., the bottom boundary), the method will be used to determine if an out-of-bounds (OOB) error has occurred. Figure 3C When the first data loading engine reaches position 302 of the last row of the first column, or the second data loading engine reaches position 304 of the last row of the second column, it is directly determined that the data block of the current column has been loaded. At this time, the data loading engine will trigger a step update (as shown by the long solid arrow, jumping from position 302 of the last row of the first column to position 306 of the first row of the third column and position 308 of the first row of the fourth column, respectively), to continue loading the next batch of data. This design, which uses a multiplexed comparator instead of a large counter, greatly simplifies the underlying circuitry and achieves high hardware efficiency.

[0111] In step S208, in response to the completion of loading a data block of a preset size, the first data loading engine updates the current coordinates of the starting point by adding a step size in the first dimension, so as to continue loading another data block of a preset size based on the updated current coordinates of the starting point.

[0112] In some embodiments, the stride length is determined based on the number of multiple data loading engines and a preset number (i.e., the preset size of the data block). It should be noted that, since the hardware state machine maintains and moves the first-dimensional coordinates differently under different loading dimensions, the specific calculation logic of the stride length will be adaptively adjusted according to the current coordinate movement state to ensure that the starting point of the next target data block can be aligned after the stride update.

[0113] Specifically, when the loading dimension is the same as the first dimension (e.g., both are width dimension W), since the data loading engine maintains its current coordinates advancing unit by unit (e.g., pixel by pixel) in the width dimension during the loading process, when a data block of a preset size is loaded, these current coordinates have already progressed with the loading process and covered the preset size of the data block. To skip data blocks loaded by other data loading engines, the actual distance (i.e., step size) that the data loading engine needs to traverse in the first dimension is determined to be the product of the number of data loading engines loading in parallel minus one and the preset number.

[0114] For example, in an embodiment configured with two data loading engines and a preset size of 8 pixels, the step size is determined to be (2 - 1) × 8 = 8 pixels. Figure 3B As shown, the current coordinates of the second data loading engine are after it has moved pixel by pixel through the currently loading 8 pixels ( Figure 3B Once the coordinates (0,15) of position 303 in the second solid box of the first row are loaded, add the step size (8 pixels) to its current coordinates (i.e., the next coordinate (0,16) of (0,15) to accurately cross the area that the first data loading engine is responsible for next, and jump to update to the next target starting point (0, 24) (as shown by arrow 307, jump to position 305 in the first dashed box of the first row).

[0115] When the loading dimension is an adjacent dimension higher than the first dimension (e.g., the loading dimension is the height dimension H, and the first dimension is the width dimension W), the data loading engine's loading action proceeds row by row downwards along the height dimension (H dimension). During this vertical loading process, its coordinates in the first dimension (width dimension W) remain at the starting point of that column without shifting. When the data block in that column is fully loaded (e.g., reaching the bottom boundary of the tensor), the distance the engine needs to traverse in the first dimension (i.e., the stride size) in order to switch to the next target data block is determined to be the product of the number of data loading engines and the preset number.

[0116] For example, in the same embodiment with two engines and a preset quantity of 8, the step size is determined to be 2 × 8 = 16 pixels. Figure 3C As shown, when the first data loading engine reaches the bottom boundary of the column (position 302 of the last row of the first column), the current coordinate of its width dimension is still 0. At this time, by adding the step size (16) to the width coordinate, it can jump to the new column top starting point (0, 16) (as shown by the long solid arrow, jump to position 306 of the first row of the third column).

[0117] The step-up update method disclosed herein achieves linear expansion of data bandwidth while avoiding memory access conflicts. By calculating the step size based on the number of data engines and the preset size of fixed-size data blocks, multiple data loading engines form non-overlapping interleaved read trajectories in the tensor in parallel intervals. This method enables the total memory loading bandwidth to increase linearly with the increase of the number of engines, thus filling the bandwidth gap caused by the wide data path of the computing unit.

[0118] Furthermore, embodiments of this disclosure decouple the control logic of multiple engines, reducing the complexity of the hardware state machine. In concurrent architectures of related technologies, multiple readers often need to communicate their reading progress through complex arbitrators or frequent handshake signals to prevent conflicts. However, in the embodiments of this disclosure, after each data loading engine is assigned the initial misaligned coordinates, it only needs to independently calculate and add a fixed step size based on its own loading progress. True asynchronous autonomy is achieved between the engines, greatly simplifying the underlying hardware circuit design and reducing chip power consumption.

[0119] Furthermore, this disclosure also implements dimension-adaptive coordinate pointer maintenance. The hardware state machine can adaptively identify whether the current coordinate pointer is in a state of continuous loading or requires a step transition based on the current splitting strategy (different loading dimensions), and adaptively select (N-1)×size or N×size as the step size. This unified and flexible pointer operation logic ensures that when facing complex and ever-changing tensor memory arrangements, the data loading engine can switch loading strategies between different dimensions while guaranteeing the accuracy of the loading coordinates.

[0120] In applications such as deep learning, tensors to be processed often have arbitrary shapes and sizes. These dimensions may not be divisible by the preset size of the data block (e.g., 8 pixels) in the first dimension (e.g., the width dimension W). Furthermore, to meet the alignment and edge processing requirements of specific operators such as convolution, the effective data region of the tensor to be processed usually includes a padding region. Therefore, the boundary of a tensor can often be divided into an internal "effective data boundary" and an external "padding boundary".

[0121] In combination with the above Figure 3B and Figure 3C The example shown illustrates this if the data loading engine uses... Figure 3B Using position 305 within the first dashed box of the first row or position 308 in the first row of the fourth column as the new starting coordinates, when attempting to load a data block of a full preset size, the requested coordinate region can easily cross the valid data boundary of the tensor to be processed. Therefore, the underlying hardware must introduce an efficient boundary handling method to adaptively handle out-of-bounds (OOB) requests, thereby correctly acquiring valid data or writing padding data.

[0122] The continuity of the underlying physical data across different dimensions determines the preset quantity acquired in a single load request. This difference leads to variations in how the hardware state machine handles out-of-bounds situations under different loading dimensions.

[0123] As mentioned above, when the loading dimension is width, the data loading engine proceeds pixel by pixel, and each loading request corresponds to only one data element (e.g., one pixel). Therefore, for a single loading request, the one pixel it acquires is either completely within the valid boundary or completely outside the valid boundary; there is no situation where the data of a single request is cut in the middle by the boundary.

[0124] However, as the data loading engine progresses pixel by pixel, it is highly likely to encounter the valid data boundary of the tensor. At this point, the hardware does not need to perform complex request splitting, but directly triggers dimensional wraparound and jump, that is, updates the coordinates in the height dimension, and continues to load the remaining pixels from the starting position of the next row to fill the preset size of the current data block.

[0125] Figure 6 A schematic diagram illustrating boundary processing and coordinate jump when the loading dimension is the width dimension according to an embodiment of the present disclosure is shown.

[0126] exist Figure 6 In the example shown, after loading a complete data block (containing 8 pixels) represented by the first solid box 601 in the first row, the first data loading engine jumps to the new starting point indicated by arrow 603 by adding a step size. Subsequently, the engine continues loading to the right in units of 1 pixel. When it has loaded 4 pixels and reached the end position 605 of the second solid box in the first row, its current coordinates have touched the valid right boundary of the tensor to be processed in that width dimension.

[0127] At this point, the engine's preset size (8 pixels) for the current data block has not yet been met, with 4 pixels remaining to be loaded. In response to this boundary event, the data loading engine increments its height coordinates by one unit (i.e., moves to the next row), while resetting its width coordinates to the starting effective offset position of the next row. As shown by the downward arrow in the height direction, the current coordinates jump back to the starting position 607 of the third solid-line box in the second row, and continue loading the remaining 4 pixels pixel by pixel to the right, thus ensuring the continuous loading of a data block.

[0128] In scenarios with concurrent loading by multiple engines, line breaks and wrapping in the aforementioned dimensions may lead to coordinate conflicts among the engines. Due to the misalignment of the current coordinates of each data loading engine, when one engine touches the boundary and performs a line break during loading (such as from the end position 605 of the second solid box in the first row to the beginning position 607 of the third solid box in the second row), the theoretical new coordinates calculated by other parallel data loading engines after adding the step size when performing coordinate jumps may completely exceed the valid right boundary of the tensor to be processed. This may cause multiple engines to overlap in reading or addressing conflicts in the next row.

[0129] To ensure conflict-free interleaving loading by multiple data loading engines after line breaks, some embodiments provide a coordinate wraparound method based on overflow calculation. Specifically, when any data loading engine updates its coordinates, if its theoretical target coordinates exceed the boundary (i.e., current coordinates + stride size > right boundary of valid data), the hardware state machine automatically calculates the pixel overflow amount of the coordinate update action in the current line. For example, this overflow amount can be calculated using the following formula: Overflow = Current coordinates + Step size - Right boundary of valid data.

[0130] Subsequently, the data loading engine increments its height dimension coordinates (i.e., jumps to the next line) and resets its width dimension's new starting coordinates to: New width coordinates = Left boundary of valid data + Overflow amount.

[0131] Since the loading of a single data element only occurs within the valid data range of the tensor and outside the valid data range, the underlying state machine only needs two states, S0 and S1, to describe and handle out-of-bounds data loading situations. That is, the S0 state within the boundary and the S1 state outside the boundary.

[0132] The following combination Figure 6 The embodiments in the text will be explained in a more intuitive way.

[0133] For example, if a second data loading engine exists, and the right boundary of the valid data of the tensor to be processed is set to a width coordinate of 27 (i.e., 28 valid pixels per row, with a valid left boundary of 0), when the second data loading engine is loading a whole data block after the starting coordinate (0, 16) shown by arrow 603 (i.e., loading to coordinate (0, 23)) in a pixel-by-pixel manner, according to the aforementioned step size of (number of engines - 1) × preset data block size = (2-1) × 8 = 8, the theoretical target width coordinate of the next loading request of the second data loading engine should step to 23 + 8 = 31.

[0134] At this point, the theoretical coordinate 31 exceeds the right boundary of the valid data 27. The hardware state machine determines to jump from state S0 to state S1. The state transition triggers overflow calculation, and the overflow amount is found to be 31 - 27 = 4. Therefore, the width coordinate corresponding to the next loading request is reset to the valid left boundary 0 plus the overflow amount 4, i.e., 0 + 4 = 4. At the same time, the height coordinate is incremented by 1, meaning that the final step update coordinate of the second data loading engine falls back to (1, 4). At this point, the state machine can jump from state S1 back to state S0, and the updated coordinates of the second data loading engine connect with the position where the first data loading engine ends loading at (1, 3), achieving conflict-free interleaving after multi-engine line breaks.

[0135] The above-described coordinate wrapping calculation method using the width dimension as the loading dimension is merely an illustrative example. Those skilled in the art should understand that the method for calculating wrapping coordinates generated by advancing pixel by pixel is not limited to the simple arithmetic logic based on addition and subtraction. In actual hardware implementations, the wrapping calculation can also be implemented using various equivalent hardware structures such as modulo operations, shift register configuration, lookup table mapping, or conditional jump logic based on state machines. Furthermore, the effective data boundary is not limited to the effective data boundary or padding boundary of a tensor; it can also be a virtual clipping boundary dynamically configured by upper-level software through instructions. All substitutions and variations of the above embodiments do not depart from the core inventive concept of this disclosure.

[0136] However, although the above-mentioned line-wrapping logic in the width dimension can resolve coordinate conflicts, if the continuous data of the tensor to be processed in the width dimension is short, the frequent boundary touch, foldback and overflow calculations in the width dimension will significantly increase the computational and power consumption burden of the underlying hardware state machine.

[0137] Therefore, as discussed above, when faced with situations where the continuous data in the width dimension is too short, the multi-engine concurrent loading strategy will more adaptively yield and switch to loading vertically downwards along the height dimension (H dimension). The following will elaborate on the multi-engine request splitting and boundary handling methods based on the data block dimension when using the height dimension as the loading dimension.

[0138] When the loading dimension is height, a single loading request contains a whole row of pixels of a preset size. Therefore, the data block corresponding to the current coordinates is very likely to cross the valid data boundary. To address this situation, this embodiment dynamically splits the original loading request into multiple loading sub-requests based on the relative relationship between the current coordinates and the valid data boundary of the tensor to be processed. This ensures that the data requested by each loading sub-request either belongs entirely to the valid data range of the tensor to be processed or does not belong to the valid data range of the tensor to be processed at all.

[0139] Figure 7 An exemplary flowchart of a boundary handling method 700 according to an embodiment of this disclosure when the loading dimension is the height dimension is shown. Figure 7 As shown, method 700 specifically includes different boundary processing scenarios for steps S702 to S710. The actual boundary processing scenario can be determined and the corresponding operation performed by executing method 700. It is understood that the order of steps shown in method 700 is not limited to this and can be adjusted according to the actual situation.

[0140] In step S702, in response to the fact that the data block of the preset size corresponding to the current coordinate is completely outside the valid left boundary of the tensor to be processed, a loading sub-request is generated to obtain the filling data of the tensor to be processed.

[0141] In step S704, in response to the data block of the preset size corresponding to the current coordinate crossing the left boundary of the valid data (i.e. part in the left fill area and part in the valid area), two loading sub-requests are generated. One loading sub-request is used to obtain the fill data, and the other loading sub-request is used to obtain data from the valid data range.

[0142] In step S706, in response to the fact that the data block of the preset size corresponding to the current coordinate is completely located between the left boundary and the right boundary of the valid data, a loading sub-request is generated to obtain data completely from the valid data range.

[0143] In step S708, in response to the data block of the preset size corresponding to the current coordinate crossing the right boundary of the valid data (i.e. part in the valid area and part in the right fill area), two loading sub-requests are generated. One loading sub-request is used to obtain data from the valid data range, and the other loading sub-request is used to obtain fill data.

[0144] In step S710, in response to the fact that the data block of the preset size corresponding to the current coordinate is completely outside the right boundary of the valid data, a loading sub-request is generated to obtain the filling data.

[0145] In response to a load sub-request requesting data that is not within the valid data range of the tensor to be processed, the data loading engine writes a preset constant value as padding data to the processor's target buffer. For the aforementioned load sub-request to obtain padding data, the data loading engine does not need to initiate a real physical read transaction to external mass storage; instead, the hardware directly writes the preset constant value (e.g., constant 0 or NaN in a specific format) into the processor's target buffer. This padding method avoids unnecessary read and write operations, thus saving memory bus bandwidth.

[0146] To efficiently implement the above-mentioned splitting logic in hardware circuits, this disclosure introduces a state machine mechanism. When the height dimension is the loading dimension, the state machine is divided into three main states: S2, S3, and S4 by comparing the current coordinates of the data loading engine in the width dimension with the valid data boundary of the tensor to be processed. (It should be noted that, for ease of distinction, S0 and S1 are used for the case of width dimension priority loading described above.)

[0147] Figure 8 A schematic diagram illustrating the mapping between state machine states and data regions when the loading dimension is the height dimension according to an embodiment of this disclosure is shown.

[0148] like Figure 8 As shown, taking the horizontal direction as the width dimension (W dimension) and the vertical direction as the height dimension (H dimension) as an example, the tensor to be processed has two valid data boundaries on the left and right, as well as two padding data boundaries on the left and right sides. By comparing the width dimension coordinate value (w_coord) of the current coordinate with the boundaries of the tensor's valid data (for example, setting the left boundary to 0 and the right boundary to tensor_w), the hardware state of the current coordinate can be accurately determined.

[0149] Table 1 below shows examples of state machine state conditions when the loading dimension is the height dimension according to embodiments of this disclosure:

[0150] Table 1 Given the current hardware state (S2, S3, or S4) at which the coordinates are located, the hardware logic can determine with extremely low latency whether the current loading request needs to be split into sub-requests based on the specific value of the coordinates and the preset size of the data block.

[0151] Figure 9 The diagram illustrates a splitting scenario where, according to an embodiment of this disclosure, the loading dimension is the height dimension, and the loading request crosses the right boundary of the valid data (i.e., triggers step S708 as described above).

[0152] Figure 9 Each grid cell in the image represents 4 pixels in the width dimension. It is understandable that, although... Figure 9 The diagram shows a split in 4-pixel units, but the splitting method is not limited to this. Those skilled in the art can split the data block to be loaded into any suitable size based on the actual situation and state conditions. When the current coordinates of the data loading engine are in state S3 (within the valid data area) and a "Load Request 1" is initiated, the request falls completely into the valid area without splitting.

[0153] As the multi-engine stepping method progresses, when an engine jumps and initiates "Load Request 2", it is assumed that it requests to load a data block containing 8 pixels. At this time, the first 4 pixels of the request are within the right boundary of the valid data, while the last 4 pixels cross the right boundary and fall into the fill area of ​​state S4.

[0154] In this scenario, the hardware state machine will split "Load Request 2" into two sub-requests: Sub-request 1 has a size of 4 pixels and is used to initiate a real read transaction to the memory storing the tensor to be processed, and to obtain 4 pixels within the right boundary of the valid data.

[0155] Sub-request 2 requests a size of 4 pixels. Instead of initiating a memory read, it directly writes a preset padding value (such as 0) to the corresponding address in the target memory.

[0156] The following, in conjunction with Table 2, elaborates on the different judgment conditions and execution actions under different states.

[0157]

[0158] Table 2 In the examples in Table 2, for ease of understanding, Table 2 shows the case where the step size is 8 pixels and the preset size of the data block is 8 pixels when updating coordinates by step. It is important to note that the examples in Table 2 also include the constraint that the width of the tensor to be processed cannot be less than the preset size of a data block, i.e., tensor_w >= 8.

[0159] In S2 state, the current coordinates are out of bounds (left of the left boundary of the valid tensor data). At this point, the absolute value of the current width dimension coordinate is checked. If the absolute value is greater than or equal to the preset size of a data block, it means that the current coordinates are more than the preset size of a data block away from the left boundary of the valid data. In this case, a single entry of fill data can be written, and the coordinates can be updated.

[0160] Similarly, in S2 state, if the absolute value is less than the preset size of a data block, it means that the next load request will have some data outside the left boundary of the valid data and some within the valid data range. In this case, the load request can be split into two sub-requests. The specific splitting process will not be elaborated here. After the sub-instructions are loaded, the coordinates can continue to be updated.

[0161] If the state machine is in state S3, then the coordinates in the width dimension plus the preset size of the data block are directly compared with the effective data width of the tensor. Based on this comparison, it can be determined whether the data of the next loading request is entirely within the valid range or will exceed the right boundary. If it exceeds the right boundary, i.e., the current coordinates plus the preset size exceed the effective data width of the tensor, then the request needs to be split into sub-requests.

[0162] In state S4, the valid right boundary of the tensor has been exceeded. If the coordinate value of the width dimension plus the preset size of the data block is less than or equal to the right padding boundary (bound_w) of the tensor, it means that padding data of the preset size can be written directly. If the right padding boundary has been exceeded, the state should return to S2 or S3. At this point, the state may have entered the left padding boundary, the valid data boundary, or the left padding boundary of the tensor to be processed.

[0163] The aforementioned state machine and request splitting method bring significant technical advantages to the underlying data handling. Because the engine progresses vertically during height-dimensional (H-dimension) loading, its width-dimensional (W-dimension) coordinates remain unchanged within the same column. Therefore, the conditional judgment for determining whether to split requests is extremely efficient. Unlike width-dimensional (W-dimension) loading, the hardware logic does not need to determine whether cross-row wrapping or perform complex overflow calculations after each request. In height-dimensional loading mode, the underlying hardware only needs to calculate the splitting ratio (i.e., the truncation position of effective pixels and fill pixels) for all requests in that column at once, based on the initial state of the current request (S2, S3, or S4) and the absolute distance of the current coordinates from the boundary. This arithmetic-level splitting and constant padding method greatly accelerates the loading speed of edge data and reduces the pressure on the memory controller.

[0164] Regardless of which dimension (width or height) the underlying hardware chooses as the loading dimension, multiple data loading engines can efficiently and accurately complete their respective data loading through dynamic coordinate allocation and boundary state machine mechanisms.

[0165] In this process, the stride step is not merely a simple arithmetic increment; it is essentially a physical mapping at the hardware level of the topological relationship between multi-engine concurrency and tensor data arrangement. By associating the stride step with the number of data loading engines and the preset size of the data blocks, the underlying hardware spatially decouples a massive tensor loading task into multiple orthogonal subtasks. Each data loading engine only needs to update its coordinates based on its local stride step, enabling interleaved loading and conflict-free addressing of multiple data loading engines, thus avoiding the high arbitration and synchronization overhead of multi-engine concurrency.

[0166] The data loading method disclosed herein dynamically splits linear loading instructions into starting coordinates of data blocks adapted to multiple data loading engines, and relies on a stepping method to update and drive the interleaved advancement of multiple engines. Simultaneously, for each segmented data block, especially when it reaches the valid data boundary of a tensor, it is further split into continuous physical memory requests through an underlying hardware state machine, thereby completing the in-situ filling of out-of-bounds data.

[0167] Using the above method, the embodiments of this disclosure effectively overcome the deep dependence on frequent software loading instructions and software reordering instructions in related technical solutions. The splitting of sub-requests, the coordinate step-by-step update of the parallel data loading engine, and the boundary trimming process are all automatically closed-loop by the data loading engine hardware logic independent of the main computing core, realizing the efficient conversion of tensor data from large-capacity linear layout storage to high-speed cache block layout storage without additional software intervention.

[0168] This not only frees up the control computing power of general-purpose computing units (such as streaming processor clusters), but also matches the huge input data path bandwidth requirements of the arithmetic logic unit in the computing core by pooling the bandwidth after multiple sets of data loading engines fetch data in parallel, thus multiplying the bus utilization and significantly improving the overall throughput performance of general-purpose graphics processors when dealing with complex artificial intelligence and deep learning computing.

[0169] Figure 10 An exemplary block diagram of a data loading apparatus according to an embodiment of the present disclosure is shown. It can be utilized... Figure 10 The system shown is used to execute Figure 2 Method 200 is described.

[0170] like Figure 10 As shown, in some embodiments, the data loading device 1000 includes an instruction acquisition module 1002, an instruction parsing module 1004, a data loading module 1006, and a coordinate update module 1008. In some embodiments, the functions of the instruction acquisition module 1002 and the instruction parsing module 1004 can be combined and implemented by a single integrated instruction parsing unit, and the functions of the data loading module 1006 and the coordinate update module 1008 can be combined and implemented by a single integrated instruction parsing unit.

[0171] The instruction acquisition module 1002 is used to acquire the loading instructions of the tensor to be processed.

[0172] The instruction parsing module 1004 is used to determine the loading dimension and the preset size of the data block based on the loading instruction, as well as the starting coordinates of each of the multiple data loading engines, wherein the first starting coordinate of the first data loading engine and the second starting coordinate of the second data loading engine are spaced apart by the preset size of the data block.

[0173] The data loading module 1006 includes multiple data loading engines, which are configured to load data blocks of a preset size from the tensor to be processed along the loading dimension direction, respectively, using the first starting coordinate and the second starting coordinate as the current coordinates. The data blocks include a preset number of data elements arranged continuously in the first dimension of the tensor to be processed. The coordinate update module is used to respond to the completion of loading a data block of a preset size. It controls the corresponding data loading engine to update the current coordinates of the starting point in the first dimension by adding a step size, so as to continue loading another data block of a preset size based on the updated current coordinates of the starting point. The step size is determined based on the number of multiple data loading engines and the preset number.

[0174] Among them, the loading dimension is the same as the first dimension, or the loading dimension is an adjacent dimension that is higher than the first dimension.

[0175] It should be understood that Figure 10 The various modules or units of the apparatus 1000 shown can be used with reference to Figure 2 , Figure 4 , Figure 5 and Figure 7 The steps described in methods 200, 400, 500, and 700 correspond to each other. Therefore, the operations, features, and advantages described above for these methods also apply to apparatus 1000 and its included modules and units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0176] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0177] According to one aspect of this disclosure, a processor is also provided. Figure 11 An exemplary block diagram of a processor according to an embodiment of the present disclosure is shown. Figure 11 The processor 1100 may include an instruction parsing unit 1102 and an execution unit 1104.

[0178] The instruction parsing unit 1102 can be used to receive and parse data loading instructions, wherein the data loading instructions instruct the execution of loading the tensor to be processed from memory into the buffer.

[0179] The execution unit 1104 can be used to execute the data loading instruction.

[0180] The execution unit executes data loading instructions, including implementing the data loading method described above to load the data of the tensor to be processed from memory through multiple data loading engines.

[0181] In some embodiments, execution unit 1104 may be implemented as a direct memory access controller, a load / store unit, or a dedicated tensor access unit. Internally, execution unit 1104 may have hard-wired hardware units implementing the state machine of this disclosure.

[0182] It is understood that the processor 1100 can be implemented to perform the data loading method according to embodiments of the present disclosure. For details regarding the specific implementation of the data loading method, please refer to the description above, which will not be repeated here.

[0183] According to one aspect of this disclosure, an electronic device is also provided. Figure 12 An exemplary block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 12 As shown, the electronic device 1200 includes a memory 1220, a processor 1240, and a computer program stored in the memory. The memory 1220 and the processor 1240 may be coupled via an internal bus or other communication mechanism. The processor 1240 is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0184] In some embodiments, the processor 1240 may employ, for example... Figure 1 or Figure 11 The processor's microarchitecture design is shown.

[0185] Figure 13 A block diagram of an example computer device 2000 according to an embodiment of the present disclosure is shown.

[0186] The computer device 2000 may specifically correspond to the electronic device or processing system in the foregoing embodiments. It should be noted that... Figure 13 The components of the computer device 2000 shown are merely exemplary and not restrictive. Depending on the actual application requirements, the computer device 2000 may have other components or omit some components.

[0187] like Figure 13As shown, computer device 2000 includes processing device 2010. Processing device 2010 can perform various appropriate actions and processes according to computer-executable instructions stored in read-only memory (ROM) 2020 or loaded from storage device 2080 into random access memory (RAM) 2030. For example, processing device 2010 can be configured to execute the above-referenced... Figures 2 to 7 The described data loading or data storage method. Various programs and data required for the operation of the computer device 2000 can also be temporarily stored in RAM 2030. The processing device 2010, ROM 2020, and RAM 2030 are interconnected and exchange data via bus 2040. Input / output (I / O) interface 2050 is also connected to bus 2040.

[0188] In some embodiments, the processing device 2010 may include a central processing unit (CPU), a graphics processing unit (GPU), a tensor processor (TPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In specific implementations of embodiments of this disclosure, the processing device 2010 may have a built-in or coupled data loading unit and hardware state machine logic as described above to directly process tensor block loading tasks from mass storage (such as system memory) to on-chip cache (such as local cache) at the underlying level.

[0189] Typically, the following devices can be connected to the I / O interface 2050: input devices 2060 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 2070 including, for example, liquid crystal displays (LCDs), light-emitting diode displays (LEDs), speakers, vibrators, etc.; storage devices 2080 including, for example, magnetic tapes, hard disks, solid-state drives (SSDs), flash memory cards, etc.; and communication devices 2090.

[0190] The storage device 2080 can serve as a high-capacity, non-volatile storage medium for persistently storing the operating system, applications, and raw tensor data to be processed, such as large-scale neural network weights and massive input feature maps. The communication device 2090 allows the computer device 2000 to exchange data with other devices (such as cloud servers, other computing nodes in a distributed computing cluster, etc.) via a network, either wirelessly or via wired communication. The communication device 2090 may include a local area network (LAN) card, a wireless local area network (WLAN) card, a Bluetooth module, a near field communication (NFC) module, a cellular communication module, etc.

[0191] Although Figure 13A computer device 2000 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the computer device 2000 may alternatively implement or possess more or fewer devices. For example, when the computer device 2000 is used as an AI computing accelerator card or server node in a data center, it may not include input devices 2060 and output devices 2070 for human-computer interaction such as touch screens and keyboards, but may mainly consist of a processing unit 2010, high-bandwidth memory, and high-speed communication interfaces.

[0192] The functions of the systems, apparatuses, modules, or units described above herein can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on. Furthermore, the methods described in this disclosure can also be implemented by software or microcode containing computer-executable instructions that, when executed by the processing device 2010, cause the machine to perform the operations defined in this disclosure.

[0193] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided.

[0194] Figure 14 An exemplary block diagram of a computer-readable storage medium according to embodiments of the present disclosure is shown. Figure 14 As shown, a computer-readable storage medium 3000 stores computer-executable instructions 3010, which, when executed by a processor, implement the steps of any of the method embodiments described above.

[0195] In some embodiments, the computer-readable storage medium 3000 may include any combination of one or more computer program products. For example, without limitation, the non-transitory computer-readable storage medium may include volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0196] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0197] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of this disclosure is not limited by these embodiments or examples, but only by the claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A data loading method, characterized in that, The method is applied to a processor that includes multiple data loading engines, and the method includes: Obtain the loading instructions for the tensor to be processed; Based on the loading instruction, the loading dimension and the preset size of the data block are determined, as well as the starting coordinates of each of the plurality of data loading engines, wherein the first starting coordinate of the first data loading engine and the second starting coordinate of the second data loading engine are spaced apart by the preset size of the data block. The first data loading engine and the second data loading engine use the first starting coordinate and the second starting coordinate as the current coordinates of the starting point, respectively, to load a data block with the preset size from the tensor to be processed along the loading dimension direction. The data block includes a preset number of data elements arranged continuously in the first dimension of the tensor to be processed. In response to the completion of loading of the data block of the preset size, the first data loading engine updates the current coordinates of the starting point by adding a step size in the first dimension, so as to continue loading another data block of the preset size based on the updated current coordinates of the starting point. The step size is determined based on the number of the plurality of data loading engines and the preset number. Wherein, the loading dimension is the same as the first dimension, or the loading dimension is an adjacent dimension that is higher than the first dimension.

2. The data loading method according to claim 1, characterized in that, The data of the tensor to be processed is stored contiguously in the first dimension, and the loading dimension is the same as the first dimension. Loading a data block of the preset size from the tensor to be processed along the loading dimension direction includes: The data loading engine generates a loading request based on the current coordinates and the preset size to obtain the pixel of the tensor to be processed at the position corresponding to the current coordinates; In response to the generation of the loading request, the coordinate value of the current coordinates in the loading dimension is incremented by a unit step, and the cumulative number of consecutive loadings is incremented by one to generate the next loading request. In response to the consecutive loading count reaching a preset number, it is determined that the data block has been loaded, and the consecutive loading count is reset.

3. The data loading method according to claim 1, characterized in that, The data of the tensor to be processed is stored non-contiguously in the first dimension, and the loading dimension is an adjacent dimension higher than the first dimension. Loading a data block of the preset size from the tensor to be processed along the loading dimension direction includes: The data loading engine generates a loading request based on the current coordinates and the preset size to obtain a preset number of pixels in a row of the tensor to be processed at the position corresponding to the current coordinates. In response to the generation of the loading request, the coordinate value of the current coordinates in the loading dimension is incremented by a unit step to generate the next loading request; In response to the current coordinates exceeding the size of the data block in the loading dimension, it is determined that the data block has been loaded.

4. The data loading method according to claim 3, characterized in that, The first dimension is the width dimension of the tensor to be processed, and the loading dimension is the height dimension of the tensor to be processed.

5. The data loading method according to claim 4, characterized in that, The method further includes: based on the relative relationship between the current coordinates and the effective data boundary of the tensor to be processed and the preset size of the data block, splitting the loading request into multiple loading sub-requests, so that the data requested to be loaded by each loading sub-request is either entirely within the effective data range of the tensor to be processed, or none of them are within the effective data range of the tensor to be processed.

6. The data loading method according to claim 5, characterized in that, The step of splitting the loading request into multiple loading sub-requests based on the relative relationship between the current coordinates and the effective data boundary of the tensor to be processed, and the preset size of the data block, includes: In response to the fact that the data of the preset size corresponding to the current coordinate is completely outside the left boundary of the valid data of the tensor to be processed, a loading sub-request is generated to obtain the filling data of the tensor to be processed. In response to the data of a preset size corresponding to the current coordinates crossing the left boundary of the valid data, two loading sub-requests are generated. One loading sub-request is used to obtain the fill data, and the other loading sub-request is used to obtain data from the valid data range. In response to the fact that the data of the preset size corresponding to the current coordinates is completely located between the left boundary and the right boundary of the valid data, a loading sub-request is generated to obtain data from the valid data range; In response to the data of a preset size corresponding to the current coordinates crossing the right boundary of the valid data, two loading sub-requests are generated. One loading sub-request is used to obtain data from the valid data range, and the other loading sub-request is used to obtain the fill data. In response to the fact that the data of the preset size corresponding to the current coordinates is completely outside the right boundary of the valid data, a loading sub-request is generated to obtain the filling data.

7. The data loading method according to claim 5, characterized in that, In response to the fact that the data requested by the loading sub-request does not belong to the valid data range of the tensor to be processed, the data loading engine writes a preset constant value to the target cache of the processor as fill data.

8. The data loading method according to claim 2, characterized in that, The number of consecutive loads is recorded using a hardware counter.

9. The data loading method according to claim 1, characterized in that, The multiple data loading engines begin loading data from the tensor to be processed in parallel at the same clock cycle.

10. The data loading method according to claim 1, characterized in that, The multiple data loading engines asynchronously load data from the tensor to be processed at different clock cycles.

11. The data loading method according to claim 1, characterized in that, The processor also includes a computing unit. The number of the plurality of data loading engines is determined based on the data channel bandwidth of the computing unit and the preset size of the data block, so that the data acquired in parallel by the plurality of data loading engines is assembled to meet the input bandwidth requirements of the computing unit.

12. A data loading device, characterized in that, The device includes: The instruction acquisition module is configured to acquire the loading instructions for the tensor to be processed; The instruction parsing module is configured to determine the loading dimension and the preset size of the data block, as well as the starting coordinates of each data loading engine among multiple data loading engines, based on the loading instruction, wherein the first starting coordinate of the first data loading engine and the second starting coordinate of the second data loading engine are spaced apart by the preset size of the data block. The data loading module is configured to include the plurality of data loading engines, wherein the plurality of data loading engines are configured to load data blocks having the preset size from the tensor to be processed along the loading dimension direction, respectively using the first starting coordinate and the second starting coordinate as the current coordinates of the starting point; wherein the data block includes a preset number of data elements arranged continuously in the first dimension of the tensor to be processed. The coordinate update module is configured to, in response to the completion of loading a data block of the preset size, control the corresponding data loading engine to update the current coordinates of the starting point in the first dimension by adding a step size, so as to continue loading another data block of the preset size based on the updated current coordinates of the starting point. The step size is determined based on the number of the plurality of data loading engines and the preset number. Wherein, the loading dimension is the same as the first dimension, or the loading dimension is an adjacent dimension that is higher than the first dimension.

13. A processor, characterized in that, The processor includes: An instruction parsing unit is configured to receive and parse a data loading instruction, wherein the data loading instruction instructs the execution of loading a tensor to be processed from memory into a buffer; and The execution unit is configured to execute the data loading instruction. The execution unit executes the data loading instruction, including implementing the data loading method according to any one of claims 1-11, to load the data of the tensor to be processed from the memory through a plurality of data loading engines.

14. An electronic device, characterized in that, The electronic device includes: The processor according to claim 13; and the memory connected to the processor, The processor is configured to run computer-executable instructions, which, when executed by the processor, implement the data loading method according to any one of claims 1-11.

15. A non-transitory computer-readable storage medium, characterized in that, It stores computer-executable instructions, wherein when the computer-executable instructions are executed by a processor, they implement the data loading method according to any one of claims 1-11.

Images