Data loading method and device, processor, electronic equipment and medium

Abstract

The present disclosure provides a data loading method, device, processor, electronic device and storage medium. The method comprises: receiving a data loading instruction for loading a to-be-processed scaling factor to a cache area, to determine a base address for writing the to-be-processed scaling factor to the cache area; determining, based on a data storage format of an original scaling factor tensor in a memory, a pixel unit loaded by each request of a plurality of requests for loading the to-be-processed scaling factor, wherein the plurality of requests are used to sequentially load pixel units in the original scaling factor tensor in a first dimension and write the pixel units to the cache area in a data block, and a size of the pixel units in the first dimension is the same as a size of the data block in the first dimension; determining, based on the base address and a data size of the pixel unit, a target write address in the cache area of each request of the plurality of requests; and sequentially sending the plurality of requests, and writing data corresponding to each request to the cache area based on the target write address corresponding to the request.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and specifically to a data loading method, apparatus, processor, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence and deep learning technologies, neural network models are widely used in various computing tasks. In these models, data is typically represented and computed in the form of tensors. Tensors can generally be viewed as a generalization of scalars, vectors, and matrices to higher dimensions; essentially, they are multidimensional arrays used to hold input data, weight parameters, feature maps, and various auxiliary computational data during model training and inference. To improve the execution efficiency of computing hardware (such as general-purpose graphics processing units (GPGPUs) and tensor processors), reduce memory access bandwidth usage, and lower system power consumption, the industry widely adopts model quantization techniques. Quantization techniques typically convert high-precision floating-point numbers (such as FP32 and FP16) into low-precision data formats (such as INT8, FP8, or FP4) to perform core tensor operations such as matrix multiplication.

[0003] In quantization or dequantization, a scaling factor is typically introduced to reduce the bit width while preserving as much of the original data's expressive power and computational precision as possible. The scaling factor is used to proportionally scale and adjust the numerical range of data during format conversion. In practical general-purpose computing, the scaling factor is also stored in the memory as a tensor. When performing low-precision tensor operations, the computing unit needs to simultaneously load a large amount of scaling factor data from memory to complete the calculation. Summary of the Invention

[0004] This disclosure provides a data loading method, apparatus, processor, electronic device, and storage medium for loading a scaling factor to be processed from a memory storing the original scaling factor tensor to a target memory, and storing it in the target memory in units of data blocks of a preset size. This improves memory access bandwidth when loading the scaling factor to be processed into the target memory, effectively enhancing loading performance and hardware computing utilization.

[0005] According to one aspect of this disclosure, a data loading method is provided for loading a scaling factor to be processed from an original scaling factor tensor in memory into a cache, wherein the scaling factor to be processed is stored in the cache in units of data blocks, each data block having a preset size, and data within the same data block being stored contiguously in the cache. The method includes: receiving a data loading instruction for loading the scaling factor to be processed into the cache, thereby determining a base address for writing the scaling factor to be processed into the cache; determining, based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded by each of a plurality of requests for loading the scaling factor to be processed, wherein the plurality of requests are used to sequentially load the pixel units in the original scaling factor tensor in a first dimension and write them into the cache in units of data blocks, wherein the size of the pixel unit in the first dimension is the same as the size of the data block in the first dimension; determining a target write address in the cache for each of the plurality of requests based on the base address and the data size of the pixel unit; and sequentially sending the plurality of requests, writing the data corresponding to each request into the cache based on its corresponding target write address.

[0006] In some embodiments, determining the pixel unit loaded in each of a plurality of requests for loading the scaling factor to be processed, based on the data storage format of the original scaling factor tensor in memory, includes: determining that the pixel unit includes the same number of consecutive data elements in a first dimension as the data block in a first dimension, in response to the data storage format of the original scaling factor tensor in memory being a linear layout; and determining that the pixel unit includes consecutive data elements of the same preset size as the data block, in response to the data storage format of the original scaling factor tensor in memory being a block-based layout in units of data blocks.

[0007] In some embodiments, the data storage format of the original scaling factor tensor is a linear layout. Based on the base address and the data size of the pixel unit, the target write address in the buffer for each request in multiple requests is determined, including: continuously writing the data of the pixel unit loaded by each request into the buffer along the second dimension until the amount of data continuously written in the second dimension is equal to the number of bytes of a data block; wherein the offset between the target write addresses of two consecutive requests is equal to the number of bytes of the data size of the pixel unit; in response to the amount of data continuously written in the second dimension reaching the preset size of the data block in the second dimension, updating the base address based on the total size of the scaling factor to be processed in the first dimension and the preset size of the data block, so that the next consecutive request uses the updated base address as the corresponding target write address.

[0008] In some embodiments, the data storage format of the original scaling factor tensor is a block layout. Based on the base address and the data size loaded in each request, the target write address of each request in the cache is determined, including: determining that the target write addresses corresponding to two consecutive requests are linearly increasing, and the offset between the target write addresses corresponding to two consecutive requests is equal to the number of bytes of a data block.

[0009] In some embodiments, for the same coordinate position of the scaling factor to be processed in the second dimension, the difference between the target write addresses of two adjacent data blocks in the cache along the first dimension is equal to the number of bytes of the data block.

[0010] In some embodiments, the data loading method further includes: in response to the range of the scaling factor to be processed crossing a boundary range defined by a boundary value, using a state machine to generate multiple sub-requests based on the request, such that the data loaded by each sub-request is either entirely within the data range of the original scaling factor tensor defined by the boundary value of the original scaling factor tensor or entirely outside the data range of the original scaling factor tensor defined by the boundary value.

[0011] In some embodiments, for valid sub-requests where all loaded data falls within the data range of the original scaling factor tensor defined by the boundary value: the base address of the target write address of the valid sub-request in the cache is equal to the starting base address of the data loaded by the valid sub-request in the second dimension, plus the amount of valid data before the data loaded by the valid sub-request in the first dimension divided by the preset size of the data block in the first dimension, and then multiplied by the total number of bytes in the data block; and byte offset within the data block is performed based on the pixel unit loaded by the valid sub-request.

[0012] In some embodiments, for invalid sub-requests where all loaded data is not within the data range of the original scaling factor tensor: the invalid sub-request is converted into an operation of writing a preset value to the target write address corresponding to the invalid sub-request, wherein the base address of the target write address of the invalid sub-request in the buffer is equal to the starting base address of the data loaded by the valid sub-request in the second dimension, plus the amount of valid data before the data loaded by the valid sub-request in the first dimension divided by the preset size of the data block in the first dimension, multiplied by the total number of bytes of the data block, and the byte offset within the data block is performed based on the pixel unit loaded by the invalid sub-request.

[0013] In some embodiments, a state machine is used to determine multiple requests for loading the scaling factor to be processed, wherein: in response to the scaling factor to be processed being discontinuous in the first dimension, the state machine determines to divide the requests in the second dimension; in response to the scaling factor to be processed being discontinuous in the second dimension, the state machine determines to divide the requests in the third dimension; wherein the data size loaded by each request is equal to the preset size of the data block, and the data loaded by each sub-request after division is either entirely within the data range of the original scaling factor tensor defined by the boundary value of the original scaling factor tensor or entirely outside the data range of the original scaling factor tensor defined by the boundary value.

[0014] In some embodiments, in response to the state machine determining to adopt the requested partitioning in the second dimension, when the state machine determines the addressing coordinates in memory, each increment of its coordinates in the second dimension is equal to the size of the data elements included in the data block in the second dimension.

[0015] In some embodiments, when the state machine partitions the request in the second dimension, the termination condition for its state loop transition includes: the remaining amount of data to be copied in the third dimension is not equal to 1; or, the remaining amount of data to be copied in the second dimension is not equal to the size of the data elements included in the data block in the second dimension.

[0016] In some embodiments, the original scaling factor tensor and the scaling factor to be processed include a channel dimension, a width dimension, and a batch dimension, wherein the first dimension is the channel dimension, the second dimension is the width dimension, and the third dimension is the batch dimension.

[0017] According to one aspect of this disclosure, a data loading apparatus is provided for loading a scaling factor to be processed from an original scaling factor tensor in memory into a cache, wherein the scaling factor to be processed is stored in the cache in units of data blocks, each data block having a preset size, and data within the same data block being stored contiguously in the cache. The data loading apparatus includes: an instruction acquisition module configured to receive a data loading instruction for loading the scaling factor to be processed into the cache, thereby determining a base address for writing the scaling factor to be processed into the cache; an instruction parsing module configured to determine, based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded by each of a plurality of requests for loading the scaling factor to be processed, wherein the plurality of requests are used to sequentially load the pixel units in the original scaling factor tensor in a first dimension and write them into the cache in units of data blocks, wherein the size of the pixel unit in the first dimension is the same as the size of the data block in the first dimension; a request generation module configured to determine, based on the base address and the data size of the pixel unit, the target write address of each of the plurality of requests in the cache; and a request execution module configured to sequentially send the plurality of requests, writing the data corresponding to each request into the cache based on its corresponding target write address.

[0018] According to one aspect of this disclosure, a processor is provided, comprising: an instruction parsing unit configured to receive and parse a data loading instruction, wherein the data loading instruction instructs execution to load a scaling factor to be processed from memory into a cache; and an execution unit configured to execute the data loading instruction, wherein the execution unit executes the data loading instruction, including implementing the above-described data loading method to store the scaling factor to be processed in a cache in units of data blocks of a preset size.

[0019] According to one aspect of this disclosure, an electronic device is provided, including: the aforementioned processor; and a memory connected to the processor, wherein the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the aforementioned data loading method.

[0020] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein when the computer-executable instructions are executed by a processor, the above-described data loading method is implemented. Attached Figure Description

[0021] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of this disclosure. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0022] Figure 1A schematic diagram of the structure of a general-purpose graphics processor according to an embodiment of the present disclosure is shown.

[0023] Figure 2 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown.

[0024] Figure 3A A schematic diagram of the original scaling factor tensor with a linear storage layout according to an embodiment of the present disclosure is shown.

[0025] Figure 3B A schematic diagram of the original scaling factor tensor being a block storage layout according to an embodiment of the present disclosure is shown.

[0026] Figure 4A A schematic diagram of pixel unit partitioning of the original scaling factor tensor under a linear storage layout according to an embodiment of the present disclosure is shown.

[0027] Figure 4B A schematic diagram of pixel unit partitioning of the original scaling factor tensor in a block storage layout according to an embodiment of the present disclosure is shown.

[0028] Figure 5 A schematic diagram illustrating the mapping calculation of the target write address for writing a loaded scaling factor to be processed into the cache area in a linear memory layout according to an embodiment of the present disclosure is shown.

[0029] Figure 6A This diagram illustrates that all the requested data is within the range of the original scaling factor tensor.

[0030] Figure 6B This diagram illustrates a portion of the requested data that is within the range of the original scaling factor tensor and a portion that is outside the range of the original scaling factor tensor.

[0031] Figure 6C This diagram illustrates that all the requested data is outside the range of the original scaling factor tensor.

[0032] Figure 7 A pixel cell view of the original scaling factor tensor along the second dimension (W dimension) in a block layout is shown according to an embodiment of the present disclosure.

[0033] Figure 8 An exemplary block diagram of a data loading apparatus according to an embodiment of the present disclosure is shown.

[0034] Figure 9 An exemplary block diagram of a processor according to an embodiment of the present disclosure is shown.

[0035] Figure 10 An exemplary block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0036] Figure 11 A block diagram of an example computer device according to an embodiment of the present disclosure is shown.

[0037] Figure 12 An exemplary block diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0038] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0039] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0040] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0041] In general-purpose computing and deep learning acceleration architectures, when performing operations such as low-precision quantized matrix multiplication, computing units need to retrieve a large amount of scale factor (SF) data from global memory to complete inverse quantization calculations. How to efficiently load this data from global memory and store it in a buffer close to the computing unit directly determines the utilization rate of memory access bandwidth, and thus affects the hardware execution efficiency of the computing unit.

[0042] In relevant technical solutions, scaling factor data is typically stored in global memory using a conventional linear layout (e.g., NHW or NWH arrangement, where N represents the batch dimension, H represents the height dimension, and W represents the width dimension). Under this linear arrangement, scaling factors for the same channel in different sequences (e.g., the width dimension) are often stored discretely at physical addresses. For example, when a computing unit needs to read the scaling factors corresponding to 64 consecutive rows of data across the width dimension (depending on the layout, the row dimension can correspond to any dimension such as width or height), because the data is arranged linearly, the hardware typically needs to initiate a separate non-contiguous address read request for each row (i.e., it requires 64 cycles to read 64 discrete addresses). This frequent and fragmented memory access leads to extremely low data read efficiency, severely limiting the utilization of bus bandwidth.

[0043] To alleviate this problem, a data block rearrangement mechanism can be introduced to reorganize originally discrete data into physically continuous two-dimensional data blocks, thereby significantly improving the read bandwidth of computing units. However, loading and mapping linear data or native block data in memory to the block structure of the target cache involves complex address offset calculations and tensor boundary judgments. In particular, the scaling factor tensor itself has special dimensional properties (usually no longer possessing a spatial dimension of depth, but only containing three logical dimensions: batch, width, and channel). If the traditional fine-grained element-by-element splitting logic for regular five-dimensional tensors continues to be used, it will not only introduce a large amount of redundant computing resource overhead, but also make it difficult to achieve a unified hardware processing architecture across different storage layouts (linear layout and block layout).

[0044] Therefore, the relevant processor architecture lacks a general data loading mechanism that can be customized and optimized for the specific dimensional attributes of the scaling factor, and can also efficiently and reliably support the conversion between linear layout and block layout using unified hardware logic.

[0045] To address the aforementioned issues, embodiments of this disclosure provide an efficient data loading method. This disclosure uses data blocks of a preset size as storage units and differentiates basic pixel units based on different original storage formats to perform dynamic address calculations and state machine step transitions. This allows the disclosure to seamlessly accommodate data transfer scenarios in both linear and block-based layouts using a unified underlying splitting logic. This solution not only fully utilizes the three-dimensional dimensionality reduction characteristics of the scaling factor, greatly simplifying the computational overhead and out-of-bounds handling complexity of multi-dimensional addresses, but also significantly increases the data bandwidth from loading the scaling factor from memory to the cache, effectively improving the hardware utilization of the computing unit and overall processing performance.

[0046] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0047] Figure 1 A schematic diagram of the structure of a general-purpose graphics processor according to an embodiment of the present disclosure is shown.

[0048] like Figure 1 As shown, a general-purpose graphics processor can actually be viewed as an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster. Figure 1 The diagram illustrates streaming processor clusters 1 through M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one or more streaming processor clusters can collaboratively process parallel computational tasks. Multiple streaming processor clusters can be interconnected through a global cache or global memory (e.g., ...). Figure 1 High-bandwidth memory (HBM) is used for data sharing and interaction.

[0049] Taking streaming processor cluster 1 as an example, each streaming processor cluster can include multiple computing units, such as Figure 1 The computation unit consists of computation units 1, 2, and N, where N is a positive integer. Each computation unit is used to perform specific arithmetic and logical operations, such as accumulation, reduction, regular addition, subtraction, multiplication, division, and matrix multiplication and addition operations for multidimensional tensors.

[0050] A computing unit can include multiple cores, each of which may include an arithmetic logic unit (ALU), a floating-point unit, etc., to execute specific computational tasks. In addition, a computing unit may include a register file and shared memory for hierarchical storage of source and destination data related to the computational task. Cores within a computing unit can share data and communicate with each other through this shared memory.

[0051] The streaming processor cluster may also include a buffer, which can be used to share data between different computing units within the streaming processor cluster, or as a low-latency relay station for loading external data (such as tensor data in high-bandwidth memory) into the computing unit.

[0052] When performing general computing tasks such as artificial intelligence and deep learning (e.g., low-precision quantized matrix multiplication), computing units need to process large amounts of data. This data includes not only regular feature maps and weight tensors, but also scaling factors used to adjust the data representation range during quantization / dequantization. Typically, this data containing scaling factors is initially stored in large-capacity storage components, such as… Figure 1 In the high bandwidth memory (HBM) shown.

[0053] During computation, the specific method by which scaling factors and other data are efficiently moved from high-bandwidth memory (HBM) to the buffer or shared memory of the streaming processor cluster via load instructions becomes crucial in determining memory access bandwidth and computing unit hardware utilization. As mentioned earlier, the original storage method of scaling factors in high-bandwidth memory (e.g., linear layout) often leads to discrete memory access. Therefore, the embodiments of this disclosure are based on... Figure 1 The hardware architecture shown provides an efficient data loading mechanism that aims to significantly improve the overall performance of the graphics processor by efficiently loading scaling factors from high-bandwidth memory to the cache through customized state machine logic and data block rearrangement.

[0054] Those skilled in the art should understand that Figure 1 The embodiments shown are merely exemplary and do not constitute a limitation on the scope of this disclosure. In actual hardware architectures, the coupling of modules or units in the various figures is not limited to the manner illustrated. Furthermore, Figure 1 The number of each module (M, N) can be flexibly set according to specific computing power requirements and hardware resources, and this disclosure does not impose specific restrictions on this.

[0055] Figure 2 An exemplary flowchart of a data loading method according to an embodiment of the present disclosure is shown.

[0056] In embodiments of this disclosure, for example, having Figure 1 The device or electronic device shown is a general-purpose graphics processor hardware architecture used to perform... Figure 2 The data loading method 200 is shown in the figure.

[0057] Figure 2 Exemplary steps of method 200 are shown.

[0058] In step S202, a data loading instruction for loading the scaling factor to be processed into the buffer is received to determine the base address for writing the scaling factor to be processed into the buffer.

[0059] In step S204, based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded in each of the multiple requests used to load the scaling factor to be processed are determined.

[0060] In step S206, the target write address in the cache for each of the multiple requests is determined based on the base address and the data size of the pixel unit.

[0061] In step S208, multiple requests are sent sequentially, and the data corresponding to each request is written to the cache based on its corresponding target address.

[0062] In tensor computations (such as matrix multiplication) of deep learning models, scaling factors are typically calculated during floating-point quantization or dequantization. Scaling factors are used to scale and adjust the numerical representation range of data during data format conversion to minimize computational precision loss. These multidimensional data sets describing the scaling factors, generated during quantization or stored beforehand, constitute the original scaling factor tensor in this disclosure.

[0063] It should be noted that the original scaling factor tensor can be stored in various types of memory, such as dynamic random access memory (DRAM), high-bandwidth memory (HBM), global memory, or multi-level caches (such as L2 / L3 cache). The embodiments of this disclosure do not limit the specific type of storage component where the original scaling factor tensor resides; for ease of description, they will be collectively referred to as memory below.

[0064] For example, a feature map tensor may include five dimensions (e.g., batch dimension, depth dimension, height dimension, width dimension, and number of channels dimension). However, for a scaling factor tensor, since it is typically determined for a specific channel or a specific spatial pixel, it is usually flattened or omitted in the spatial dimensions (e.g., depth and / or height dimensions). Therefore, in embodiments of this disclosure, the original scaling factor tensor and the scaling factor to be processed have a specific number of dimensions, for example, only three logical dimensions: batch dimension, channel dimension (C dimension), and width dimension (W dimension). The width dimension can also be mapped to a pixel dimension in the data processing described below, meaning pixels are stored along the width dimension, and each pixel may include data corresponding to its respective channel dimension.

[0065] For example, the scaling factor to be processed can be the complete original scaling factor tensor. Alternatively, in some embodiments, due to capacity limitations of the cache within the computing unit or the actual requirement for tensor block computation, the scaling factor to be processed can also be a part of the original scaling factor tensor. For example, based on current computational needs, the computing unit can obtain only the scaling factors of a specific batch, a specific pixel range, or a specific channel range from the original scaling factor tensor as the scaling factor to be processed.

[0066] Furthermore, the data storage format of the original scaling factor tensor in memory can be a linear layout or a block layout consisting of data blocks of a preset size. For example, when the data storage format is a linear layout, the scaling factor tensor is stored contiguously in memory in a one-dimensional linear arrangement. When loading the scaling factors to be processed in a linear layout into the cache as fixed-size data blocks, fragmented memory accesses may occur because the scaling factors for specific pixels and channels to be acquired are not contiguous in physical address.

[0067] On the other hand, the data storage format of the original scaling factor tensor can also be a block layout, meaning it is stored in memory in units of fixed-size data blocks (e.g., containing a preset number of pixels and channels). However, even in a block layout, the dimensional order in which the original scaling factor tensor is stored contiguously in memory (e.g., prioritizing channel dimension contiguity) may conflict with the contiguity required by the scaling factor during actual loading. This mismatch in dimensional contiguity can also lead to discrete and complex memory access operations when loading across data blocks.

[0068] Based on the above, embodiments of this disclosure provide corresponding data loading and writing strategies for two different scenarios: the original scaling factor tensor has a linear storage layout and a data block unit storage layout. The method provided by these embodiments enables the processing of data with different storage layouts using unified underlying logic, ensuring that the scaling factors to be processed are ultimately stored efficiently and continuously in the target cache as data blocks.

[0069] Specific examples of this disclosure will be described in detail below.

[0070] In step S202, a data loading instruction for loading the scaling factor to be processed into the buffer is received to determine the base address for writing the scaling factor to be processed into the buffer.

[0071] In some embodiments, the data loading instruction may be an asynchronous memory access instruction generated by a compiler from a software program running on the upper layer of the processor (such as an artificial intelligence inference program or a scientific computing application); or, in some embodiments, the data loading instruction may be a microinstruction sent by the instruction dispatch module inside the computing unit to a hardware engine (such as a data loading engine or a direct memory access controller) specifically responsible for data movement. Upon receiving the data loading instruction, the hardware unit responds to the instruction and initiates subsequent data movement operations.

[0072] It should be noted that the entire process of loading the scaling factor to be processed from memory to the cache includes two parts: a data read operation at the data storage source (e.g., memory) and a data write operation at the data storage destination (e.g., the cache). The data loading instruction may carry relevant addressing description parameters, which are used not only to determine the source address for reading the original scaling factor data from memory, but also to determine the location where the scaling factor to be processed is stored at the destination.

[0073] For example, the parameter carried in the data loading instruction, which indicates the storage location at the destination, is used to determine the base address of the tensor to be processed written to the buffer. The base address of the buffer represents the starting storage address of the contiguous storage space occupied by the scaling factor data to be processed in the destination buffer.

[0074] In some embodiments, the base address of the cache can be a specific physical memory address (e.g., a 64-bit or 32-bit hexadecimal address constant); or, in some embodiments, the base address can be a pointer, an offset relative to the start of the current cache segment, or a dynamically calculated result stored in a specific register. In subsequent write operations, the hardware will use this base address as a reference origin and, in conjunction with the size of the data block and the dimensional offset, further calculate the specific target write address of each data request in the cache.

[0075] In step S204, based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded in each of the multiple requests used to load the scaling factor to be processed are determined.

[0076] In some embodiments, multiple requests are used to sequentially load pixel units from the original scaling factor tensor along a first dimension (e.g., the channel dimension), and all are ultimately written to the target buffer in units of data blocks. The size of the pixel unit along the first dimension is the same as the size of the data block along the first dimension.

[0077] As described above, the original scaling factor tensor is typically generated in a linear, continuous form during model quantization. However, before being stored in global memory, it can be stored directly with a linear layout, or it can be stored in memory as data blocks of a preset size through a data rearrangement mechanism. The specific method for converting the original scaling factor tensor from linear to block format before storage in memory can employ the method described below or other methods. The specific implementation does not affect the understanding of the scheme disclosed herein, and therefore will not be described in detail here.

[0078] In some embodiments, to maximize bus transmission efficiency and match the computational characteristics of the underlying hardware, a suitable preset size for the data block can be pre-determined based on the amount of data required for a single load by the computation unit and the bit width limitations of the memory data channel. For example, in some examples, the hardware architecture of the tensor computation unit / tensor core determines that its optimal computation dimension within one clock cycle is a data matrix of 64 rows by 4 columns (or 64 columns by 4 rows). In this case, storing a continuous "64×4" data block as a standard data block unit (e.g., containing 64 elements in the width direction and 4 elements in the channel direction) in memory or a cache greatly benefits the hardware by allowing it to directly initiate burst transfers from a single base address to load the required amount of data. This avoids redundant address calculations and saves the loading time required to acquire data elements in different rows or columns over multiple clock cycles, thus improving data loading and computation efficiency.

[0079] It should be noted that the preset size of the data block does not affect the implementation of the technical solution of this disclosure, and this disclosure does not impose any limitations on it. For example, in other hardware architecture examples, the size of the data block can also be 32×8, 16×16, etc. For ease of understanding and explanation, in the following embodiments of this disclosure, a fixed size of "64 width elements × 4 elements (scaling factor)" (assuming that each scaling factor occupies 1 byte, then a data block is 256 bytes) is used as a data block unit for illustration.

[0080] To better understand the physical arrangement characteristics of the original scaling factor tensor under different data storage formats, a detailed explanation will be provided below with reference to the accompanying figures.

[0081] Figure 3A and Figure 3B Schematic diagrams are shown for the original scaling factor tensor according to embodiments of the present disclosure, which are linear storage layout and block storage layout, respectively.

[0082] like Figure 3A This shows one arrangement of the original scaling factor tensor in a linear layout. Figure 3AIn the diagram, the dashed grid represents the basic data elements of the multidimensional tensor after it has been flattened, and the arrows indicate the storage direction of the data in physical memory, with addresses increasing sequentially. In this example, the original scaling factor tensor is initially stored sequentially along the horizontal dimension (i.e., the channel dimension, C dimension) in the diagram; after all elements in the C dimension have been stored, the memory address will jump to the next row along the vertical dimension (i.e., the width dimension, W dimension) and continue to be stored sequentially along the C dimension.

[0083] It should be noted that, Figure 3A This is merely one specific example of a linear layout (i.e., C-dimensional continuation is prioritized). In other embodiments, the linear storage of the original scaling factor tensor may also prioritize continuation along the W-dimensional dimension (i.e., arranged continuously in columns) and then switch along the C-dimensional dimension. This disclosure does not limit which specific dimension is prioritized for continuation under linear storage.

[0084] like Figure 3B This illustrates one arrangement of the original scaling factor tensor in a block-based storage layout. In this block layout, the original tensor data is logically divided into multiple data blocks of fixed preset sizes (as shown by the large squares marked with thick lines in the figure). The physical continuity of the block layout is reflected in two levels.

[0085] Viewed in units of data blocks (e.g.) Figure 3B As shown in the left sub-diagram, within any single data block, data elements are stored contiguously in physical memory. The broken-line arrows within the left sub-diagram illustrate the continuous flow of data within the block, meaning that a computing unit can load the entire data block in a single burst read.

[0086] From the perspective of the entire tensor (e.g.) Figure 3B As shown in the right-hand sub-diagram, these data blocks are treated as individual "pixel units," and are arranged consecutively in physical memory according to a specific dimensional priority order. The long arrows in the right-hand sub-diagram illustrate the continuous storage direction between data blocks. In this example, data blocks are preferentially stored consecutively along the C-dimensional direction. After storing one row of data blocks, the process then switches to the next row along the W-dimensional direction to continue storing.

[0087] Similarly, it should be noted that, Figure 3B This is merely a specific example of a block layout. In practical applications, not only can the continuous direction of elements within a data block change, but the continuous order between data blocks (e.g., prioritizing the continuous arrangement of data blocks along the W-dimensional direction) can also vary depending on the underlying hardware and driver configuration. This disclosure does not impose any specific restrictions on this.

[0088] As described above, to leverage a unified underlying hardware logic (e.g., a unified direct memory access controller or state machine) to accommodate data loading across different layouts, this disclosure introduces the concept of a "pixel unit." In deep learning data processing, the "width dimension (W dimension)" of a scaling factor tensor is typically used to represent flattened spatial nodes or sequence nodes; therefore, this width dimension can also be mapped to and collectively referred to as the "pixel" dimension (e.g., containing data points with coordinates p0, p1...pn). The "pixel unit" is the smallest logical unit operated by the hardware state machine in this disclosure when initiating a single load request.

[0089] Specifically, based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded in each of the multiple requests used to load the scaling factor to be processed are determined. The following is combined with... Figure 4A and Figure 4B Examples are provided, describing two cases based on linear layout and block layout.

[0090] Figure 4A A schematic diagram of pixel unit partitioning of the original scaling factor tensor under a linear storage layout according to an embodiment of the present disclosure is shown.

[0091] like Figure 4A As shown, the vertical axis represents the width dimension (W dimension, i.e., pixel rows p0 to pm), and the horizontal axis represents the channel dimension (C dimension, i.e., scaling factors SF0 to SFn).

[0092] In response to the fact that the original scaling factor tensor is stored in memory in a linear layout, it is determined that the pixel unit comprises the same number of consecutive data elements in the first dimension as the data block in the first dimension.

[0093] For example, in a linear storage layout, scaling factor data is typically stored contiguously according to the channel dimension (C dimension, corresponding to the first dimension). This means that for a given pixel (i.e., a specific width coordinate W), the scaling factors of all its corresponding channels are physically adjacent. In this case, a pixel unit can be defined as a continuous segment of channel data with a length equal to the channel width of the data block at a single pixel coordinate.

[0094] Combination Figure 4A For example, taking a preset data block size of 64×4 (64row×4SF) as an example, in a linear layout, one pixel unit is equivalent to four consecutive scaling factors on a specific row. Figure 4AAs shown in the gray shaded area, "Pixel Unit 1" corresponds to consecutive SF0 to SF3 in the first row (p0); the following "Pixel Unit 2" corresponds to consecutive SF0 to SF3 in the second row (p1). At this point, a single request loads these four consecutive scaling factors (if each SF is 1 byte, then the size is 4 bytes). To assemble a complete 64×4 data block in the buffer, the state machine needs to proceed row by row in the width dimension (i.e., the pixel dimension W) with a step size of 1, continuously initiating 64 requests (until all data is loaded). Figure 4A The pixel unit 64 (identified in the text) loads a pixel unit containing 4 elements each time a request is made.

[0095] Figure 4B A schematic diagram of pixel unit partitioning of the original scaling factor tensor in a block storage layout according to an embodiment of the present disclosure is shown. Figure 4A Similarly, the vertical axis represents the width dimension (W dimension), and the horizontal axis represents the channel dimension (C dimension).

[0096] In response to the fact that the original scaling factor tensor is stored in memory in a block-based layout, it is determined that the pixel unit includes continuous data elements of the same preset size as the data block.

[0097] For example, in a block-based storage layout, the original scaling factor is already stored contiguously in memory as a physical unit of data blocks of a preset size (e.g., 64×4). In this case, if the request method of "loading 4 scaling factors at a time" as in a linear layout is continued, the originally contiguous data blocks will be broken, resulting in serious bandwidth waste.

[0098] Therefore, in some embodiments of this disclosure, a "pixel unit" can be defined as an entire data block of a preset size. Combined with Figure 4B For example, taking a preset data block of 64×4 as an example, one pixel unit contains a complete set of 64 pixels (p0 to p63) and 4 scaling factors corresponding to each pixel (a total of 256 bytes of continuous data elements). Figure 4B As shown in the large gray square, the entire upper left data block containing p0 to p63 and SF0 to SF3 is defined as "pixel unit 1", while the data block immediately to its right containing p0 to p63 and SF4 to SF7 is defined as "pixel unit 2".

[0099] In this mode, a single request can directly load an entire block of data (pixel unit) into the cache. Because the granularity of the hardware request loading matches the actual granularity of the data stored in memory, the throughput of data transfer and the utilization of bus bandwidth are greatly improved.

[0100] In step S206, the target write address in the cache for each of the multiple requests is determined based on the base address and the data size of the pixel unit.

[0101] Understandably, the steps for determining the write address in the target memory for each request differ because the pixel unit data size varies due to the different storage methods of the original scaling factor tensor in memory.

[0102] Figure 5 This diagram illustrates the mapping calculation of the target write address for writing a loaded scaling factor to the cache in a linear memory layout according to an embodiment of the present disclosure. (Combined with...) Figure 5 The following is a detailed explanation using the example of a data block with the first dimension being the channel dimension (C dimension), the second dimension being the width dimension (W dimension, i.e., pixel rows p0 to pn), and the preset size of the data block being 64×4 (i.e., 64 elements in the width direction, 4 elements in the channel direction, and a size of 256 bytes), and the pixel unit being 1 row of 4 elements (i.e., a size of 4 bytes).

[0103] For example, in some embodiments where the data storage format of the original scaling factor tensor is a linear layout, the step of determining the target write address in the cache for each of multiple requests based on the base address and the data size of the pixel unit may include: The data of each pixel unit loaded by each request is continuously written to the buffer along the second dimension until the amount of data continuously written along the second dimension equals the number of bytes in a data block. The offset between the target write addresses of two consecutive requests is equal to the number of bytes in the pixel unit's data size.

[0104] like Figure 5 The first data block in the top left corner (p0 to p63, SF0 to SF3) is shown. Assume the initial base address of the buffer is gsm. When the state machine initiates the first request, it writes the loaded pixel units (SF0 to SF3 in row p0) to the target address gsm. Subsequently, the state machine progresses down along the second dimension (W dimension), initiating the second request for row p1, with a target write address of gsm + 4b (i.e., increasing the data size of one pixel unit by 4 bytes). This continues, with the target write addresses linearly increasing sequentially for requests initiated from p2, p3, up to p63: gsm + 8b, gsm + 12b, ... gsm + 252b.

[0105] At this point, 64 pixel units were written continuously along the W dimension. The total amount of data written continuously was exactly equal to the number of bytes of a data block (64 × 4 Byte = 256 Byte), thus assembling the first physically continuous data block that conforms to the preset size in the buffer.

[0106] It should be noted that this strategy of prioritizing sequential writing along the W dimension within a data block allows the data arrangement within the cache to match the memory access pattern of subsequent tensor computation units (such as when the tensor kernel performs dequantization operations) sliding along the pixel dimension, thereby significantly improving the cache hit rate and the processing efficiency of the computation units.

[0107] In response to the amount of data continuously written in the second dimension reaching the preset size of the data block in the second dimension, the base address is updated based on the total size of the data block in the first dimension and the preset size of the data block, so that the next consecutive request will use the updated base address as the corresponding target write address.

[0108] like Figure 5 As shown, after the first data block (p0 to p63) is written, if there is still data remaining in the W dimension of the scaling factor to be processed (i.e., the pixels p64 and beyond need to be processed), the state machine needs to jump and calculate the starting base address of the next data block (p64 to p127, SF0 to SF3).

[0109] At this point, in order to reserve full storage space in the cache for the scaling factor (SF) on all channels of the current 64 rows (p0 to p63), the base address update needs to span the entire first dimension (C dimension). Combined with... Figure 5 For example, suppose the total size of the scaling factors to be processed in the first dimension is total_copy_sf_num = 8, that is, 8 scaling factors, and the preset size of each data block in the first dimension is 4. The state machine calculates the span offset using the formula (total_copy_sf_num / 4) × 256Byte, which is (8 / 4) × 256 = 512Byte.

[0110] Therefore, when the state machine processes SF0 to SF3 in line p64, the updated target write base address is gsm+512b.

[0111] In some embodiments, the base address may not be updated. Instead, when a second data block is written consecutively along the W dimension, the number of bytes calculated by the following formula (1) is added to the base address: (Total size of the scaling factor to be processed in the first dimension / Size of the data block of the preset size in the first dimension) × Total bytes of the data block (Formula 1).

[0112] Based on the above-mentioned arrangement strategy of reserved space, for the same coordinate position of the scaling factor to be processed in the second dimension, the difference between the target write addresses of two adjacent data blocks along the first dimension in the cache is equal to the number of bytes of the data block.

[0113] like Figure 5 As shown, for the same coordinate p0 in dimension W, its target write address in the first data block (SF0 to SF3) is gsm; while in the second data block (SF4 to SF7) adjacent to it in dimension C, the target write address for the same coordinate p0 is gsm + 256b. Similarly, for p1, the addresses of the two adjacent data blocks are gsm + 4b and gsm + 260b, respectively. The difference between the two is limited to the total number of bytes in a data block (256 bytes). This regular address offset greatly simplifies the hardware computational complexity of multi-dimensional address mapping.

[0114] It should be noted that the above Figure 5 The address mapping strategy shown is based on the specific arrangement principle of "prioritizing the clearing of the entire C-dimensional address space corresponding to a fixed 64 rows in the cache". In other embodiments, the state machine may also adopt different dimension-priority mapping strategies based on the specific computing needs of the underlying processor or the shared memory allocation mechanism.

[0115] For example, in another embodiment, the cache can be configured to "prefer to arrange data blocks along the second dimension (W dimension)". In this case, after the first data block (p0 to p63, SF0 to SF3) is filled, the data block stored immediately below it at the physical address is not the data block on the right (SF4 to SF7), but the second data block directly below it (p64 to p127, SF0 to SF3). At this time, the target write address of p64 is gsm + 256b.

[0116] Under this dimension-first strategy, the aforementioned constraint that "the address difference between two adjacent data blocks in the same W-coordinate in the C-dimension is 256 bytes" no longer applies. In this case, two adjacent data blocks in the first dimension (C-direction) will be separated by the entire W-dimension data space. For example, if the total length of the scaling factor to be processed in the W-dimension is 128 rows (i.e., containing two data blocks in the W-direction), then for the same coordinate p0, its address in the SF0 to SF3 data blocks is gsm, while the address in the adjacent SF4 to SF7 data blocks will become gsm + (2 × 256b) = gsm + 512b. That is, the difference in the target write address is equal to the total number of bytes in the data blocks contained in the entire W-dimension space.

[0117] The embodiments of this disclosure, by flexibly configuring the base address update formula, can uniformly support the above-mentioned various target memory layout requirements, thereby maximizing the adaptation to the data reading preferences of different architecture computing units.

[0118] For example, in some embodiments where the data storage format of the original scaling factor tensor is a block layout, the step of determining the target write address in the cache for each of multiple requests, based on the base address and the data size loaded for each request, may include: The target write addresses corresponding to two consecutive requests are determined to be linearly increasing, and the offset between the target write addresses corresponding to two consecutive requests is equal to the number of bytes of one data block.

[0119] As mentioned above Figure 4B As described above, in the block storage layout, the original scaling factor tensor is stored contiguously in memory in units of fixed-size data blocks. In this case, a pixel unit contains contiguous data of the entire data block's preset size (e.g., 64 rows × 4 SF, totaling 256 bytes of contiguous data). Since a single request can fully load the entire data block, when writing to the cache, unlike linear layouts where data blocks are assembled line by line, the overall address mapping is performed directly with the data block size as the step size.

[0120] For example, combining Figure 4B As shown, assuming the initial base address of the buffer is also gsm, when the state machine initiates the first request, it writes the entire loaded "pixel unit 1" (i.e., the 256-byte data block containing p0 to p63 and SF0 to SF3 in the upper left corner) to the target address gsm.

[0121] In an embodiment where the target cache is configured to preferentially arrange data blocks along the first dimension (C dimension), the second request that follows will load... Figure 4B The adjacent "pixel unit 2" (i.e., the data block containing p0 to p63, SF4 to SF7). Since this is a sequential write of two complete data blocks, the state machine directly determines that the target write address for the second request is gsm+256b.

[0122] As can be seen, in this mode, the target write addresses of two consecutive requests exhibit a simple linear increasing relationship in the cache, and the offset difference is exactly equal to the total number of bytes in a data block (e.g., 256 bytes). This continuous write method, which uses the entire data block as the pixel unit, greatly reduces the complexity of the address calculation logic.

[0123] Similar to linear layout, when a row (or column) of data block is written in a priority dimension, if there is still data remaining in another dimension if the scaling factor to be processed, the calculation of the target write address will also involve jumping and updating the base address.

[0124] In the example above where data is preferentially arranged along the C dimension, after writing all C-dimensional data blocks within the current width range (p0 to p63), the state machine needs to process the next width range (p64 to p127). At this point, for a request to load the first data block (p64 to p127, SF0 to SF3) within this range, the target write address needs to skip the entire row of C-dimensional data blocks already written above. The formula for calculating its updated base address is similar to that of the linear layout, still being the starting address + (total_copy_sf_num / 4) × 256 bytes. That is, the address jump logic in the block layout here essentially applies the formula for calculating the single-row span in the linear layout directly to the span calculation of pixel units, where data blocks are the unit. It should be understood that although data is read and written in units of data blocks, the total number of SFs represented by total_copy_sf_num remains unchanged compared to the linear layout storage case.

[0125] In some embodiments where the target cache is configured to preferentially arrange data blocks along the second dimension (W dimension), the write order will change. When completed... Figure 4B After writing "pixel unit 1", the second consecutive request will not load SF4 to SF7 on the right, but will instead load the data blocks directly below (i.e., p64 to p127, SF0 to SF3). At this time, the target write addresses corresponding to these two consecutive requests still maintain a linear increase (i.e., gsm and gsm + 256b). Only after all the data blocks in the W direction corresponding to the current C dimension (SF0 to SF3) have been loaded will the state machine perform an address jump across the entire W dimension space to find the starting write position of the next C dimension data block (SF4 to SF7).

[0126] In summary, regardless of whether a linear or block layout is used, and regardless of the choice of specific priority loading dimensions, the embodiments of this disclosure reuse the underlying unified state machine address update and jump logic by flexibly adjusting the granularity of pixel units. This design not only ensures the flexibility of data block arrangement in the target memory, but also minimizes the time and resource overhead of hardware when handling complex multidimensional tensor transfers.

[0127] In neural network computation, to meet the requirements of tensor computation units (such as tensor kernels) for data alignment and fixed data block size (e.g., 64×4), the size of the scaling factor tensor to be processed (copy_SF, i.e., the data actually requested by the computation unit) is often rounded up. However, the true logical dimension of the original scaling factor tensor (tensor_SF, i.e., the valid data actually existing in memory) may not be an integer multiple of the preset size of the data block. This leads to the scaling factor requesting data beyond the true physical boundaries of the original scaling factor tensor when it makes a loading request to memory.

[0128] To better understand the handling mechanism in the case of boundary crossing, the following section combines the appendix... Figures 6A to 6C Please provide a detailed explanation. Figures 6A to 6C The diagrams illustrate three different positional relationships of the data to be loaded by the scaling factor request relative to the boundary of the original scaling factor tensor according to embodiments of the present disclosure.

[0129] like Figure 6A The diagram illustrates a scenario where all requested data is within the range of the original scaling factor tensor. In this example, the scaling factor request to be processed loads SF0 to SF7, and when the request loads SF4 to SF7, this data is already contained within the original scaling factor tensor (assuming the original scaling factor tensor includes at least SF0 to SF8).

[0130] like Figure 6C The diagram illustrates a scenario where all requested data is outside the range of the original scaling factor tensor. For example, due to alignment requirements, the scaling factors to be processed still need to be loaded from SF8 to SF11, but the original scaling factor tensor ends at SF7 (in the case where the original scaling factor tensor includes SF0 to SF7).

[0131] It should be noted that, for Figure 6A (All included) and Figure 6C In the case of (all outside), since the data attributes of the entire pixel unit are uniform, the state machine does not need to perform special splitting processing for the request. The state machine can directly use the method described above for determining the target write address for address mapping in linear layout storage. The only difference is that, for Figure 6C In this completely external scenario, the hardware will not initiate a real read request to the memory, but will directly convert the request into an operation of filling the target write address with a preset value (such as filling with zeros).

[0132] However, when the loading granularity of pixel units crosses the true boundary of the original scaling factor tensor data, problems such as... occur. Figure 6B This illustrates a cross-boundary situation where part of the boundary is inside and part is outside. Figure 6BIn the example, each requested pixel unit contains four scaling factors (SF8 to SF11), but the true boundary of the original scaling factor tensor falls exactly at SF9 (if the original scaling factor tensor includes SF0-SF9). This means that SF8 and SF9 are valid data existing in memory (inside tensor_SF), while SF10 and SF11 are invalid data outside the physical memory boundary (outside tensor_SF). Typically, the hardware bus cannot simultaneously handle the reading of valid physical addresses and the padding of invalid addresses within an atomic memory read transaction; therefore, such cross-boundary requests must be split.

[0133] Specifically, the data loading method provided in this disclosure also includes: In response to the range of the scaling factor to be processed crossing the boundary range defined by the boundary value, a state machine is used to generate multiple sub-requests based on the request, such that the data loaded by each sub-request is either entirely within the data range of the original scaling factor tensor defined by the boundary value, or entirely outside the data range of the original scaling factor tensor defined by the boundary value.

[0134] like Figure 6B As shown, when the state machine detects that a pixel unit (SF8 to SF11) crosses the boundary, it performs sub-request splitting at the boundary. The original single request to load four elements is split into two sub-requests: one is a valid sub-request to load SF8 to SF9, and the other is an invalid sub-request corresponding to SF10 to SF11. This splitting ensures that each hardware operation is pure and singular in terms of data attributes.

[0135] For the split sub-requests, the target write address in the target cache needs to be remapped.

[0136] For valid sub-requests where all loaded data falls within the data range of the original scaling factor tensor defined by the boundary value: The base address of the target write address of the valid sub-request in the cache is equal to the starting base address of the data loaded by the valid sub-request in the second dimension (W dimension), plus the amount of valid data before the data loaded by the valid sub-request in the first dimension (C dimension) divided by the preset size of the data block in the first dimension, and then multiplied by the total number of bytes of the data block; and the byte offset within the data block is based on the pixel unit loaded by the valid sub-request.

[0137] by Figure 6BTaking SF8 to SF9 as an example, assume they are located in row p0. Their starting base address in the W dimension is gsm. The effective data volume before the first dimension is 8 (i.e., SF0 to SF7). Since the preset size of the data block in the first dimension is 4, the total number of bytes is 256 bytes. Therefore, the base address of the data block containing this effective sub-request is gsm + (8 / 4) × 256 bytes = gsm + 512 bytes. As for the remaining p2 to p63 in the W dimension, the local byte offset in the W dimension is based on the W dimension where the currently loaded pixel unit is located. Since the data block is written continuously row by row along the W dimension when converting from linear layout to block layout, for example, the write address offset of (p1, SF8) relative to (p0, SF8) is 4 bytes.

[0138] For invalid sub-requests where all loaded data is outside the data range of the original scaling factor tensor: The invalid sub-request is converted into an operation that writes a preset value (e.g., padding with 0) to the target write address corresponding to the invalid sub-request. The base address of the target write address of the invalid sub-request in the cache is equal to the starting base address of the data loaded by the valid sub-request in the second dimension, plus the amount of valid data preceding the data loaded by the valid sub-request in the first dimension divided by the preset size of the data block in the first dimension, and then multiplied by the total number of bytes in the data block; and the byte offset within the data block is performed based on the pixel unit loaded by the invalid sub-request.

[0139] by Figure 6B Take the invalid sub-requests SF10 to SF11, split from row p0, as an example. Because this invalid data must be concatenated immediately after the valid data in the cache to form a complete 64×4 data block, the parameters used to calculate the base address of its data block portion can borrow the parameters from the aforementioned valid sub-requests. That is, its data block base address is also calculated as gsm + 512 bytes, and both share the same data block space. To avoid address conflicts, byte offsets within the data block can be used. For the invalid sub-requests SF10 to SF11 in row p1, the row offset is 4 bytes. However, since the SF portion of the invalid request consists of the last two elements of the four elements in that row (corresponding to a column offset of 2 bytes), its byte offset within the data block is 4 bytes + 2 bytes = 6 bytes. Finally, the target zero-padding address for this invalid sub-request is located as gsm + 512 bytes + 6 bytes, with a write length of 2 bytes.

[0140] Through the above design, when the hardware state machine processes valid data reading and out-of-bounds invalid data filling, it can not only fully share the same set of jump addressing arithmetic logic units based on data blocks, but also achieve seamless splicing of valid data and zero-padding data within the same buffer data block through the natural misalignment of different column offsets within the data block.

[0141] For cases where the original scaling factor tensor is stored in a block-based distribution as a unit of data, this embodiment utilizes the underlying hardware state machine to automatically split and coordinate the loading request for different cases where continuous loading is possible under different dimensions.

[0142] Figure 7 A pixel cell view of the original scaling factor tensor along the second dimension (W dimension) in a blocky layout is shown according to an embodiment of the present disclosure. Figure 7 As shown, P0, P1, P2...Pn represent data blocks arranged continuously along the W dimension (e.g., each data block contains 64 W coordinates and 4 C coordinates). In a block layout, the data is physically packaged into such data block granularity.

[0143] In ideal conditions (e.g., when the scaling factor to be processed is completely contiguous and aligned with the physical memory layout in the channel C, width W, and batch dimensions), the state machine does not need to perform fragmented partitioning. It can even initiate a massive direct memory access request through a single base address to load an entire row of data or even an entire batch of tensor data at once, achieving a global no-split strategy. Since the data is already stored contiguously at the data block level, this single large-scale load can maximize the utilization of memory access bandwidth.

[0144] However, in actual calculations, the scaling factor to be processed often exhibits a discontinuous state in certain dimensions (e.g., only a specific channel or a scaling factor of a specific width needs to be extracted). In this case, a state machine must be used for dimensionality reduction and decomposition.

[0145] In some embodiments of this disclosure, regardless of how the state machine is divided, the minimum data granularity of each loading request is equal to the preset size of the data block, and the data loaded by each sub-request after division must satisfy boundary conditions, that is, all of it must belong to the original data range or all of it must belong to the zero-padding area outside the range (the specific processing is as described above). Figures 6A-6C (as described).

[0146] Specifically, a state machine is used to determine multiple requests for loading the scaling factor to be processed, wherein: In response to the scaling factor being discontinuous in the first dimension (e.g., dimension C), the state machine determines to partition the request in the second dimension (e.g., dimension W).

[0147] For example, when the computing unit only needs certain discontinuous channels (discontinuous in dimension C), the state machine cannot initiate long-span continuous reads along dimension C. In this case, combined with... Figure 7 The state machine will load data blocks one by one along the W dimension. That is, it will first load the 256-byte data block containing P0, and then jump to load the data block containing P1. This splitting method ensures that even when the C dimension is limited, the maximum continuous bandwidth can still be guaranteed in the W dimension in units of data blocks.

[0148] In response to the scaling factor being discontinuous in the second dimension (such as the W dimension), the state machine determines to partition the request in the third dimension (such as the H dimension or the Batch dimension).

[0149] For example, if not only is channel C discontinuous, but the reading of width W is also skipped, the state machine will further seek continuous loading in higher dimensions, splitting in the H dimension or batch dimension, and locating and loading individual independent data blocks row by row or batch by batch.

[0150] To support the above-mentioned splitting logic based on data blocks as pixel units, this disclosure improves the coordinate calculation unit and jump judgment condition of the state machine that uses a single data element as the data loading unit.

[0151] In a tensor loading state machine for linear data, when the state machine steps along the W dimension, its coordinates are usually incremented by 1 element by element (i.e., the step size is +1), and the loop termination condition is usually that the amount of data to be copied is equal to 1 (remain_copy_any dimension == 1).

[0152] However, in this disclosure, since the original scaling factor tensor can be used in a block layout, the basic operation unit has changed from a single element to a data block of a preset size (e.g., a data block containing 64 rows of W-dimensional data).

[0153] Therefore, in response to the state machine determining to use the partitioning of the request in the second dimension (W dimension), when the state machine determines the addressing coordinates in memory, each increment of its coordinates in the second dimension is equal to the size of the data elements included in the data block in the second dimension.

[0154] Specifically, the accumulator in the state machine responsible for controlling the W-dimensional coordinates (w_coord) changes its increment step from the traditional +1 to +64 when splitting along the W-dimensional axis. This is because for each request, the state machine physically traverses a complete data block containing 64 W coordinates (e.g., ...). Figure 7 The state machine jumps directly from P0 to P1, thus greatly reducing the number of cycle times.

[0155] Furthermore, in order to accurately determine when to end the block loading in the current dimension and jump to a higher dimension, the termination conditions for the state machine's state loop transition when dividing the request in the second dimension include: The amount of remaining data to be copied in the highest dimension (e.g., the batch dimension, N) is not equal to 1 (i.e., remain_copy_n != 1); or, The amount of data to be copied in the second dimension (W dimension) is not equal to the size of the data elements included in the data block in the second dimension (i.e., remain_copy_w != 64).

[0156] For example, when the state machine determines whether it has reached the boundary of dimension W, it no longer checks whether the remaining data is 1. Instead, it checks whether the amount of data in dimension W to be copied (remain_copy_w) is equal to the size of the data block in dimension W (e.g., 64) within the preset size of the data block. If it is equal to 64, it means that a complete data block can still be formed, and the state machine continues to loop. If it is not equal to 64 (e.g., due to the tensor's true boundary problem, there are only 32 rows of data remaining), the termination condition is triggered, the state machine exits the current loop, and enters the out-of-bounds splitting and zero-padding processing logic as described above.

[0157] By deeply customizing the state machine step size and boundary judgment conditions, this disclosure maps the complex multidimensional tensor addressing to data block transportation, which not only retains the advantage of the hardware state machine automatically calculating coordinates, but also improves the overall data loading efficiency.

[0158] In step S208, multiple requests are sent sequentially, and the data corresponding to each request is written to the cache based on its corresponding target address.

[0159] Specifically, after the pixel units are divided and the target write address mapping for each request (or sub-request) is completed in the cache, a data loading engine or direct memory access controller will take over the physical execution of these requests.

[0160] In some embodiments, the execution unit will sequentially initiate these data read transactions with specific granularity along the configured dimensional direction to the memory bus.

[0161] In some embodiments, for operations where the requested data falls within the valid range of the original scaling factor tensor, the hardware will directly read the corresponding data in bursts from the calculated source physical address via the high-bandwidth memory interface and accurately write it to the corresponding target write address in the cache. For invalid sub-requests that cross the boundaries of the original tensor due to boundary alignment or other reasons, the hardware will not send a real read request to the memory bus to save bandwidth. Instead, it will directly convert this into an operation of writing a preset value (such as padding with zeros) to the corresponding cache target address internally. Since the write address has been offset according to the preset size of the data block (such as 64×4), discrete or partially out-of-bounds scaling factor data will ultimately be perfectly concatenated into physically continuous, aligned, and standardized data blocks in the cache, allowing subsequent tensor calculation cores to directly call them.

[0162] The data loading method disclosed herein dynamically abstracts the original scaling factor data of different storage formats (linear layout or block layout) into pixel units that adapt to the preset size of the target data block, and relies on a state machine-customized auto-incrementing step size and span mechanism to drive a unified address mapping. Simultaneously, for the valid tensor data boundaries encountered during data loading, the underlying hardware state machine intelligently splits them into continuous valid physical memory requests and independent invalid zero-padding requests, thereby achieving seamless splicing and in-situ filling of out-of-bounds data.

[0163] Using the above method, the embodiments of this disclosure effectively overcome the fragmented memory access caused by the special dimensional attributes of scaling factors in related technical solutions, as well as the deep dependence on redundant computing resources and complex software processing logic when converting between different storage layouts. The dynamic matching of pixel units, state machine address jumps and updates, and the automatic splitting and pruning process for boundary out-of-bounds errors are all automatically closed-loop by the underlying hardware state machine logic, realizing the efficient conversion of tensor data from a linear or native block layout in global memory to a cache block layout without additional software intervention.

[0164] This not only frees up the control computing power of general-purpose computing units for complex multi-dimensional addressing and out-of-bounds judgment, but also matches the huge continuous data path bandwidth requirements of tensor computing cores when performing operations such as dequantization by reusing unified underlying splitting logic and block data rearrangement that fits hardware characteristics. This multiplies the utilization rate of bus access bandwidth and significantly improves the overall throughput performance and hardware utilization of general-purpose graphics processors when dealing with quantitative calculations in artificial intelligence and deep learning.

[0165] Figure 8 An exemplary block diagram of a data loading apparatus according to embodiments of the present disclosure is shown. It can be utilized... Figure 8 The system shown is used to execute Figure 2 Method 200 is described.

[0166] like Figure 8 As shown, in some embodiments, the data loading device 800 includes an instruction acquisition module 802, an instruction parsing module 804, a request generation module 806, and a request execution module 808. In some embodiments, the functions of the instruction acquisition module 802 and the instruction parsing module 804 can be combined into a single integrated instruction parsing unit, and the functions of the request generation module 806 and the request execution module 808 can be combined into a single integrated instruction execution unit.

[0167] The instruction acquisition module 802 is configured to receive a data loading instruction for loading the scaling factor to be processed into the cache, so as to determine the base address for writing the scaling factor to be processed into the cache.

[0168] The instruction parsing module 804 is configured to determine the pixel units loaded by each of a plurality of requests for loading the scaling factor to be processed, based on the data storage format of the original scaling factor tensor in memory. The plurality of requests are used to sequentially load the pixel units in the original scaling factor tensor in a first dimension and write them to the cache in units of data blocks. The size of the pixel unit in the first dimension is the same as the size of the data block in the first dimension.

[0169] The request generation module 806 is configured to determine the target write address of each of the plurality of requests in the cache based on the base address and the data size of the pixel unit.

[0170] The request execution module 808 is configured to send the multiple requests sequentially and write the data corresponding to each request into the cache area based on its corresponding target write address.

[0171] It should be understood that Figure 8 The various modules or units of the device 800 shown can be used with reference to Figure 2 The steps in method 200 described correspond to each other. Therefore, the operations, features, and advantages described above for these methods also apply to device 800 and its included modules and units. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0172] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific actions performed by the modules discussed herein include the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that performs the action, called or otherwise accessed by the specific module.

[0173] According to one aspect of this disclosure, a processor is also provided. Figure 9 An exemplary block diagram of a processor according to an embodiment of the present disclosure is shown. Figure 9 The processor 900 may include an instruction parsing unit 902 and an execution unit 904.

[0174] The instruction parsing unit 902 can be used to receive and parse data loading instructions, wherein the data loading instructions instruct the execution of loading the scaling factor tensor to be processed from memory into the cache.

[0175] Execution unit 904 can be used to execute the data loading instruction.

[0176] The execution unit executes data loading instructions, including implementing the data loading method described above.

[0177] In some embodiments, execution unit 904 may be implemented as a direct memory access controller, a load / store unit, or a dedicated tensor access unit. Internally, execution unit 904 may have hard-wired hardware units implementing the state machine of this disclosure.

[0178] It is understood that the processor 900 can be implemented to perform the data loading method according to embodiments of the present disclosure. For details regarding the specific implementation of the data loading method, please refer to the description above, which will not be repeated here.

[0179] According to one aspect of this disclosure, an electronic device is also provided. Figure 10 An exemplary block diagram of an electronic device according to an embodiment of the present disclosure is shown. Figure 10 As shown, the electronic device 1000 includes a memory 1020, a processor 1040, and a computer program stored in the memory. The memory 1020 and the processor 1040 may be coupled via an internal bus or other communication mechanism. The processor 1040 is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0180] In some embodiments, the processor 1040 may employ, for example... Figure 1or Figure 9 The processor's microarchitecture design is shown.

[0181] Figure 11 A block diagram of an example computer device 2000 according to an embodiment of the present disclosure is shown.

[0182] The computer device 2000 may specifically correspond to the electronic device or processing system in the foregoing embodiments. It should be noted that... Figure 11 The components of the computer device 2000 shown are merely exemplary and not restrictive. Depending on the actual application requirements, the computer device 2000 may have other components or omit some components.

[0183] like Figure 11 As shown, computer device 2000 includes processing device 2010. Processing device 2010 can perform various appropriate actions and processes according to computer-executable instructions stored in read-only memory (ROM) 2020 or loaded from storage device 2080 into random access memory (RAM) 2030. For example, processing device 2010 can be configured to execute the above-referenced... Figures 2 to 7 The described data loading or data storage method. Various programs and data required for the operation of the computer device 2000 can also be temporarily stored in RAM 2030. The processing device 2010, ROM 2020, and RAM 2030 are interconnected and exchange data via bus 2040. Input / output (I / O) interface 2050 is also connected to bus 2040.

[0184] In some embodiments, the processing device 2010 may include a central processing unit (CPU), a graphics processing unit (GPU), a tensor processor (TPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In specific implementations of embodiments of this disclosure, the processing device 2010 may have a built-in or coupled data loading unit and hardware state machine logic as described above to directly process tensor block loading tasks from mass storage (such as system memory) to on-chip cache (such as local cache) at the underlying level.

[0185] Typically, the following devices can be connected to the I / O interface 2050: input devices 2060 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 2070 including, for example, liquid crystal displays (LCDs), light-emitting diode displays (LEDs), speakers, vibrators, etc.; storage devices 2080 including, for example, magnetic tapes, hard disks, solid-state drives (SSDs), flash memory cards, etc.; and communication devices 2090.

[0186] The storage device 2080 can serve as a high-capacity, non-volatile storage medium for persistently storing the operating system, applications, and raw tensor data to be processed, such as large-scale neural network weights and massive input feature maps. The communication device 2090 allows the computer device 2000 to exchange data with other devices (such as cloud servers, other computing nodes in a distributed computing cluster, etc.) via a network, either wirelessly or via wired communication. The communication device 2090 may include a local area network (LAN) card, a wireless local area network (WLAN) card, a Bluetooth module, a near field communication (NFC) module, a cellular communication module, etc.

[0187] Although Figure 11 A computer device 2000 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the computer device 2000 may alternatively implement or possess more or fewer devices. For example, when the computer device 2000 is used as an AI computing accelerator card or server node in a data center, it may not include input devices 2060 and output devices 2070 for human-computer interaction such as touch screens and keyboards, but may mainly consist of a processing unit 2010, high-bandwidth memory, and high-speed communication interfaces.

[0188] The functions of the systems, apparatuses, modules, or units described above herein can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field-Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on. Furthermore, the methods described in this disclosure can also be implemented by software or microcode containing computer-executable instructions that, when executed by the processing device 2010, cause the machine to perform the operations defined in this disclosure.

[0189] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is also provided.

[0190] Figure 12 An exemplary block diagram of a computer-readable storage medium according to embodiments of the present disclosure is shown. Figure 12 As shown, a computer-readable storage medium 3000 stores computer-executable instructions 3010, which, when executed by a processor, implement the steps of any of the method embodiments described above.

[0191] In some embodiments, the computer-readable storage medium 3000 may include any combination of one or more computer program products. For example, without limitation, the non-transitory computer-readable storage medium may include volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0192] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0193] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of this disclosure is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. A data loading method for loading a scaling factor to be processed from a raw scaling factor tensor in memory into a cache, wherein, The scaling factor to be processed is stored in the cache in units of data blocks, each data block having a preset size, and data within the same data block being stored contiguously in the cache. The method includes: Receive a data loading instruction for loading the scaling factor to be processed into the cache area, so as to determine the base address for writing the scaling factor to be processed into the cache area; Based on the data storage format of the original scaling factor tensor in memory, the pixel units loaded by each of the multiple requests for loading the scaling factor to be processed are determined. Specifically, for cases where the data storage format of the original scaling factor tensor in memory is a block layout in units of data blocks, the pixel units are determined to include continuous data elements with the same preset size as the data blocks. Furthermore, the multiple requests are used to sequentially load the pixel units in the original scaling factor tensor along a first dimension and write them to the cache in units of data blocks. The size of the pixel unit in the first dimension is the same as the size of the data block in the first dimension. When the original scaling factor tensor adopts the block layout and the scaling factor to be processed is discontinuous in the first dimension, the state machine is used to determine the plurality of requests for loading the scaling factor to be processed, wherein the state machine determines the requests by dividing them in the second dimension. Based on the base address and the data size of the pixel unit, determine the target write address in the cache for each of the plurality of requests; and The multiple requests are sent sequentially, and the data corresponding to each request is written to the cache area based on its corresponding target write address.

2. The data loading method according to claim 1, wherein, The step of determining the pixel units loaded in each of the multiple requests for loading the scaling factor to be processed, based on the data storage format of the original scaling factor tensor in memory, includes: In response to the fact that the original scaling factor tensor is stored in memory in a linear layout format, it is determined that the pixel unit comprises the same number of consecutive data elements in the first dimension as the data block in the first dimension.

3. The data loading method according to claim 1, wherein, The original scaling factor tensor is stored in a block layout. Determining the target write address in the cache for each of the plurality of requests based on the base address and the data size of the pixel unit loaded in each request includes: The target write addresses corresponding to two consecutive requests are determined to be linearly increasing, and the offset between the target write addresses corresponding to two consecutive requests is equal to the number of bytes of one data block.

4. The data loading method according to claim 2, wherein, The original scaling factor tensor is stored in a linear layout format. Determining the target write address in the cache for each of the plurality of requests based on the base address and the data size of the pixel unit includes: The data of each requested pixel unit is continuously written to the cache along the second dimension until the amount of data continuously written along the second dimension is equal to the number of bytes of a data block; wherein the offset between the target write addresses of two consecutive requests is equal to the number of bytes of the data size of the pixel unit; In response to the data volume continuously written in the second dimension reaching the preset size of the data block in the second dimension, the base address is updated based on the total size of the scaling factor to be processed in the first dimension and the preset size of the data block, so that the next consecutive request will use the updated base address as the corresponding target write address.

5. The data loading method according to claim 4, wherein, For the same coordinate position of the scaling factor to be processed in the second dimension, the difference between the target write addresses of two adjacent data blocks along the first dimension in the cache is equal to the number of bytes of the data block.

6. The data loading method according to claim 5, wherein, The data loading method further includes: In response to a request targeting a scaling factor whose range crosses a boundary value-defined range, a state machine is used to generate multiple sub-requests based on the request, such that the data loaded in each sub-request either falls entirely within the data range of the original scaling factor tensor defined by the boundary value or entirely outside the data range of the original scaling factor tensor defined by the boundary value.

7. The data loading method according to claim 6, wherein, For valid sub-requests where all loaded data falls within the data range of the original scaling factor tensor defined by the boundary value: The base address of the target write address of the valid sub-request in the cache is equal to the starting base address of the data loaded by the valid sub-request in the second dimension, plus the amount of valid data before the data loaded by the valid sub-request in the first dimension divided by the preset size of the data block in the first dimension, and then multiplied by the total number of bytes of the data block; and the byte offset within the data block is performed based on the pixel unit loaded by the valid sub-request.

8. The data loading method according to claim 7, wherein, For invalid sub-requests where all loaded data is outside the data range of the original scaling factor tensor: The invalid sub-request is converted into an operation of writing a preset value to the target write address corresponding to the invalid sub-request. The base address of the target write address of the invalid sub-request in the cache is equal to the starting base address of the data loaded by the valid sub-request in the second dimension, plus the amount of valid data before the data loaded by the valid sub-request in the first dimension divided by the preset size of the data block in the first dimension, multiplied by the total number of bytes of the data block, and the byte offset within the data block is performed based on the pixel unit loaded by the invalid sub-request.

9. The data loading method according to claim 3, wherein, A state machine is used to determine multiple requests for loading the scaling factor to be processed, wherein: In response to the scaling factor to be processed being discontinuous in the first dimension, the state machine determines to divide the request in the second dimension. In response to the scaling factor to be processed being discontinuous in the second dimension, the state machine determines to divide the request in the third dimension; In this context, the data size loaded by each request is equal to the preset size of the data block, and the data loaded by each sub-request after division is either entirely within the data range of the original scaling factor tensor defined by the boundary value of the original scaling factor tensor or entirely outside the data range of the original scaling factor tensor defined by the boundary value.

10. The data loading method according to claim 9, wherein: In response to the state machine determining to use the requested partitioning in the second dimension, when the state machine determines the addressing coordinates in the memory, each increment of its coordinates in the second dimension is equal to the size of the data elements included in the data block in the second dimension.

11. The data loading method according to claim 10, wherein, When the state machine partitions requests in the second dimension, the termination conditions for its state loop transitions include: The amount of remaining data to be copied in the third dimension is not equal to 1; or, The amount of remaining data to be copied in the second dimension is not equal to the size of the data elements included in the data block in the second dimension.

12. The data loading method according to any one of claims 1 to 11, wherein, The original scaling factor tensor and the scaling factor to be processed include a channel dimension, a width dimension, and a batch dimension, wherein the first dimension is the channel dimension, the second dimension is the width dimension, and the third dimension is the batch dimension.

13. A data loading apparatus for loading a scaling factor to be processed from a raw scaling factor tensor in memory into a cache, wherein, The scaling factor to be processed is stored in the cache in units of data blocks. Each data block has a preset size, and data within the same data block is stored contiguously in the cache. The data loading device includes: The instruction acquisition module is configured to receive a data loading instruction for loading the scaling factor to be processed into the cache area, so as to determine the base address for writing the scaling factor to be processed into the cache area; The instruction parsing module is configured to determine the pixel unit loaded by each of a plurality of requests for loading the scaling factor to be processed, based on the data storage format of the original scaling factor tensor in memory. Specifically, for cases where the data storage format of the original scaling factor tensor in memory is a block layout in units of data blocks, the module determines that the pixel unit includes continuous data elements with the same preset size as the data block. Furthermore, the plurality of requests are used to sequentially load the pixel units in the original scaling factor tensor along a first dimension and write them to the cache in units of data blocks, wherein the size of the pixel unit in the first dimension is the same as the size of the data block in the first dimension. A request generation module is configured to, when the original scaling factor tensor adopts the block layout and the scaling factor to be processed is discontinuous in the first dimension, use a state machine to determine the plurality of requests for loading the scaling factor to be processed, wherein the state machine determines that the requests are divided in a second dimension, and determines the target write address of each of the plurality of requests in the cache based on the base address and the data size of the pixel unit; and The request execution module is configured to send the multiple requests sequentially and write the data corresponding to each request into the cache area based on its corresponding target write address.

14. A processor, comprising: An instruction parsing unit is configured to receive and parse a data loading instruction, wherein the data loading instruction instructs the execution of loading a scaling factor to be processed from memory into a cache; and The execution unit is configured to execute the data loading instruction. The execution unit executes the data loading instruction, including implementing the data loading method according to any one of claims 1-12, to store the scaling factor to be processed in the cache area in units of data blocks of a preset size.

15. An electronic device comprising: The processor according to claim 14; and the memory connected to the processor, The processor is configured to run computer-executable instructions, which, when executed by the processor, implement the data loading method according to any one of claims 1-12.

16. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions, wherein, When the computer-executable instructions are executed by the processor, they implement the data loading method according to any one of claims 1-12.

Images