Data loading methods, processors, electronic devices, and storage media

Abstract

Embodiments of this disclosure provide a data loading method, a processor, an electronic device, and a storage medium. The data loading method includes: acquiring padding information of a tensor to be processed, shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor, wherein the padding information indicates a padding mode when padding at specified edges of the tensor to be processed and padding parameters related to the padding mode; the tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a channel number dimension; determining multiple padding requests for loading padding data based on the padding information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, wherein the padding data is determined based on the padding mode and the padding parameters; sequentially sending the multiple padding requests, and writing the sub-data returned by each padding request into a buffer to load the padding data into the buffer. The padding mode includes a first padding mode and a second padding mode.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data loading method, a processor, an electronic device, and a storage medium. Background Technology

[0002] A tensor is a multilinear mapping defined on the Cartesian product of some vector spaces and some dual spaces. For example, a scalar can be viewed as a 0-dimensional tensor, a vector as a one-dimensional tensor, and a matrix as a two-dimensional tensor. Tensors can have any number of dimensions. Tensor operations are widely used in processors such as parallel processors.

[0003] With the development of artificial intelligence and machine learning, new requirements have been placed on numerous parallel processor devices, such as multi-core processors and digital signal processors. In general computing, the computing units of parallel processors require a large amount of data, which is typically stored in the parallel processor's storage components, such as memory. Data loading instructions can retrieve this data from the storage components to the cache for computation, and data storage instructions can store the data from the cache into memory. Summary of the Invention

[0004] At least one embodiment of this disclosure provides a data loading method, the method comprising: acquiring filling information of a tensor to be processed, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor, wherein the filling information is used to indicate a filling mode when filling a specified edge of the tensor to be processed and filling parameters related to the filling mode, the tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a channel number dimension; determining a plurality of filling requests for loading filling data based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, wherein the filling data is determined based on the filling mode and the filling parameters; sequentially sending the plurality of filling requests, and writing the sub-data returned by each filling request into a buffer to load the filling data into the buffer.

[0005] The filling mode includes a first filling mode and a second filling mode. The first filling mode indicates that the sub-data returned by each filling request comes from the tensor to be processed. The sub-data returned by each filling request is the data corresponding to a pixel in the tensor to be processed, which is determined based on the filling parameters and the filling mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second filling mode indicates that the sub-data returned by the filling request is not located in memory.

[0006] At least one embodiment of this disclosure provides a data loading method, the method comprising: receiving a data loading instruction, wherein the data loading instruction includes filling information of a tensor to be processed as input parameters, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor, wherein the filling information is used to indicate a filling mode when filling a specified edge of the tensor to be processed and filling parameters related to the filling mode, and the tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a channel number dimension; parsing the data loading instruction and then executing the data loading instruction using an execution unit, wherein executing the data loading instruction using the execution unit includes: according to the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor, the data loading instruction includes: receiving a data loading instruction, wherein the data loading instruction includes filling information of a tensor to be processed as input parameters ... The starting coordinates of the tensor are determined to identify multiple fill requests for loading fill data, wherein the fill data is determined based on the fill mode and the fill parameters. The multiple fill requests are sent sequentially, and the sub-data returned by each fill request is written to a buffer to load the fill data into the buffer. The fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

[0007] At least one embodiment of this disclosure provides a processor, including an instruction parsing unit and an execution unit. The instruction parsing unit is configured to receive and parse a data loading instruction, wherein the data loading instruction includes filling information of a tensor to be processed as input parameters, shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor. The filling information is used to indicate a filling mode when filling a specified edge of the tensor to be processed, and filling parameters related to the filling mode. The tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a number of channels dimension. The execution unit executes the data loading instruction after the instruction parsing unit parses the data loading instruction. When the execution unit executes the data loading instruction, it includes performing the following operations: according to the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor. The shape information of the tensor to be processed and its starting coordinates are given. Multiple fill requests are determined for loading fill data, wherein the fill data is determined based on the fill mode and the fill parameters. The multiple fill requests are sent sequentially, and the sub-data returned by each fill request is written into the buffer to load the fill data into the buffer. The fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is data corresponding to a pixel in the tensor to be processed, determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

[0008] At least one embodiment of this disclosure provides an electronic device, including: at least one processor; at least one memory including one or more computer program modules, wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the data loading method described in at least one embodiment above.

[0009] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed by at least one processor, perform the data loading method described in the at least one embodiment above.

[0010] In at least one embodiment of the data loading method provided in this disclosure, a hardware-level automatic filling implementation scheme is proposed. This scheme can automatically complete the filling operation during the tensor data copying process (e.g., loading from memory to the cache), without relying on additional software configuration or complex user instruction configuration. This significantly reduces programming complexity and control overhead, while greatly improving the bandwidth and efficiency during data memory access, thereby improving the hardware utilization of the computing unit and enhancing hardware performance. Furthermore, the data loading method provided in this disclosure supports multiple different filling modes, which can flexibly adapt to diverse computing needs. This scheme provides efficient and low-latency data loading support for application scenarios sensitive to data layout, such as deep learning inference, image processing, and high-performance computing, significantly improving the overall system performance. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0012] Figure 1 This is a schematic diagram of the structure of a general-purpose graphics processing unit (GPGPU);

[0013] Figure 2A This is a schematic structure of a tensor;

[0014] Figure 2B This is a schematic diagram of the NDHWC storage format;

[0015] Figure 2C A schematic diagram of the N(C / 32)DHW(32C) storage format;

[0016] Figure 3 A schematic flowchart illustrating a data loading method provided in at least one embodiment of this disclosure;

[0017] Figure 4A A schematic structural diagram of the original tensor provided for at least one embodiment of this disclosure;

[0018] Figure 4B This is a schematic diagram of a tensor to be processed provided in an embodiment of the present disclosure;

[0019] Figure 4C A schematic diagram of a tensor to be processed provided in another embodiment of this disclosure;

[0020] Figure 5A A schematic diagram of the tensor to be processed provided in at least one embodiment of this disclosure;

[0021] Figure 5B A schematic diagram illustrating a copy-fill pattern provided for at least one embodiment of this disclosure;

[0022] Figure 5C A schematic diagram illustrating a mirror-fill pattern provided for at least one embodiment of this disclosure;

[0023] Figure 5D A schematic diagram illustrating a polymer fill pattern provided for at least one embodiment of this disclosure;

[0024] Figure 5E A schematic diagram of a first internal padding sub-pattern provided for at least one embodiment of this disclosure;

[0025] Figure 5F A schematic diagram of a second boundary filling sub-pattern provided for at least one embodiment of this disclosure;

[0026] Figure 6 A schematic flowchart illustrating a data loading method provided in at least one embodiment of this disclosure;

[0027] Figure 7 A schematic block diagram of an electronic device provided in one embodiment of this disclosure;

[0028] Figure 8 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure;

[0029] Figure 9 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0031] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0032] Figure 1 This is a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0033] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0034] like Figure 1 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple cores (also called computational kernels), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These cores are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 1The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0035] like Figure 1 As shown, each streaming processor cluster also provides a buffer for caching data across the N computing units within the streaming processor cluster.

[0036] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0037] In each computing unit, the thread beam scheduling / distribution module ( Figure 1 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 1 High Bandwidth Memory (HBM) is used for read and write operations.

[0038] like Figure 1 As shown, general computational operations, such as matrix calculations or convolution operations, typically require large amounts of data. This data is usually stored in memory, such as in HBM. During general computational operations, data needs to be loaded from memory (Load operation), and when the calculation result is obtained, data needs to be stored back into memory (Store operation). The way data is stored in memory affects memory access bandwidth, which in turn affects the hardware utilization of the computing unit.

[0039] For example, the shape and size of a tensor can be represented by a1, a2, a3, a4, and a5, where a1, a2, a3, a4, and a5 indicate the size of the tensor data in five dimensions, and a1, a2, a3, a4, and a5 are positive integers. For example, the five dimensions include [N, D, H, W, C], where N represents the batch size (the number of data samples captured in one training iteration), D represents the depth, H represents the height, W represents the width, and C represents the number of channels. For example, taking the first tensor as an example, a1 can be an N-dimensional tensor, a2 can be a D-dimensional tensor, a3 can be an H-dimensional tensor, a4 can be a W-dimensional tensor, and a5 can be a C-dimensional tensor. Of course, this disclosure does not impose specific limitations on this. It should be noted that although this description uses a tensor containing five specific dimensions as an example, the embodiments of this disclosure are not limited thereto. In the embodiments of this disclosure, the number of dimensions of a tensor can be flexibly set according to specific needs. That is, a tensor may include more or fewer dimensions, and the arrangement order and size of each dimension are not limited.

[0040] For example, general computational operations can include General Matrix Multiplication (GEMM). The data that needs to be multiplied by GEMM includes a first tensor and a second tensor.

[0041] For example, general computational operations can include convolution operations. The data to be convolved includes both the convolution kernel and the input data (activation). For example, the input data can be an image that needs to be convolved. Convolution operations can include convolution multiplication calculations, such as those used in neural network processing (e.g., convolutional neural networks) in fields such as artificial intelligence, such as multi-channel convolution and multi-dimensional convolution.

[0042] The elements of a tensor are in memory (e.g., Figure 1 Tensors can be arranged in various formats in memory, called data storage layouts. Data storage layouts are used to indicate the storage order and dimensional arrangement of tensors in storage components.

[0043] For example, Figure 2A This is a schematic structure of a tensor. Figure 2A In the tensors shown, a1 has an N-dimensional dimension and equals 1, a2 has a D-dimensional dimension and equals 1, a3 has an H-dimensional dimension and equals 5, a4 has a W-dimensional dimension and equals 4, and a5 has a C-dimensional dimension and equals 64. It should be noted that... Figure 2A This is merely an example, and the embodiments disclosed herein do not limit the number, order, or size of the dimensions of tensors.

[0044] For example, Figure 2AThe pixel elements of the tensor in the model are represented as 0, 1, 2, 3, ..., and so on. The following uses... Figure 2A The five-dimensional tensor shown is used as an example to describe different data storage formats.

[0045] For example, data storage formats may include NDHWC, also known as Linear mode. Figure 2B This is a schematic diagram of the NDHWC storage format.

[0046] For example, for NDHWC, such as Figure 2B As shown, from the first channel (a5=0, Figure 2B The first element of c0 in ( Figure 2B Start with element 0 in the first channel, then store the second channel (a5=1, ...). Figure 2B The first element of c1 in ( Figure 2B Element 20 in the middle), and so on, until the first element of all channels is laid out, for example, to the 64th channel (a5=63, Figure 2B The first element of c63 in ( Figure 2B After element 1260 in the middle, select the first channel (a5=0, Figure 2B The second element of c0) Figure 2B Element 1 in the middle), then store the second channel (a5=1, Figure 2B The second element of c1) Figure 2B (element 21 in the middle), and so on, until the second element of all channels is laid out, and so on.

[0047] For example, data storage formats can also include N(C / x)DHW(xC), also known as interleave mode, where x can be 8, 16, 32, etc., as needed.

[0048] N(C / x)DHW(xC) is similar to NDHWC, but there is a key difference: in the memory layout of N(C / x)DHW(xC), the a5 channels are divided into a5 / x groups, with x channels in each group: the first group consists of channels a5=0 to a5=x-1, and the second group consists of channels a5=x to a5=2x-1. Each group is arranged in the NDHWC format.

[0049] Figure 2C This is a schematic diagram of the storage format of N(C / 32)DHW(32C).

[0050] like Figure 2C As shown, the 64 channels are divided into two groups of 32 channels each. The first group consists of channel a5=0 ( Figure 2C From c0 to a5=31 ( Figure 2CThe first group consists of c31), and the second group consists of channels a5=32 to a5=63. Then each group is arranged in NDHWC format.

[0051] It should be noted that, Figure 2B and Figure 2C Using a five-dimensional tensor as an example, two data storage formats are shown to illustrate different ways of organizing multidimensional tensors in memory. However, the embodiments disclosed herein are not limited to this, nor are they limited to the scenario of five-dimensional tensors. For example, for a three-dimensional tensor including H, W, and C dimensions, the HWC data storage format can be used, that is, all channel values ​​at the same spatial location (H, W) are stored contiguously.

[0052] Because multidimensional data often employs complex data storage formats, its physical distribution in memory is frequently inconsistent with its logical structure. When extracting or storing portions of the original tensor, data across different dimensions may not be accessible contiguously, resulting in non-consecutive memory access. This non-consecutive access pattern makes it difficult to accurately determine the requested data volume and location during data loading or storage, thus significantly reducing data bandwidth and hardware computing efficiency.

[0053] In practical applications, because computational units or subsequent processing modules have specific requirements for the size of input data, the extracted tensors often need to be padded to meet the format constraints and performance requirements of subsequent computational tasks. For example, in convolutional neural networks, convolution operations often result in the output feature map being smaller than the input feature map, leading to a loss of spatial information. By adding appropriate padding (such as zero padding) to the tensor boundaries, the spatial size consistency between the input and output feature maps can be effectively maintained.

[0054] Furthermore, padding helps preserve the boundary information of the input data. Without padding, the convolution kernel can only cover the internal region of the input feature map, and edge pixels participate in the computation less frequently, resulting in the loss of edge semantic information. By introducing boundary padding, edge region data can also fully participate in the convolution operation, improving the model's ability to perceive edge features such as contours and textures.

[0055] In deep network architectures, successive downsampling operations (such as pooling or convolution) accelerate the decrease in feature map resolution. Appropriate padding strategies can slow down the reduction in spatial dimensions, allowing higher-level networks to retain sufficient spatial details. This enhances the ability to recognize small objects, fine-grained structures, or complex shapes, and improves the overall stability of feature extraction.

[0056] Traditional approaches typically rely on software configuration to perform data padding operations, such as writing additional code to explicitly pad tensor boundaries with zeros or predefined values. This approach not only introduces redundant data processing steps and increases memory access overhead, leading to decreased efficiency in tensor data copying (i.e., loading or storing), but also significantly increases the complexity and maintenance cost of software programming. This problem is particularly pronounced when dealing with non-consecutive memory access or complex data layouts, impacting overall system performance.

[0057] Therefore, how to ensure efficient tensor data copying while supporting flexible and configurable filling operations at the hardware level, and realizing automatic filling operations in continuous or non-continuous access scenarios, has become an urgent problem to be solved in order to improve the overall performance of computing systems.

[0058] This disclosure provides at least one embodiment of a data loading method, a processor, an electronic device, and a non-transitory computer-readable storage medium.

[0059] The data loading method provided in at least one embodiment of this disclosure includes: acquiring filling information of a tensor to be processed, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed in a coordinate system determined by an original tensor, wherein the filling information is used to indicate the filling mode when filling at a specified edge of the tensor to be processed and the filling parameters related to the filling mode, and the tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a channel number dimension; determining multiple filling requests for loading filling data based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, wherein the filling data is determined based on the filling mode and the filling parameters; sequentially sending multiple filling requests, and writing the sub-data returned by each filling request into a buffer to load the filling data into the buffer.

[0060] The filling mode includes a first filling mode and a second filling mode. The first filling mode indicates that the sub-data returned by each filling request comes from the tensor to be processed. The sub-data returned by each filling request is the data corresponding to a pixel in the tensor to be processed, which is determined based on the filling parameters and the filling mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second filling mode indicates that the sub-data returned by the filling request is not located in memory.

[0061] In at least one embodiment of the data loading method provided in this disclosure, a hardware-level automatic filling implementation scheme is proposed. This scheme can automatically complete the filling operation during the tensor data copying process (e.g., loading from memory to the cache), without relying on additional software configuration or complex user instruction configuration. This significantly reduces programming complexity and control overhead, while greatly improving the bandwidth and efficiency during data memory access, thereby improving the hardware utilization of the computing unit and enhancing hardware performance. Furthermore, the data loading method provided in this disclosure supports multiple different filling modes, which can flexibly adapt to diverse computing needs. This scheme provides efficient and low-latency data loading support for application scenarios sensitive to data layout, such as deep learning inference, image processing, and high-performance computing, significantly improving the overall system performance.

[0062] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0063] Figure 3 This is a schematic flowchart illustrating a data loading method provided in at least one embodiment of the present disclosure.

[0064] like Figure 3 As shown, the data loading method provided in at least one embodiment of this disclosure includes at least steps S10-S30.

[0065] For example, at least one embodiment of this disclosure provides a data loading method for loading fill data into a cache, such as a buffer in a streaming processor cluster.

[0066] Step S10: Obtain the filling information of the tensor to be processed, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor.

[0067] For example, the original tensor is stored in memory, and the tensor to be processed can be a part of the original tensor, or the tensor to be processed can be the original tensor itself.

[0068] For example, the tensor to be processed and the original tensor include at least H, W, and C dimensions. It should be noted that the tensor to be processed and the original tensor in this disclosure embodiment may also include more dimensions, such as N or D dimensions, thereby forming a four-dimensional, five-dimensional, or higher-dimensional tensor structure. The embodiments of this disclosure are applicable to such multi-dimensional tensor filling scenarios and have good scalability.

[0069] For example, taking a three-dimensional tensor (including H, W, and C dimensions) as both the tensor to be processed and the original tensor, the HWC data storage format can be used. This means that for a given coordinate (H, W), all its corresponding channel values ​​(C) are arranged consecutively and sequentially in memory. For a more intuitive understanding of this storage format, please refer to... Figure 2B The diagram shows the NDHWC storage format. Although... Figure 2B This describes a five-dimensional tensor that further includes N and D dimensions, but its low-dimensional data arrangement logic is consistent with the HWC storage format. Ignoring the N and D dimensions reveals the specific storage method of the HWC storage format.

[0070] The following will be through Figures 4A-4C The original tensor and the tensor to be processed involved in the embodiments of this disclosure are further introduced to explain the meaning of "shape information of the tensor to be processed" and "starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor" in step S10.

[0071] Figure 4A A schematic structural diagram of the original tensor provided for at least one embodiment of this disclosure.

[0072] Figure 4A In this tensor, each solid-line cube represents an element, and the tensor composed of multiple solid-line cubes is the original tensor.

[0073] Although Figure 4A Only three dimensions are shown, but this can be extended to higher-dimensional tensors. Taking a five-dimensional tensor as an example... Figure 4A This diagram shows a batch and a tensor at a certain depth within a batch. However, the original tensor may contain multiple batches, and each batch may contain multiple tensors in the depth dimension, with a structure similar to... Figure 4A The same applies as shown, so it will not be described again here.

[0074] Assumption Figure 4A The element pointed to by the arrow is the origin of the preset coordinate system of the original tensor. C, W, and H represent the three coordinate axes, respectively. row is the coordinate value in the H dimension, col is the coordinate value in the W dimension, and c is the coordinate value in the C dimension. The coordinates of the element pointed to by the arrow in the preset coordinate system are: c=0, row=0, and col=0. The channel, width, and height values ​​in the coordinates of other elements increase in the direction of the arrow. Of course, the original tensor can also include higher-dimensional coordinates, such as D-dimensional and N-dimensional coordinates, which are not shown here.

[0075] For example, if the original tensor is a three-dimensional tensor, then the shape and size of the original tensor are represented by tensor_c, tensor_w, and tensor_h. Here, tensor_h is the H-dimensional dimension, tensor_w is the W-dimensional dimension, and tensor_c is the C-dimensional dimension. Figure 4A In the example shown, tensor_h=4, tensor_w=7, and tensor_c=8.

[0076] It should be noted that in this disclosure, the original tensor and the tensor to be processed use the same coordinate system and are both preset coordinate systems.

[0077] For example, a preset coordinate system can be determined by using an element in the original tensor as the origin of the coordinate system, for example, referring to... Figures 4B-4C In all embodiments, the top-left vertex of the original tensor is used as the origin of the preset coordinate system. Of course, this disclosure is not limited thereto.

[0078] For example, the initial coordinates of the tensor to be processed in a preset coordinate system determined by the original tensor are, for example, as follows: Figures 4B-4C In the example, the coordinates of the top-left vertex of the tensor to be processed are given. The starting coordinates of the tensor to be processed are the minimum coordinates among all elements in the tensor.

[0079] Figure 4B This is a schematic diagram of a tensor to be processed provided in an embodiment of this disclosure.

[0080] As an example, Figure 4B A schematic diagram illustrating the acquisition of a tensor to be processed in a non-continuous manner according to an embodiment of the present disclosure is shown. Figure 4B In the diagram, each solid-line cube with a white background and a black border represents an element stored in memory that belongs to the original tensor. A tensor composed of multiple solid-line cubes is the original tensor. Figure 4B The tensor to be processed shown can be used, for example, to perform matrix calculation operations.

[0081] Although Figure 4B Only three dimensions are shown, but this can be extended to higher-dimensional tensors. Taking a five-dimensional tensor as an example... Figure 4B This diagram shows a batch and a tensor at a certain depth within a batch. However, the original tensor may contain multiple batches, and each batch may contain multiple tensors in the depth dimension, with a structure similar to... Figure 4B The same applies as shown, so it will not be described again here.

[0082] For example, in Figure 4B In the example, the dashed box represents the tensor to be processed, which includes a portion of the elements from the original tensor. Figure 4B In the example, the top-left corner of the tensor to be processed has coordinates c=0, row=0, col=3, and the dimensions of C dimension are 5, W dimension is 3, and H dimension is 3.

[0083] Figure 4C This is a schematic diagram of a tensor to be processed, provided for another embodiment of this disclosure.

[0084] As an example, Figure 4C A schematic diagram illustrating the sequential acquisition of tensors to be processed according to embodiments of the present disclosure is shown. Figure 4CIn the diagram, each tensor element is shown as a square, and the original tensor covers all the solid-line squares (i.e., it includes both white squares and gray-shaded squares). Figure 4C The tensor to be processed shown can be used, for example, to perform convolution calculations.

[0085] The initial coordinates of this original tensor are represented as (0, 0, 0), which means c=0, row=0, col=0. Then, in... Figure 4C In the diagram, the gray-shaded squares represent the tensors to be processed, with their starting coordinates represented as (c=0, row=0, col=2). This means data is acquired starting from the second pixel in the first row of the original tensor. Then, in... Figure 4C The diagram schematically illustrates the process of acquiring the tensor to be processed in a sequential (or continuous) manner according to an embodiment of this disclosure. Specifically, as shown by the dashed arrow, starting from the starting point (c=0, row=0, col=2), data is acquired from the original tensor, for example, pixel-by-pixel, until the number of acquired pixels reaches copy_pixel_num. It should be noted that in this embodiment, pixels are a tensor description method that does not consider the C-dimensional dimension; that is, the C-dimensional dimension does not affect the number of pixels.

[0086] exist Figure 4C In the example, the shape information of the tensor to be processed includes the number of pixels included in the tensor to be processed (e.g., denoted as copy_pixel_num). Figure 4C Only three dimensions, H, W, and C, are shown. It's understandable that if the data in these three dimensions still doesn't reach the number of pixels included in the tensor to be processed, higher-dimensional data can be obtained; this is not limited here. Furthermore, as... Figure 4C As shown, dimension C itself does not affect the number of pixels. Assuming... Figure 4C The original tensors in the memory are stored in HWC format. Figure 4C In the example shown, the total number of pixels in the tensor to be processed is copy_pixel_num=24.

[0087] For example Figure 4B In cases where the tensor to be processed is obtained in a non-continuous manner, the shape information of the tensor to be processed can be, for example, the shape and size of the tensor in various dimensions (e.g., H-dimension, W-dimension, C-dimension); for example... Figure 4C The example shown illustrates the scenario where tensors to be processed are acquired sequentially. The shape information of the tensors to be processed can be, for example, the number of pixels to be loaded.

[0088] For example, padding information is used to indicate the padding mode and padding parameters associated with padding mode when padding is performed at specified edges of the tensor to be processed. For example, the specified edges may include the boundaries of the tensor to be processed, or they may include the internal pixel boundaries of the tensor to be processed.

[0089] For example, the fill pattern can include a first fill pattern and a second fill pattern.

[0090] For example, the first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, which is determined based on the fill parameters and fill mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions.

[0091] For example, the second padding mode indicates that the sub-data returned by the padding request is not in memory.

[0092] It should be noted that the first and second fill modes can be further subdivided into more fill modes, which will be introduced in detail later.

[0093] Step S20: Based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, determine multiple filling requests for loading the filling data.

[0094] For example, based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed obtained in step S10, multiple filling requests for loading filling data can be determined. For example, the filling data can be determined based on the filling mode and filling parameters. That is, in this embodiment of the disclosure, the filling data may be different under different filling modes and filling parameters.

[0095] For example, each fill request may include a data read address indicating the starting position for reading data from memory, a data write address indicating the starting position for writing data to the buffer, and the length of data loaded by the fill request.

[0096] Step S30: Send multiple fill requests sequentially, and write the sub-data returned by each fill request into the cache area to load the fill data into the cache area.

[0097] For example, a cache is a buffer in a cluster of streaming processors.

[0098] It should be noted that there is an order relationship between the fill requests, but these fill requests are not required to be sent consecutively; correspondingly, the write positions of the sub-data returned by each fill request in the buffer do not need to be adjacent to each other.

[0099] The data loading method provided in at least one embodiment of this disclosure may further include step S40.

[0100] Step S40: Based on the shape information of the tensor to be processed, the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor, and the shape information of the original tensor, determine multiple tensor loading requests for loading the tensor to be processed into the buffer.

[0101] For example, tensor loading requests can be divided along the width or height dimension of the original tensor. The sub-data returned by each tensor loading request is located in memory, and the sub-data returned by multiple tensor loading requests come from the original tensor and are stored contiguously in memory.

[0102] The specific methods for dividing tensor loading requests can be found in the description above, and will not be repeated here.

[0103] The data loading method provided in at least one embodiment of this disclosure may further include step S50.

[0104] Step S50: Send multiple tensor loading requests sequentially, and write the sub-data returned by each tensor loading request into the buffer to load the tensor to be processed into the buffer.

[0105] It should be noted that there is an order relationship between the tensor load requests, but these tensor load requests are not required to be sent consecutively; correspondingly, the write positions of the sub-data returned by each tensor load request in the buffer do not need to be adjacent to each other.

[0106] In this embodiment of the disclosure, the fill request for loading fill data into the buffer and the tensor load request for loading tensors to be processed into the buffer may be arranged alternately, and the arrangement order varies depending on the fill mode.

[0107] The embodiments of this disclosure split the tensor to be processed into multiple different tensor loading requests according to a certain rule and send them, and write the returned data into the cache area, thereby efficiently loading the tensor to be processed into memory.

[0108] During the splitting process, if the size relationship between the tensor to be processed and the original tensor in the first dimension makes it impossible to load the tensor to be processed continuously in the first dimension but it is possible to load it continuously in all dimensions below the first dimension, then further requests are split in the second dimension. This allows for a more reasonable splitting of requests, preserving as much data as possible that is stored continuously in memory, reducing the number of requests, efficiently loading the tensor to be processed, significantly increasing the bandwidth when loading or storing data from memory, improving performance, and increasing the efficiency of the hardware computing unit.

[0109] For example, the coordinates of the sub-data returned by each request satisfy the following conditions: the coordinates of the tensor to be processed are different in the first dimension and all dimensions below the first dimension, but the coordinates are the same in the second dimension and all dimensions above the second dimension, wherein the data storage format of the tensor to be processed indicates that the first dimension takes precedence over the second dimension when stored or loaded, and the first dimension and the second dimension are adjacent.

[0110] For example, in response to the tensor to be processed having a size copy_t in the first dimension that is not equal to the original tensor having a size tensor_t in the first dimension, and / or the tensor to be processed having a starting coordinate value t_coord_b in the first dimension that is not equal to the original tensor having a starting coordinate value in the first dimension, it is determined that the tensor to be processed cannot be loaded continuously in the first dimension.

[0111] Taking data storage format NDHWC and second coordinate value 0 as an example, assuming the first dimension is dimension C and the second dimension is dimension W, it is determined that the data is discontinuous in dimension C when any of the following conditions are met:

[0112] (1) The initial coordinates c_coord_b of the tensor to be processed in dimension C are not equal to 0.

[0113] (2) The size of the tensor to be processed in dimension C, copy_c, is greater than the size of the original tensor in dimension C, tensor_c.

[0114] (3) The size of the tensor to be processed in dimension C, copy_c, is smaller than the size of the original tensor in dimension C, tensor_c.

[0115] For example, the sub-data returned by each request may have different coordinates in the C dimension, but the same coordinates in the W, H, D, and N dimensions. In this case, it can be understood that the original tensor is expanded into a one-dimensional vector composed of multiple pixels in the WHDN order without considering the depth direction. The sub-data returned by each request is the data within one pixel. That is, the request is divided in the W dimension. The next request may load the data in the next adjacent pixel, that is, the request is divided in a pixel-skipping manner.

[0116] Taking the data storage format as NDHWC and the second coordinate value as 0 as an example, assuming the first dimension is W and the second dimension is H, it is determined that the data is discontinuous in the W dimension when any of the following conditions are met:

[0117] (1) The initial coordinates of the tensor to be processed in dimension W are not equal to 0.

[0118] (2) The size of the tensor to be processed in the W dimension, copy_w, is greater than the size of the original tensor in the W dimension, tensor_w.

[0119] (3) The size of the tensor to be processed in the W dimension, copy_w, is smaller than the size of the original tensor in the W dimension, tensor_w.

[0120] At this point, when dimension C is continuous, the coordinates of the sub-data returned by each request are different in dimensions C and W, but the coordinates are the same in dimensions H, D, and N. This can be understood as the sub-data returned by each request belonging to the same row; that is, when the requests are divided along dimension H, the sub-data returned by different requests are located in different rows.

[0121] In this embodiment of the disclosure, taking NDHWC data storage format as an example, the second dimension can be either the W dimension or the H dimension, and correspondingly, the first dimension can be either the C dimension or the W dimension. That is, this embodiment of the disclosure can divide the tensor loading request on the W dimension or the H dimension of the original tensor.

[0122] For example, when dividing tensor loading requests along the W dimension, the next tensor loading request can load data from the next adjacent pixel, that is, the requests are divided in a pixel-skipping manner; when dividing tensor loading requests along the H dimension, the sub-data returned by the same tensor loading request belongs to the same row, and the sub-data returned by different tensor loading requests are located in different rows.

[0123] The request partitioning logic for tensors in data storage format N(C / x)DHW(xC) is the same as that of NDHWC. The difference lies in the dimension arrangement of N(C / x)DHW(xC). For N(C / x)DHW(xC), due to its special interleaving structure, it is assumed that (xC) is necessarily continuous. Therefore, the lowest dimension is considered to be W, followed by H, then D, then C, and the highest dimension is N. For the conditions under which continuous loading is not allowed when data storage format is N(C / x)DHW(xC) and the request partitioning principles, please refer to the relevant content of NDHWC, which will not be listed in detail here.

[0124] For example, each request (fill request and tensor load request) may include the following parameters: data read address indicating the starting position for reading data from memory, data write address indicating the starting position for writing data to the buffer, and the length of the loaded data.

[0125] The following section details how multiple fill requests are determined for loading fill data.

[0126] For example, in some embodiments, step S20 may include steps S101 to S103.

[0127] Step S101: In response to the first filling mode, determine the data read address for each filling request based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed; in response to the second filling mode, determine the sub-data returned by each filling request.

[0128] Step S102: Determine the data length to be loaded for each filling request based on the filling information and the shape information of the tensor to be processed.

[0129] Step S103: Determine the data write address for each fill request based on the fill information and the data length loaded for each fill request.

[0130] In the first padding mode, the padding operation can be understood as writing certain data from the tensor to be processed to a specified location in the buffer to achieve tensor boundary expansion or tensor size expansion, etc. That is, in the first padding mode, the sub-data returned by each padding request comes from the tensor to be processed, and is the data corresponding to a pixel in the tensor to be processed, determined based on the padding parameters and padding mode. Therefore, it is necessary to determine the data read address of the starting position from which each padding request reads data from memory. The first padding mode is suitable for padding scenarios such as copy padding and mirror padding.

[0131] For example, one example of step S101, "in response to the first filling mode, determine the data read address of each filling request based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed," may include: in response to the first filling mode, performing the following steps: determining the request initial coordinates corresponding to each filling request based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed; determining the data read address of each filling request based on the shape information of the original tensor and the request initial coordinates corresponding to each filling request.

[0132] For example, the initial coordinates of each fill request can be determined based on the fill parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor. The fill parameters can be used to determine, for example, which boundary of the tensor to be processed to fill, or to determine the fill size. Combining these with the shape information and starting coordinates of the tensor to be processed, the initial coordinates of each fill request can be determined. The initial coordinates of the fill request are used to determine the data read address for the fill request.

[0133] In some embodiments, an example of "determining the data read address for each fill request based on the shape information of the original tensor and the initial coordinates of each fill request" is as follows:

[0134] For example, taking NDHWC as the data storage format, the formula for calculating the data read address Addr_1 of the request (including padding requests and tensor loading requests) is as follows:

[0135] Addr_1=u_addr_base+((((n_coord×tensor_d+d_coord) ×tensor_h+h_coord)×tensor_w+ w_coord) ×tensor_c+c_coord)

[0136] Where u_addr_base represents the memory address of the element at the starting coordinate position of the original tensor, n_coord, d_coord, h_coord, w_coord, and c_coord represent the initial coordinates of the request corresponding to the nth request, and tensor_n, tensor_d, tensor_h, tensor_w, and tensor_c represent the shape and size of the original tensor in five dimensions.

[0137] For example, taking the data storage format N(C / x)DHW(xC) as an example, the formula for calculating the data read address Addr_3 of the request (including padding requests and tensor loading requests) is as follows:

[0138] Addr_3=u_addr_base+n_coord×tensor_c×tensor_d×tensor_h×tensor_w+c_coord×tensor_d×tensor_h×tensor_w+d_coord×tensor_h×tensor_w×xC+h_coord×tensor_w×xC+w_coord ×xC

[0139] Where xC represents the size occupied by x channels. The definitions of other parameters are the same as the calculation formula for Addr_1, and the repetitions will not be repeated.

[0140] For example, one example of step S102 may include: in response to a first padding mode, determining that the length of the data loaded for each padding request is the size of the tensor to be processed in the number of channels dimension.

[0141] In other words, in the first padding mode, the data length loaded by each padding request corresponds to the full channel data length contained in a single pixel of the tensor to be processed. For example, for a tensor to be processed with c0 channels, the data length loaded by each padding request is c0. Figure 4B In the example, the tensor to be processed has a size of 5 in the C dimension, so in the case of the first padding mode, it can be determined that the length of data loaded for each padding request is 5.

[0142] In some examples, the first padding pattern may include a first boundary padding sub-pattern. For example, the first boundary padding sub-pattern indicates that data derived from the tensor to be processed is padded at a specified edge of the tensor to be processed, where the specified edge is the boundary of the tensor to be processed.

[0143] For example, in some embodiments, one example of "determining the request initial coordinates corresponding to each fill request based on the fill parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed" may include: in response to a first boundary fill sub-mode, determining the request initial coordinates corresponding to each fill request based on the fill parameters and the target boundary pixel coordinates of the tensor to be processed.

[0144] For example, the target boundary pixel coordinates of the tensor to be processed are determined based on the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed.

[0145] For example, the target boundary of the tensor to be processed can be directly indicated by the padding information, thereby calculating the target boundary pixel coordinates of the tensor to be processed based on the size of the tensor in the width and height dimensions and the starting coordinates of the tensor to be processed.

[0146] For example, the target boundary pixel coordinates of the tensor to be processed can be determined based on its dimensions in the width and height dimensions, its starting coordinates, and the boundary information of the original tensor (which can be determined by the shape and size of the original tensor). For instance, the target boundary of the tensor to be processed can be a boundary that coincides with the boundary of the original tensor. Assuming the left boundary of the tensor to be processed coincides with the left boundary of the original tensor, then the target boundary of the tensor to be processed is its left boundary, and the pixel coordinates of the target boundary are, for example, the left boundary pixel coordinates (the leftmost column of pixel coordinates). Similarly, assuming the right boundary of the tensor to be processed coincides with the right boundary of the original tensor, then the target boundary of the tensor to be processed is its right boundary, and the pixel coordinates of the target boundary are, for example, the right boundary pixel coordinates (the rightmost column of pixel coordinates).

[0147] The coordinates mentioned above can all be calculated based on the dimensions of the tensor to be processed in the width and height dimensions, as well as the starting coordinates of the tensor. For example, assuming the starting coordinates of the tensor to be processed are (row0, col0), its width dimension is w0, and its height dimension is h0, then the right boundary column index of the tensor to be processed is col0+w0-1, and the left boundary column index is col0. Therefore, the set of right boundary pixel coordinates of the tensor to be processed is {(r, col0+w0-1) | r=row0,row0+1, ..., row0+h0-1}, and the set of left boundary pixel coordinates of the tensor to be processed is {(r, col0) | r=row0,row0+1, …, row0+h0-1}.

[0148] Figure 5A This is a schematic diagram of a tensor to be processed provided for at least one embodiment of the present disclosure.

[0149] It should be noted that, Figure 5A The logical layout of the tensor to be processed in the H dimension and W width is shown schematically in only two dimensions. The purpose is to illustrate the logical relative positional relationship between the tensor to be processed and the filling area during the data loading process. Figure 5A Dimension C is not shown, meaning that the channel data contained in each spatial location is not represented. In other words, Figure 5A This is for simplification only and does not represent the complete multidimensional structure of the data when it is actually stored or loaded.

[0150] For example, in Figure 5A In the example, since the right boundary of the tensor to be processed coincides with the right boundary of the original tensor, the target boundary of the tensor to be processed can be determined as the right boundary of the tensor to be processed, and the pixel coordinates of the target boundary of the tensor to be processed are the rightmost column of pixel coordinates. Figure 5A In the tensor to be processed, the starting coordinates are (row=2, col=5). The size of the tensor to be processed is 5 in the width dimension and 3 in the height dimension. Based on the above information, it can be determined that there are three target boundary pixels of the tensor to be processed, with coordinates of (row=2, col=9), (row=3, col=9), and (row=4, col=9).

[0151] For example, in Figure 5A In the example, the target boundary of the tensor to be processed can also be indicated by the padding information as the right boundary, instead of determining it by the positions of the original tensor and the tensor to be processed. The calculation method for the pixel coordinates of the target boundary is the same as above, and will not be repeated here.

[0152] For example, in the first boundary padding sub-mode, the initial coordinates of each padding request can be determined based on the padding parameters and the target boundary pixel coordinates of the tensor to be processed, which are derived in accordance with the method described above.

[0153] For example, the padding parameter can include a first quantity, which indicates the number of padding requests that need to be generated in the width dimension. For example, in Figure 5A In the example, the first quantity is 2.

[0154] For example, in some embodiments, one example of "determining the initial coordinates of each fill request based on the fill parameters and the target boundary pixel coordinates of the tensor to be processed" may include:

[0155] In response to the first boundary padding sub-mode, for each row of the tensor to be processed, perform the following operations: determine the initial request coordinates of the first padding request in the current row based on the target boundary pixel coordinates of the tensor to be processed in the current row; in response to the number of padding requests generated in the current row not reaching a first number, update the initial request coordinates of the next padding request based on the initial request coordinates of the current padding request.

[0156] For example, depending on the actual needs, the target boundary pixel coordinates of the tensor to be processed in the current row can be directly used as the initial coordinates of the first fill request in the current row; the pixel coordinates adjacent to the target boundary pixel coordinates of the tensor to be processed in the current row in the W dimension can be used as the initial coordinates of the first fill request in the current row; alternatively, a new coordinate can be calculated based on the target boundary pixel coordinates of the tensor to be processed in the current row and a first quantity, and this new coordinate can be used as the initial coordinates of the first fill request in the current row. It should be noted that "adjacent pixels" here and below refers to pixels inside the tensor to be processed, that is, pixels that cannot exceed the range of the tensor to be processed.

[0157] For example, if the target boundary is the right boundary, the initial coordinates of the first fill request in the current row can be the target boundary pixel coordinates (r, col0+w0-1) of the tensor to be processed in the current row, or the pixel coordinates (r, col0+w0-2) that are adjacent to the target boundary pixel coordinates of the tensor to be processed in the current row in the W dimension.

[0158] For example, if the target boundary is the left boundary, the initial coordinates of the first fill request in the current row can be the target boundary pixel coordinates (r, col0) of the tensor to be processed in the current row, or it can be a new coordinate (r, col0+n0) calculated based on the target boundary pixel coordinates of the tensor to be processed in the current row and the first quantity, where n0 is the first quantity.

[0159] For example, if the number of fill requests already generated in the current row is less than the first number, then subsequent fill requests need to be generated, and the initial coordinates of the next fill request are updated based on the initial coordinates of the current fill request. Suppose the first number is 3, and the number of fill requests already generated in the current row is 1, which is less than the first number, therefore a second and third fill request need to be generated. It should be noted that the value of the first number mentioned above is just an example; depending on the actual needs, the first number can be different positive integers.

[0160] For example, in some embodiments, one example of "updating the initial coordinates of the next fill request based on the initial coordinates of the current fill request" may include setting the initial coordinates of the next fill request to the initial coordinates of the current fill request. This example applies to copy fill mode.

[0161] For example, in the example above, the initial coordinates of the second and third fill requests in the current row can be set to the initial coordinates of the first fill request in the current row, that is, the initial coordinates of all fill requests in a row are the same.

[0162] Taking the target boundary as the right boundary as an example, if the initial coordinates of the first fill request in the current row are (r, col0+w0-1), then the initial coordinates of the second and third fill requests in the current row are both (r, col0+w0-1).

[0163] Taking the target boundary as the left boundary as an example, if the initial coordinates of the first fill request in the current row are (r, col0), then the initial coordinates of the second and third fill requests in the current row are both (r, col0).

[0164] For example, in other embodiments, one example of "updating the initial coordinates of the next fill request based on the initial coordinates of the current fill request" may include setting the initial coordinates of the next fill request to coordinates that are adjacent to the initial coordinates of the current fill request in the width dimension. This example applies to both mirror fill mode and aggregate fill mode.

[0165] For example, in the example above, the initial coordinates of the second fill request in the current row can be set to the coordinates adjacent to the initial coordinates of the first fill request in the current row in the width dimension, and the initial coordinates of the third fill request in the current row can be set to the coordinates adjacent to the initial coordinates of the second fill request in the current row in the width dimension.

[0166] Taking the target boundary as the right boundary as an example, if the initial coordinates of the first fill request in the current row are (r, col0+w0-1), then the initial coordinates of the second fill request in the current row are (r, col0+w0-2), and the initial coordinates of the third fill request in the current row are (r, col0+w0-3).

[0167] Taking the target boundary as the left boundary as an example, if the initial coordinates of the first fill request in the current row are (r, col0+n0), then the initial coordinates of the second fill request in the current row are (r, col0+n0-1), and the initial coordinates of the third fill request in the current row are (r, col0+n0-2).

[0168] It should be noted that the copy fill mode, mirror fill mode, and aggregate fill mode mentioned above are all classifications of the first boundary fill sub-mode. The following text will introduce the three modes with specific examples.

[0169] For example, one example of step S103 may include: in response to the first boundary padding sub-pattern, for each row of the tensor to be processed, performing the following operations: determining the data write address of the first padding request in the current row based on the data length loaded in each previous padding request and in combination with the shape information of the tensor to be processed; determining the data write address of each padding request after the first padding request in the current row based on the data write address of the first padding request in the current row.

[0170] Taking the target boundary as the left boundary as an example, for each row of the tensor to be processed, a first number of fill requests need to be sent first, followed by tensor loading requests to load the pixels of that row of the tensor to be processed. In this case, the data write address of the first fill request of the first row can be determined as the starting address (base address) in the buffer used to write the filled tensor, and this address serves as the reference point for all subsequent data write operations. For the first fill request of each subsequent row, the sum of the data lengths loaded by all tensor loading requests preceding this fill request is related to the shape information of the tensor to be processed (e.g., the size in dimension W). Therefore, the data write address of this fill request can be determined based on the data lengths loaded by each fill request preceding this fill request and the shape information of the tensor to be processed.

[0171] Taking the target boundary as the right boundary as an example, for each row of the tensor to be processed, a tensor loading request for loading the pixels of that row of the tensor to be processed needs to be sent first, followed by the first number of fill requests. In this case, the data write address of the first tensor loading request of the first row can be determined as the starting address in the buffer used to write the filled tensor, while the data write address of the first fill request of the first row is the starting address plus the data length of the first row of pixels of the tensor to be processed (which can be determined in conjunction with the shape information of the tensor to be processed). For the first fill request of each subsequent row, the sum of the data lengths loaded by all tensor loading requests before this fill request is related to the shape information of the tensor to be processed (e.g., the size of the W dimension). Therefore, the data write address of this fill request can be determined based on the data lengths loaded by each fill request before this fill request and the shape information of the tensor to be processed.

[0172] After determining the data write address of the first fill request in the current row, the data write address of each subsequent fill request in the current row can be determined based on the data write address of the first fill request in the current row.

[0173] For example, in some embodiments, one example of "determining the data write address of each subsequent fill request based on the data write address of the first fill request of the current row" may include: determining the data write address of each subsequent fill request of the current row based on the data write address of the first fill request of the current row, combined with the data length loaded by each preceding fill request of the current row. This example applies to both copy fill mode and mirror fill mode.

[0174] For example, assuming the first quantity is 3, if the data write address of the first fill request in the current row is p_addr_1, then the data write address of the second fill request in the current row is p_addr_2 = p_addr_1 + p_size_1, and the data write address of the third fill request in the current row is p_addr_3 = p_addr_2 + p_size_2, where p_size_1 is the length of the data loaded in the first fill request in the current row, and p_size_2 is the length of the data loaded in the second fill request in the current row.

[0175] For example, in copy-fill mode and mirror-fill mode, the formula for calculating the data write address Addr_2 of the nth request among all requests (including fill requests and tensor load requests) is as follows:

[0176] Addr_2=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1

[0177] Where b_addr_base is the starting address of the tensor after it has been filled in the buffer, which is also the "starting address of the buffer" mentioned above. req_size_1, req_size_2, ..., req_size_n-1 represent the length of data loaded by each of the first n-1 requests.

[0178] That is, except for the first request, the data write address of each subsequent request is the sum of the data write address of the previous request and the length of the data loaded in the previous request.

[0179] For example, in some embodiments, the data loading method provided in at least one embodiment of this disclosure may further include the following steps:

[0180] For each row of the tensor to be processed, perform the following operations: if the number of fill requests generated for the current row has not reached a first number, store the data obtained by the current fill request into a register; if the number of fill requests generated for the current row has reached a first number, perform aggregation processing on the data in the register to obtain the target data.

[0181] For example, aggregation processing may include operations such as averaging, summing, weighted fusion, finding the maximum value, and finding the minimum value, and can be extended to other similar operations; this disclosure does not limit this. For example, assuming the first quantity is n, and each fill request corresponds to a data segment of length m in the register, aggregation processing on the data in the register can be understood as performing an aggregation operation on each of these n data segments of length m, ultimately generating a target data segment of length m. In other words, the length of the target data is the same as the length of the data loaded by a single fill request.

[0182] Correspondingly, one example of "determining the data write address of each subsequent fill request based on the data write address of the first fill request in the current row" could be: using the data write address of the first fill request in the current row as the data write address of each subsequent fill request, where the data write address is used to write the target data. This example applies to aggregated fill mode.

[0183] Unlike copy fill mode and mirror fill mode, in aggregate fill mode, the data write address for all fill requests in each row is the same, and the data written is the target data calculated in the above steps, rather than the data read directly from the tensor to be processed.

[0184] For example, assuming the first quantity is 3, if the data write address of the first fill request in the current row is p_addr_1, then the data write address of the second fill request in the current row is p_addr_2=p_addr_1, and the data write address of the third fill request in the current row is p_addr_3=p_addr_2=p_addr_1.

[0185] The following sections will introduce the copy fill mode, mirror fill mode, and aggregate fill mode through specific examples.

[0186] Figure 5B This is a schematic diagram of a copy-fill pattern provided for at least one embodiment of the present disclosure.

[0187] For example, Figure 5B The example shown is where the target boundary is the right boundary. For each row of the tensor to be processed, a tensor load request is first sent to load the pixels of that row of the tensor to be processed, followed by a first number of fill requests. Figure 5B In the example, the first quantity is 3. In this case, the data write address of the first tensor load request in the first row (i.e., the request to load pixel A5 into the buffer) can be determined as the starting address in the buffer used to write the filled tensor, denoted as b_addr_base. The data length loaded by this request (and subsequent tensor load requests) is determined to be the size of the tensor to be processed in dimension C, denoted as c0. The initial coordinates of this request are (row=2, col=5). Next, the tensor load request to load pixel A4 into the buffer is determined, with a data write address of b_addr_base+c0 and an initial coordinate of (row=2, col=6). This continues until pixel A1, located at the right boundary of the first row of the tensor to be processed (i.e., the target boundary pixel of the first row), is processed. The tensor load request to load pixel A1 into the buffer is determined, with a data write address of b_addr_base+c0×4 and an initial coordinate of (row=2, col=9).

[0188] Based on the tensor loading requests mentioned above, all elements of the first row of the tensor to be processed have been loaded into their corresponding positions in the buffer. Before the newline, the filling requests for the first row are determined. The data length loaded by each filling request is the size c0 of the tensor to be processed in dimension C. Based on the data write address and data length of the previous request (i.e., the tensor loading request used to load A1 into the buffer), the data write address of the first filling request in the first row can be determined to be p_addr_1 = b_addr_base + c0 × 5, where "5" is actually the size of the tensor to be processed in dimension W. Furthermore, the initial coordinates of this filling request can be determined to be the target boundary pixel coordinates of the current row (row=2, col=9). This filling request actually performs a copy operation on pixel A1, writing it to the specified position in the buffer, thereby achieving copy filling. Next, based on p_addr_1 and the data length loaded by the first fill request in the first row, the data write address for the second fill request in the first row can be determined as p_addr_2 = p_addr_1 + c0, with the initial coordinates remaining (row=2, col=9); the data write address for the third fill request in the first row is p_addr_3 = p_addr_2 + c0 = p_addr_1 + c0 × 2, with the initial coordinates remaining (row=2, col=9). These three fill requests effectively copy pixel A1 three times and write it to the specified location in the buffer, thus achieving copy fill.

[0189] Since the number of fill requests in the first row has reached three, it can be determined that the next request is a tensor load request for loading the pixel directly below pixel A5 (that is, switching to the next row to continue the tensor load and fill operations). The processing flow for subsequent rows is the same as the execution logic of the tensor load and fill requests in the first row above, and will not be repeated here.

[0190] Figure 5C This is a schematic diagram of a mirror fill pattern provided for at least one embodiment of the present disclosure.

[0191] For example, Figure 5C The example shown is where the target boundary is the right boundary. For each row of the tensor to be processed, a tensor load request is first sent to load the pixels of that row of the tensor to be processed, followed by a first number of fill requests. Figure 5CIn the example, the first quantity is 3. In this case, the data write address of the first tensor load request in the first row (i.e., the request to load pixel A5 into the buffer) can be determined as the starting address in the buffer used to write the filled tensor, denoted as b_addr_base. The data length loaded by this request (and subsequent tensor load requests) is determined to be the size of the tensor to be processed in dimension C, denoted as c0. The initial coordinates of this request are (row=2, col=5). Next, the tensor load request to load pixel A4 into the buffer is determined, with a data write address of b_addr_base+c0 and an initial coordinate of (row=2, col=6). This continues until pixel A1, located at the right boundary of the first row of the tensor to be processed (i.e., the target boundary pixel of the first row), is determined. The tensor load request to load pixel A1 into the buffer is then determined, with a data write address of b_addr_base+c0×4 and an initial coordinate of (row=2, col=9).

[0192] Based on the tensor loading requests mentioned above, all elements of the first row of the tensor to be processed have been loaded into their corresponding positions in the buffer. Before the newline, the filling requests for the first row are determined. The data length loaded by each filling request is the size c0 of the tensor to be processed in dimension C. Based on the data write address and data length of the previous request (i.e., the tensor loading request used to load A1 into the buffer), the data write address of the first filling request in the first row can be determined as p_addr_1 = b_addr_base + c0 × 5, where "5" is actually the size of the tensor to be processed in dimension W. Furthermore, the initial coordinates of this filling request can be determined as the pixel coordinates adjacent to the target boundary pixel coordinates of the current row (row=2, col=8), i.e., the coordinates of pixel A2. This filling request actually performs a copy operation on pixel A2, writing it to the specified position in the buffer, thus achieving mirror filling. Next, based on p_addr_1 and the data length loaded by the first fill request in the first row, the data write address for the second fill request in the first row can be determined as p_addr_2 = p_addr_1 + c0, with the initial coordinates of pixel A3 (row=2, col=7); the data write address for the third fill request in the first row is p_addr_3 = p_addr_2 + c0 = p_addr_1 + c0 × 2, with the initial coordinates of pixel A4 (row=2, col=6). These three fill requests actually copy pixels A2, A3, and A4 sequentially and write them to the specified positions in the buffer, thus achieving mirror fill.

[0193] Since the number of fill requests in the first row has reached three, we can determine that the next request will be a tensor load request for loading the pixel directly below pixel A5. The processing flow for subsequent rows is the same as the execution logic of the tensor load requests and fill requests in the first row above, and will not be repeated here.

[0194] Figure 5D This is a schematic diagram of a polymer fill pattern provided for at least one embodiment of the present disclosure.

[0195] For example, Figure 5D The example shown is where the target boundary is the right boundary. For each row of the tensor to be processed, a tensor load request is first sent to load the pixels of that row of the tensor to be processed, followed by a first number of fill requests. Figure 5D In the example, the first quantity is 3. In this case, the data write address of the first tensor load request in the first row (i.e., the request to load pixel A5 into the buffer) can be determined as the starting address in the buffer used to write the filled tensor, denoted as b_addr_base. The data length loaded by this request (and subsequent tensor load requests) is determined to be the size of the tensor to be processed in dimension C, denoted as c0. The initial coordinates of this request are (row=2, col=5). Next, the tensor load request to load pixel A4 into the buffer is determined, with a data write address of b_addr_base+c0 and an initial coordinate of (row=2, col=6). This continues until pixel A1, located at the right boundary of the first row of the tensor to be processed (i.e., the target boundary pixel of the first row), is determined. The tensor load request to load pixel A1 into the buffer is then determined, with a data write address of b_addr_base+c0×4 and an initial coordinate of (row=2, col=9).

[0196] Based on the tensor loading requests mentioned above, all elements of the first row of the tensor to be processed have been loaded into their corresponding positions in the buffer. Before the newline, the filling requests for the first row are determined. The data length loaded by each filling request is the size c0 of the tensor to be processed in dimension C. Based on the data write address and data length of the previous request (i.e., the tensor loading request used to load A1 into the buffer), the data write address of the first filling request in the first row can be determined as p_addr_1 = b_addr_base + c0 × 5, where "5" is actually the size of the tensor to be processed in dimension W. Furthermore, the initial coordinates of this filling request can be determined as the pixel coordinates adjacent to the target boundary pixel coordinates of the current row (row=2, col=8), i.e., the coordinates of pixel A2. This filling request actually performs a copy operation on pixel A2, writing it to the specified position in the buffer, thus achieving mirror filling. Next, we can determine that the data write address for the second fill request in the first row is the same as the previous fill request, and the initial coordinates of the request are the coordinates of pixel A3 (row=2, col=7); the data write address for the third fill request in the first row is the same as the previous fill request, and the initial coordinates of the request are the coordinates of pixel A4 (row=2, col=6). Each of these three fill requests stores the read data into a register to aggregate the data in the register, obtaining the target data. The target data is then written to the data write address of each of the three fill requests (actually the same address). In effect, these three fill requests aggregate pixels A1, A2, and A3, and write the aggregated target data to the specified location in the buffer.

[0197] Since the number of fill requests in the first row has reached three, we can determine that the next request will be a tensor load request for loading the pixel directly below pixel A5. The processing flow for subsequent rows is the same as the execution logic of the tensor load requests and fill requests in the first row above, and will not be repeated here.

[0198] It should be noted that, in the above Figures 5B-5D In the example, the tensor to be processed is loaded pixel by pixel, that is, the data length loaded by each tensor loading request is the size of the tensor to be processed in the C dimension; if the tensor to be processed just covers (and does not exceed) the original tensor in the C dimension, a row-by-row loading method can also be used, where only one tensor loading request needs to be sent for each row, and the data length loaded by each tensor loading request is the product of the size of the tensor to be processed in the C dimension and the size in the W dimension.

[0199] In some examples, the first padding pattern may include a first internal padding sub-pattern. For example, the first internal padding sub-pattern indicates that data derived from the tensor to be processed is padded at a specified edge of the tensor to be processed, and the specified edge is the internal pixel boundary of the tensor to be processed.

[0200] For example, the internal pixel boundary of the tensor to be processed refers to the boundary between two adjacent pixels within the tensor. For instance, in the width dimension of the tensor, one or more new pixels can be padded between adjacent pixels, and their values ​​can be derived from copies of pixels within the tensor. For example, a forward copying method can be used, where the new pixel padded between two adjacent pixels has the same value as the previous pixel in those two adjacent pixels. It should be noted that the above copying method is only one example; other methods can be used to determine the padding value depending on the actual application requirements. This internal padding can be used in scenarios such as upsampling, image magnification, or feature map resolution expansion, helping to enhance the model's ability to represent detailed information.

[0201] For example, in the first internal padding submode, the padding parameter may include a second quantity, which indicates the number of pixels that need to be padded between the pixels inside the tensor to be processed.

[0202] For example, in some embodiments, one example of "determining the request initial coordinates corresponding to each fill request based on the fill parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed" may include: in response to a first internal fill sub-mode, determining the request initial coordinates corresponding to each fill request based on a second quantity and the internal pixel coordinates of the tensor to be processed.

[0203] For example, the internal pixel coordinates of the tensor to be processed are determined based on the shape information of the tensor and its starting coordinates. For instance, the coordinates of each pixel inside the tensor to be processed can be determined based on the dimensions of the tensor in the width and height dimensions, combined with the starting coordinates of the tensor.

[0204] Figure 5E This is a schematic diagram of a first internal padding sub-pattern provided for at least one embodiment of the present disclosure.

[0205] For example, such as Figure 5EAs shown, the starting coordinates of the tensor to be processed are (row=1, col=4). The size of the tensor in the width dimension is 3, and the size in the height dimension is 5. Based on the above information, the coordinates of each pixel inside the tensor can be determined. For example, in the first row of the tensor to be processed, the coordinates of pixel A3 are (row=1, col=4), the coordinates of pixel A2 are (row=1, col=5), the coordinates of pixel A1 are (row=1, col=6), and the pixel coordinates of subsequent rows follow the same pattern. Each time a row is moved down, the row index (row) increments by 1; within the same row, the column index increments continuously from the starting column (i.e., the column index value of the starting coordinates).

[0206] For example, such as Figure 5E As shown, in the first internal padding sub-mode, for the first row of the tensor to be processed, the padding request used to fill between pixels A3 and A2 needs to be determined first, denoted as request PR1. Assuming the forward copying method described above is used, the initial coordinates of request PR1 are the coordinates of pixel A3, i.e., (row=1, col=4). Assuming the second quantity is 1, that is, the number of pixels to be filled between the pixels inside the tensor to be processed is 1. Therefore, the padding request used to fill between pixels A2 and A1 needs to be determined next, denoted as request PR2. The initial coordinates of request PR2 are the coordinates of pixel A2, i.e., (row=1, col=5). And so on, the initial coordinates of each padding request in subsequent rows can be determined.

[0207] In the example above, assuming the second quantity is n, there are n fill requests used to fill between pixel A3 and pixel A2, and the initial coordinates of these n fill requests are all the coordinates of pixel A3 (row=1, col=4). Similarly, there are n fill requests used to fill between pixel A2 and pixel A11, and the initial coordinates of these n fill requests are all the coordinates of pixel A2 (row=1, col=5). The remaining fill requests follow the same pattern and will not be elaborated here.

[0208] In the first internal padding sub-pattern described above, one example of step S103 may include:

[0209] For each row of the tensor to be processed, perform the following steps: determine the data write address of the first fill request of the current row based on the data length loaded by the first fill request of the current row; determine the data write address of each fill request after the first fill request of the current row based on the data length loaded by each fill request preceding the current row.

[0210] In the first internal padding sub-mode, assuming the second quantity is 1, for each row of the tensor to be processed, a tensor loading request for loading the first pixel of the current row of the tensor to be processed needs to be sent first, then the first padding request of the current row needs to be sent, then the tensor loading request for loading the first pixel of the current row of the tensor to be processed needs to be sent again, and so on, until the tensor loading request for loading the last pixel of the current row of the tensor to be processed is sent.

[0211] In this example, the data write address for the tensor load request used to load the first pixel of the first row of the tensor to be processed is the starting address of the tensor after padding in the buffer. Therefore, the data write address for the first padding request in the first row is the starting address plus the data length loaded by the first tensor load request in the first row (equal to the data length loaded by the first padding request in the first row), and so on for subsequent rows.

[0212] In this example, the data length loaded by each tensor load request and padding request is the same, so during computation, the data length of the tensor load request can be replaced by the data length loaded by the padding request.

[0213] For example, assuming the second quantity is 1, if the data write address for the first fill request of the current row is p_addr_1, then the data write address for the second fill request of the current row is p_addr_2 = p_addr_1 + load_size_1 + p_size_1 = p_addr_1 + 2 × p_size_1, and the data write address for the third fill request of the current row is p_addr_3 = p_addr_2 + 2 × p_size_2, and so on. Here, p_size_1 is the length of the data loaded in the first fill request of the current row, load_size_1 is the length of the data loaded in the first tensor load request of the current row, and load_size_1 = p_size_1; p_size_2 is the length of the data loaded in the second fill request of the current row, load_size_2 is the length of the data loaded in the second tensor load request of the current row, and load_size_2 = p_size_2.

[0214] For example, in the first internal padding submode, the formula for calculating the data write address Addr_2 of the nth request among all requests (including padding requests and tensor load requests) is as follows:

[0215] Addr_2=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1

[0216] Where b_addr_base is the starting address of the tensor after it has been filled in the buffer, which is also the "starting address of the buffer" mentioned above. req_size_1, req_size_2, ..., req_size_n-1 represent the length of data loaded by each of the first n-1 requests.

[0217] That is, except for the first request, the data write address of each subsequent request is the sum of the data write address of the previous request and the length of the data loaded in the previous request.

[0218] In the second padding mode, the padding operation can be understood as writing a predetermined value to a specified location in the buffer without reading data from the tensor to be processed. Therefore, the second padding mode eliminates the need to determine the data read address, simplifying request generation logic and improving padding efficiency. For example, the predetermined value can be zero, a constant value, etc. The second padding mode is suitable for scenarios such as zero-padding or constant padding, and supports operations such as tensor size expansion and tensor boundary expansion.

[0219] For example, one example of "determining the sub-data returned by each fill request in response to the second fill mode" in step S101 may include: determining the sub-data returned by each fill request in response to the second fill mode, wherein the sub-data is a predetermined value.

[0220] For example, if the predefined value is zero, then each fill request returns zero sub-data (or a data unit consisting of multiple zeros, depending on the length of the data loaded by the fill request). In real-world use cases, these fill requests are translated into, for example, writing multiple predefined values ​​to the buffer using hardware. For instance, in some examples, the predefined value can be explicitly specified by the fill parameter; in others, the predefined value can be a system-preset default value without additional indication.

[0221] In some examples, the second padding mode may include a second inner padding sub-mode. For example, the second inner padding sub-mode indicates that a predetermined value is padded at a specified edge of the tensor to be processed, and the specified edge is the inner pixel boundary of the tensor to be processed.

[0222] For example, one example of step S102 may include: in response to the second internal padding sub-pattern, determining the length of the data loaded for each padding request as the size of the tensor to be processed in the number of channels dimension.

[0223] In other words, in the second internal padding submode, the data length loaded for each padding request corresponds to the full channel data length contained in a single pixel of the tensor to be processed. For example, for a tensor to be processed with c0 channels, the data length loaded for each padding request is c0. Figure 4BIn the example, the tensor to be processed has a size of 5 in the C dimension. Therefore, in the second internal padding sub-mode, it can be determined that the length of data loaded for each padding request is 5.

[0224] It should be noted that the method used in the first internal padding sub-pattern to determine the write address of the padding request data can also be applied to the second internal padding sub-pattern, and will not be elaborated here. The only difference between the first and second internal padding sub-patterns is the source of the data used for padding; other related descriptions can be found in the description of the first internal padding sub-pattern described above.

[0225] In some examples, the second padding pattern may include a second boundary padding sub-pattern. For example, the second boundary padding sub-pattern indicates that a predetermined value is padded at a specified edge of the tensor to be processed, and the specified edge is the boundary of the tensor to be processed.

[0226] In the second boundary padding sub-mode, the padding information also includes padding boundary information, which indicates in which boundary direction of the tensor to be padded (e.g., one or more of the top, bottom, left, and right boundaries). The padding parameters can include a first padding size and a second padding size. The first padding size indicates the number of pixels to be padded in the width dimension, and the second padding size indicates the number of pixels to be padded in the height dimension. In other words, the padding information can specify which boundaries of the tensor to be padded on, and the number of pixels to be padded on those boundaries. Therefore, the second boundary padding sub-mode can also be called the specified boundary padding mode.

[0227] In the second boundary padding sub-mode, padding requests can be further divided into two types: first padding requests and second padding requests. For example, when the padding boundary information indicates padding at the left or right boundary of the tensor to be processed, it can be determined that at least one first padding request is included among multiple padding requests; when the padding boundary information indicates padding at the upper or lower boundary of the tensor to be processed, it can be determined that at least one second padding request is included among multiple padding requests. It can be understood that the first padding request is used to pad at the left or right boundary of the tensor to be processed, and the second padding request is used to pad at the upper or lower boundary of the tensor to be processed.

[0228] For example, the number of first padding requests can be determined based on the size of the tensor to be processed in the height dimension. If the padding boundary information indicates that unilateral padding is performed in the width dimension (i.e., indicating padding only at the left boundary or only at the right boundary of the tensor to be processed), then the number of first padding requests is equal to the size of the tensor to be processed in the height dimension; if the padding boundary information indicates that bilateral padding is performed in the width dimension (i.e., indicating padding at both the left and right boundaries of the tensor to be processed), then the number of first padding requests is equal to twice the size of the tensor to be processed in the height dimension.

[0229] For example, the number of second fill requests can be determined based on the fill boundary information. If the fill information indicates that one-sided fill is performed in the height dimension (i.e., it indicates that fill is performed only at the upper boundary of the tensor to be processed or only at the lower boundary of the tensor to be processed), then the number of second fill requests is equal to 1; if the fill information indicates that two-sided fill is performed in the height dimension (i.e., it indicates that both the upper and lower boundaries of the tensor to be processed are filled), then the number of second fill requests is equal to 2.

[0230] For example, one example of step S102 may include: in response to a second boundary padding sub-mode, performing the following operations: in response to a padding boundary information indicating padding at the left or right boundary of the tensor to be processed, determining the product of a first padding size and the size of the tensor to be processed in the channel number dimension as the data length of a first padding request to be loaded for the left or right boundary padding; in response to a padding boundary information indicating padding at the upper or lower boundary of the tensor to be processed, determining the data length of a second padding request to be loaded for the upper or lower boundary padding based on the sum of the first padding size and the size of the tensor to be processed in the width dimension, the second padding size, and the size of the tensor to be processed in the channel number dimension.

[0231] For example, when it is determined that a first fill request is included among multiple fill requests, the data length loaded by the first fill request can be further determined to be the product of the first fill size and the size of the tensor to be processed in the channel number dimension. The first fill size can be represented by a set of values, such as (pw_size1, pw_size2), where pw_size1 represents the number of pixels to be filled in the left boundary direction, and pw_size2 represents the number of pixels to be filled in the right boundary direction. Assuming the size of the tensor to be processed in the channel number dimension is denoted as c0, the data length loaded by the first fill request for left boundary filling can be determined to be pw_size1×c0, and the data length loaded by the first fill request for right boundary filling can be determined to be pw_size2×c0.

[0232] For example, when it is determined that a second fill request is included among multiple fill requests, the data length loaded by the second fill request can be further determined based on the sum of the first fill size and the size of the tensor to be processed in the width dimension, the second fill size, and the size of the tensor to be processed in the channel number dimension.

[0233] The second padding size can be represented by a set of values, such as (ph_size1, ph_size2), where ph_size1 represents the number of pixels to be padded in the upper boundary direction, and ph_size2 represents the number of pixels to be padded in the lower boundary direction. Assuming the size of the tensor to be processed is denoted as c0 in the channel dimension, w0 in the width dimension, and h0 in the height dimension, the length of the data loaded for the second padding request used for upper boundary padding can be determined as (pw_size1 + pw_size2 + w0) × ph_size1 × c0, and the length of the data loaded for the second padding request used for lower boundary padding is (pw_size1 + pw_size2 + w0) × ph_size2 × c0.

[0234] Figure 5F This is a schematic diagram of a second boundary filling sub-pattern provided for at least one embodiment of the present disclosure.

[0235] For example, in Figure 5F In the example, padding is performed only at the top and left boundaries, and pw_size1=x=3, pw_size2=0, ph_size1=y=1, ph_size2=0, w0=5, h0=3. Based on the above description, the data length loaded by the first padding request for the left boundary is pw_size1×c0=3×c0, and the data length loaded by the second padding request for the top boundary is (pw_size1+pw_size2+w0)×ph_size1×c0=(3+0+5)×1×c0=8×c0. Furthermore, the number of the first padding requests is equal to h0, which is 3, and the number of the second padding requests is 1.

[0236] For example, one example of step S103 may include: in response to the second boundary padding sub-mode, performing the following operations: determining the data write address of each first padding request based on padding boundary information, the data length loaded by each first padding request, and the shape information of the tensor to be processed; in response to the padding boundary information indicating padding at the upper boundary of the tensor to be processed, determining the data write address of the corresponding second padding request as the starting address of the buffer; in response to the padding boundary information indicating padding at the lower boundary of the tensor to be processed, determining the data write address of the corresponding second padding request based on the starting address and the data length loaded by each prior request.

[0237] Assuming that a second padding request exists among multiple padding requests, the data write address for the second padding request can be determined as follows:

[0238] When the padding boundary information indicates that the tensor to be processed is being padded at its upper boundary, the second padding request for upper boundary padding will be the first padding request among all requests (including padding requests and tensor loading requests). Correspondingly, the starting address of the buffer is set to the data write address of the second padding request for upper boundary padding. It should be noted that "the starting address of the buffer" refers to the starting address in the buffer where the padded tensor is written.

[0239] When the padding boundary information indicates padding at the lower boundary of the tensor to be processed, the second padding request for lower boundary padding will be the last padding request among all requests (including padding requests and tensor loading requests). Correspondingly, the data write address for the second padding request for lower boundary padding is determined based on the starting address of the buffer and the data length loaded by each preceding request. For example, the data write address for this second padding request is determined based on the starting address of the buffer and the sum of the data lengths loaded by all requests sent before it.

[0240] Assuming that a first padding request exists among multiple padding requests, the data write address of the first padding request can be determined as follows:

[0241] Based on the fill boundary information, the length of data loaded in each first fill request, and the shape information of the tensor to be processed, the data write address for each first fill request is determined.

[0242] In this embodiment of the disclosure, the formula for calculating the data write address Addr_2 of the nth request among all requests (including padding requests and tensor loading requests) is as follows:

[0243] Addr_2=b_addr_base+req_size_1+ req_size_2+…+ req_size_n-1

[0244] Where b_addr_base is the starting address (base address) of the tensor after it is filled in the buffer, which is also the "starting address corresponding to the buffer" mentioned above. req_size_1, req_size_2, ..., req_size_n-1 represent the length of data loaded by each of the first n-1 requests.

[0245] That is, except for the first request, the data write address of each subsequent request is the sum of the data write address of the previous request and the length of the data loaded in the previous request.

[0246] If the padding boundary information indicates padding at the upper boundary, then the first request out of all requests is the second padding request, and its data write address is b_addr_base. If the padding boundary information further indicates padding at the left or right boundary, then the data write address for each first padding request can be calculated using the formula above.

[0247] If the padding boundary information indicates padding at the left boundary and not the top boundary, then the first request out of all requests is the first padding request, and its data write address is b_addr_base. The data write addresses of the remaining first padding requests can be calculated using the formula above.

[0248] For the first fill request whose data write address is calculated using the above formula, the sum of the data lengths loaded by all the tensor load requests preceding it is related to the shape and size of the tensor to be processed (e.g., the size in W dimensions). Therefore, the data write address of the first fill request can be determined based on the data lengths loaded by each first fill request preceding it and the shape and size of the tensor to be processed.

[0249] For example, in Figure 5F In this example, padding only occurs at the top and left boundaries. Therefore, a second padding request is first sent for the top boundary padding. Its data write address is the starting address b_addr_base corresponding to the buffer, and the loaded data length is 8 × c0 (the specific calculation method has been explained above). The data write address for each subsequent request is the sum of the data write address of the previous request and the data length loaded in the previous request. Next, a first padding request is sent for padding the first row of the left boundary of the tensor to be processed. Its loaded data length is 3 × c0, its data write address is b_addr_base + 8 × c0, and the written value is a preset value. Next, a tensor load request is sent to load the first five pixels of the tensor to be processed. Specifically, if the tensor to be processed does not cover the entire original tensor in dimension c, five tensor load requests need to be sent, each loading data of length 1×c0. If the tensor to be processed exactly covers the entire original tensor in dimension c, only one tensor load request needs to be sent, loading data of length w0×c0=5×c0, with the data written to the address b_addr_base+8×c0+3×c0. Next, a first padding request is sent to fill the second row of the tensor to be processed from its left boundary. This padding request loads data of length 3×c0, with the data written to the address b_addr_base+8×c0+3×c0+5×c0, and the written value is a preset value. This process continues until all tensor load requests for loading the last five pixels of the tensor to be processed have been sent.

[0250] For example, if only Figure 5F The lower and right boundaries of the tensor to be processed are filled, and assuming the fill size is the same as in the example above, a tensor loading request is first sent to load the first five pixels of the tensor to be processed. Specifically, if the tensor to be processed does not cover the entire original tensor in dimension c, five tensor loading requests need to be sent, each loading data of length 1×c0, and the data write address of the first tensor loading request is b_addr_base; if the tensor to be processed exactly covers the entire original tensor in dimension c, only one tensor loading request needs to be sent, loading data of length w0×c0=5×c0, and the data write address is b_addr_base. Next, a first fill request is sent to fill the first row of the right boundary of the tensor to be processed, loading data of length 3×c0, with the data write address b_addr_base+5×c0, and the written value is a preset value. Next, a tensor load request is sent to load five pixels from the second row of the tensor to be processed. This process continues until the first padding request is sent to fill the last row of the right boundary of the tensor to be processed. The data write address is b_addr_base + 5×c0 + 3×c0 + 5×c0 + 3×c0 + 5×c0, and the written value is a preset value. Finally, a second padding request is sent to fill the lower boundary. The data write address is b_addr_base + 5×c0 + 3×c0 + 5×c0 + 3×c0 + 5×c0 + 3×c0, the loaded data length is 8×c0, and the written value is a preset value.

[0251] For example, if only Figure 5F To fill the left boundary of the tensor to be processed, a first fill request is sent to fill the first row of the left boundary of the tensor. The data length loaded is 3×c0, the data write address is b_addr_base, and the written value is a preset value. Subsequent steps can be referred to the aforementioned embodiment, and will not be repeated here.

[0252] Examples of filling in other specified boundary combinations can be derived from the above description and will not be repeated here.

[0253] It should be noted that, Figures 5A-5F The shape formed by the tensor to be processed and the filling region shown is only a logical arrangement for ease of understanding and does not represent its actual physical storage form in the cache.

[0254] Considering that state machines have the advantages of clear logical structure, easy maintenance and expansion, and are particularly suitable for handling logical scenarios with multiple conditions and multiple branches, this embodiment of the disclosure uses a state machine to automatically update the initial coordinates of each request and the length of the loaded data, avoiding complex nested conditions, and making the logic clear, easy to maintain and highly scalable.

[0255] First, it should be noted that in the descriptions of the states below, unless otherwise specified (e.g., it is emphasized that the data write address remains unchanged), the data write address of each request is updated by the sum of the data write address of the previous request and the length of the loaded data. That is, the data write address of each request is obtained by summing the lengths of the data loaded in all previous requests.

[0256] In this embodiment of the disclosure, a first state s1 is set for tensor loading requests, that is, all requests in the first state are tensor loading requests.

[0257] In this embodiment of the disclosure, a second state s2 is set for the copy fill mode in the first boundary fill sub-mode, and the request in the second state is a fill request.

[0258] For example, in response to the copy-fill mode and filling at the right boundary, the initial state of the state machine for the first request is determined to be the first state s1, the initial coordinates of the first request are determined to be the starting coordinates of the tensor to be processed, and the data write address of the first request is determined to be the starting address in the buffer used to write the filled tensor. If the tensor to be processed does not completely cover the original tensor in the C dimension, the length of the data loaded by the first request is determined to be the size of the tensor to be processed in the C dimension. If the tensor to be processed exactly covers the original tensor in the C dimension, the length of the data loaded by the first request is determined to be the product of the size of the tensor to be processed in the C dimension and the size in the W dimension.

[0259] For example, in response to the current request being in state s1 (first state) and satisfying the first loading condition, it is determined that the next request will also be in state s1 (first state). The initial coordinates of the next request in the first dimension are the first coordinates (t_coord_b) of the tensor to be processed in the first dimension. The initial coordinates in the second dimension are incremented by 1, while the third and higher dimensions remain unchanged until the tensor to be processed reaches the boundary of the second dimension. Afterward, the coordinates in the second dimension are updated back to the initial coordinates, and the coordinates in the third dimension are incremented. The third dimension is adjacent to and higher than the second dimension, and so on for each dimension. Coordinates in dimensions lower than the first dimension remain unchanged. For example, the first loading condition includes that the first coordinate value of the tensor to be processed in the first dimension is greater than or equal to the second coordinate value of the original tensor's initial coordinate in the first dimension, and the sum of the first coordinate value and the size of the tensor to be processed in the first dimension is less than the third coordinate value, where the difference between the second and third coordinate values ​​equals the size of the original tensor in the first dimension. For example, if the current request is in state 1, it means that all the requested sub-data is in memory, i.e., it is all valid data. In the above example, if the first dimension is dimension C, then the second dimension is dimension W, and the third dimension is dimension H; if the first dimension is dimension W, then the second dimension is dimension H, and the third dimension is, for example, dimension D.

[0260] For example, in response to the current request being in state s1 and satisfying the first padding condition, the next request is determined to enter state s2. The initial coordinates of the next request are updated to the coordinates of the tensor to be processed at the right boundary of the current row, and the length of the data loaded in the next request is updated to the size of the tensor to be processed in dimension C. For example, the first padding condition includes padding information indicating the use of a copy padding mode, and that the loading of an entire row of data (including data in dimension C) of the tensor to be processed has been completed under the action of the current request, meaning that all pixels of the tensor to be processed in the current row have been loaded (e.g., if the second dimension is dimension W, then the first padding condition is that the initial coordinates of the current request are equal to the coordinates of the tensor to be processed at the right boundary of the current row).

[0261] For example, in response to the current request being in state s2 and satisfying the second padding condition, it is determined that the state of the next request will also be in state s2, and the initial coordinates of the next request will remain unchanged, as will the length of the data loaded in the next request. For example, the second padding condition includes: the number of consecutively generated requests in state s2 has not yet reached a first number (the first number indicates the number of padding requests that need to be generated in the W dimension).

[0262] In response to the current request being in state s2 (second state) and satisfying the third padding condition, the next request is determined to enter state s1 (first state). The initial coordinates of the next request are incremented by 1 in the H dimension and updated in the W dimension to reflect the starting coordinates of the tensor to be processed. The data length loaded in the next request is updated according to the data length loaded in the first (tensor loading) request. For example, the third padding condition includes: the number of consecutively generated requests in state 2 reaches a first quantity.

[0263] It should be noted that if a situation arises where both the first loading condition and the first / second / third filling condition are met, the state transition will be prioritized based on the first / second / third filling condition. In other words, the filling control logic has higher priority, and the state machine will first respond to the filling-related condition judgment and execute the corresponding state transition.

[0264] In this embodiment of the disclosure, a third state s3 is set for the mirror fill mode in the first boundary fill sub-mode, and the request in the third state is a fill request.

[0265] The description of the initial state and the s1->s1 state transition is the same as above, and will not be repeated here.

[0266] For example, in response to the current request being in state s1 (first state) and satisfying the fourth padding condition, the next request is determined to enter state s3 (third state). The initial coordinates of the next request are updated to the coordinates adjacent to the right boundary of the tensor to be processed in the current row (i.e., one less in the W dimension compared to the right boundary coordinates of the current row). The length of the data loaded in the next request is updated to the size of the tensor to be processed in the C dimension. For example, the fourth padding condition includes padding information indicating the use of mirror padding mode, and the fact that the current request has completed loading an entire row of data (including data in the C dimension) of the tensor to be processed, meaning that all pixels of the tensor to be processed in the current row have been loaded.

[0267] If the current request is in state 3 (s3) and the fifth padding condition is met, it is determined that the next request will also be in state 3 (s3), and the initial coordinates of the next request will be reduced by one in the W dimension, while the length of the data loaded in the next request will remain unchanged. For example, the fifth padding condition includes: the number of consecutively generated requests in state 3 has not yet reached the first number (the first number indicates the number of padding requests that need to be generated in the W dimension).

[0268] If the current request is in state 3 (s3) and the sixth padding condition is met, the next request is determined to enter state 1 (s1). The initial coordinates of the next request are incremented by 1 in the H dimension and updated in the W dimension to reflect the starting coordinates of the tensor to be processed. The data length loaded in the next request is updated according to the data length loaded in the first (tensor loading) request. For example, the sixth padding condition includes: the number of consecutively generated requests in state 3 reaches a certain threshold.

[0269] It should be noted that if a situation occurs where both the first loading condition mentioned above and the fourth / fifth / sixth filling condition are met, then the state transition will be performed based on the fourth / fifth / sixth filling condition.

[0270] In this embodiment of the disclosure, a fourth state s4 is set for the aggregated fill mode in the first boundary fill sub-mode, and the request in the fourth state is a fill request.

[0271] The description of the initial state and the s1->s1 state transition is the same as above, and will not be repeated here.

[0272] For example, in response to the current request being in state s1 (first state) and satisfying the seventh padding condition, the next request is determined to enter state s4 (fourth state). The initial coordinates of the next request are updated to the coordinates adjacent to the right boundary of the tensor to be processed in the current row (i.e., one less in the W dimension compared to the right boundary coordinates of the current row). The data length loaded in the next request is updated to the size of the tensor to be processed in the C dimension. For example, the seventh padding condition includes padding information indicating the use of aggregate padding mode, and the fact that the current request has completed loading an entire row of data (including data in the C dimension) of the tensor to be processed, meaning that all pixels of the tensor to be processed in the current row have been loaded.

[0273] In response to the current request being in state 4 (s4) and satisfying the eighth padding condition, it is determined that the next request will also be in state 4 (s4), and the initial coordinates of the next request will be decreased by one in the W dimension, but the data write address will remain unchanged (in this example, the data write address is used to write the target data, and the process of determining the target data will be described below). Specifically, this next request is also configured to write read data into a register. For example, the eighth padding condition includes: the number of consecutively generated requests in state 4 has not yet reached a first quantity (the first quantity indicates the number of padding requests that need to be generated in the W dimension).

[0274] If the current request is in state 4 (s4) and the ninth padding condition is met, additional aggregation processing (e.g., averaging, summing, weighted fusion, finding the maximum or minimum value) will be performed on the data in the register to obtain the target data (the length of the target data is the same as the length of the data loaded in a single padding request). Furthermore, it is determined that the next request will enter state 1 (s1). The initial coordinates of the next request will be incremented by 1 in the H dimension and updated in the W dimension to reflect the starting coordinates of the tensor to be processed. The length of the data loaded in the next request will be updated according to the length of the data loaded in the first (tensor loading) request. For example, the ninth padding condition includes: the number of consecutively generated requests in state 4 reaching a certain number.

[0275] It should be noted that if a situation occurs where both the first loading condition mentioned above and the seventh / eighth / ninth filling condition are met, then the state transition will be performed based on the seventh / eighth / ninth filling condition.

[0276] In the examples of the second state s2, the third state s3, and the fourth state s4 above, the length of the data loaded in each fill request is the size of the tensor to be processed in the C dimension.

[0277] The newly added second state s2, third state s3, and fourth state s4 are all examples of filling the right boundary of the tensor to be processed. The implementation principle of filling the left boundary is similar, the difference being that the initial state should be the newly defined state, instead of the first state s1. This article will not go into the specific state machine details.

[0278] In this embodiment of the disclosure, a fifth state s5 is set for the second boundary fill sub-mode (also known as the specified boundary fill mode), and the request in the fifth state is a fill request.

[0279] In response to the padding boundary information indicating that padding should be performed at the upper boundary of the tensor to be processed, a padding request is first generated. This request loads data of length (pw_size1 + pw_size2 + w0) × ph_size1 × c0, where ph_size1 represents the number of pixels to be padded at the upper boundary, pw_size1 represents the number of pixels to be padded at the left boundary, pw_size2 represents the number of pixels to be padded at the right boundary, and all of these can be included in the padding parameters. c0 represents the size of the tensor to be processed in the channel dimension, and w0 represents the size in the width dimension. The data to be written to in this request is the starting address in the buffer used to write the padded tensor.

[0280] In response to the filling boundary information indication, fill the left boundary of the tensor to be processed, determine the initial state as the fifth state s5, update the length of the data currently requested to be loaded to pw_size1×c0, and determine the sub-data returned by the current request as the predetermined value.

[0281] The description of the s1->s1 state transition is the same as above, and will not be repeated here.

[0282] In response to the current request being in state 5 (s5), the next request is determined to enter state 1 (s1), and the data length loaded in the next request is determined to be the data length loaded in the first tensor loading request. Specifically, when determining that the next request will enter state 1 (s1) from the initial state, the initial coordinates of the next request are determined to be the starting coordinates of the tensor to be processed; if the tensor to be processed does not completely cover the original tensor in dimension C, the data length loaded in the current request is determined to be the size of the tensor to be processed in dimension C; if the tensor to be processed exactly covers the original tensor in dimension C, the data length loaded in the current request is determined to be the product of the size of the tensor to be processed in dimension C and the size in dimension W.

[0283] In response to the current request being in state 1 (s1) and satisfying the tenth padding condition, the next request is determined to enter state 5 (s5), and the length of the data loaded in the current request is updated to pw_size1×c0. For example, the tenth padding condition includes padding boundary information indicating that padding is performed on the left boundary of the tensor to be processed, and that the loading of an entire row of data (including data in dimension C) of the tensor to be processed has been completed under the action of the current request, that is, all pixels of the tensor to be processed in the current row have been loaded.

[0284] In response to the padding boundary information instruction, padding is performed at the lower boundary of the tensor to be processed. After loading the last row of data of the tensor to be processed, a padding request is generated. The data length loaded in this request is (pw_size1+pw_size2+w0)×ph_size2×c0, where ph_size2 represents the number of pixels to be padded in the lower boundary direction, pw_size1 represents the number of pixels to be padded in the left boundary direction, pw_size2 represents the number of pixels to be padded in the right boundary direction, and all of these can be included in the padding parameters. c0 represents the size of the tensor to be processed in the channel dimension, and w0 represents the size in the width dimension.

[0285] It should be noted that if both the first loading condition and the tenth filling condition mentioned above are met, the state transition will be performed based on the tenth filling condition.

[0286] The newly added fifth state s5 is an example of filling the left boundary of the tensor to be processed. The implementation principle of filling the right boundary is similar, and the specific state machine details will not be elaborated in this article.

[0287] In this embodiment of the disclosure, a sixth state s6 is set for the first internal padding sub-mode, and the request in the sixth state is a padding request.

[0288] For example, in response to the first internal padding sub-pattern, the initial state of the state machine for the first request is determined to be the first state s1, the initial coordinates of the first request are determined to be the starting coordinates of the tensor to be processed, the data write address of the first request is determined to be the starting address in the buffer used to write the padded tensor, and the length of the data loaded by the first request is determined to be the size of the tensor to be processed in the C dimension.

[0289] The description of the s1->s1 state transition is the same as above, and will not be repeated here.

[0290] For example, in response to the current request being in state 1 (s1) and satisfying the eleventh padding condition, the next request is determined to enter state 6 (s6), and the initial coordinates of the next request remain unchanged. For example, the eleventh padding condition includes padding information indicating the use of a first internal padding sub-pattern, and the initial coordinates of the current request not reaching the right boundary of the tensor to be processed.

[0291] In response to the current request being in state 6 (s6) and satisfying the twelfth padding condition, it is determined that the state of the next request is still state 6 (s6), and the initial coordinates of the next request remain unchanged. For example, the twelfth padding condition includes: the number of consecutively generated requests in state 6 has not yet reached the second number (the second number indicates the number of pixels that need to be padded between two adjacent pixels inside the tensor to be processed).

[0292] If the current request is in state 6 (s6) and the thirteenth padding condition is met, the next request is determined to enter state 1 (s1), and the initial coordinates of the next request are incremented by 1 in the W dimension. For example, the thirteenth padding condition includes: the number of consecutively generated requests in state 6 reaches a second quantity.

[0293] The goal of the state machine described above is to insert a specified number (second number) of padding pixels between every two adjacent pixels inside the tensor to be processed during the process of loading the tensor to be processed into the buffer, and the padding pixels are derived from the tensor to be processed.

[0294] For example, such as Figure 5EAs shown, assuming the current request R1 is a tensor loading request for loading pixel A3 from the tensor to be processed, its state is the first state s1, and its initial coordinates are (row=1, col=4). Since the eleventh padding condition is met, it can be determined that the next request R2 enters the sixth state s6, and the initial coordinates of the next request R2 are still (row=1, col=4). Assuming the second quantity is 1, the twelfth padding condition is met, so it can be determined that the next request R3 enters the first state s1, and the initial coordinates of the next request R3 increment by 1 in the width dimension, that is, (row=1, col=5). In the above three requests, requests R1 and R3 are tensor loading requests, and request R2 is a padding request.

[0295] For example, such as Figure 5E As shown, assuming the current request R1 is a tensor loading request for pixel A1 in the tensor to be processed, its state is the first state s1, and the initial coordinates are (row=1, col=6). Since the initial coordinates of the current request R1 have reached the right boundary of the tensor to be processed (col=6), the eleventh padding condition is not met. The next request will not enter the sixth state s6, but will continue to the first state s1, determining the initial coordinates of the next request R2 as (row=2, col=4). It should be noted that in the above example, for ease of explanation, only the changes in the initial coordinates of the request in two dimensions (width and height) are specifically described. The update logic of other dimensions can be derived by analogy.

[0296] It should be clarified that during the state transition process from s1 to s6 to s1, the length of the data loaded by each request (whether it is a fill request or a tensor loading request) is the size of the tensor to be processed in the dimension of the number of channels.

[0297] It should be noted that if a situation occurs where both the first loading condition mentioned above and the eleventh / twelfth / thirteenth filling condition are met, then the state transition will be performed based on the eleventh / twelfth / thirteenth filling condition.

[0298] In this embodiment of the disclosure, a new seventh state s7 is added for the second internal padding sub-mode, and the request in the seventh state is a padding request.

[0299] For example, in response to the second internal padding sub-pattern, the initial state of the state machine for the first request is determined to be the first state s1, the initial coordinates of the first request are determined to be the starting coordinates of the tensor to be processed, the data write address of the first request is determined to be the starting address in the buffer used to write the padded tensor, and the length of the data loaded by the first request is determined to be the size of the tensor to be processed in the C dimension.

[0300] The description of the s1->s1 state transition is the same as above, and will not be repeated here.

[0301] For example, in response to the current request being in state 1 (s1) and satisfying the fourteenth padding condition, the next request is determined to enter state 7 (s7), and the sub-data returned by the next request is determined to be a predetermined value, while the data length loaded in the next request remains unchanged. For example, the fourteenth padding condition includes padding information indicating the use of the second internal padding sub-mode, and the initial coordinates of the current request not reaching the right boundary of the tensor to be processed.

[0302] In response to the current request being in state 7 (s7) and satisfying the fifteenth padding condition, it is determined that the state of the next request will also be state 7 (s7), and the sub-data returned by the next request will be a predetermined value. For example, the fifteenth padding condition includes: the number of consecutively generated requests in state 7 has not yet reached the second quantity.

[0303] If the current request is in state 7 (s7) and the sixteenth padding condition is met, the next request is determined to enter state 1 (s1), and the initial coordinates of the next request are incremented by 1 in the W dimension. For example, the sixteenth padding condition includes: the number of consecutively generated requests in state 7 reaches a second quantity.

[0304] The goal of the state machine described above is to insert a specified number (second number) of padding pixels between every two adjacent pixels inside the tensor to be processed during the process of loading the tensor to be processed into the buffer, and the padding pixels are a predetermined value.

[0305] It should be noted that if a situation occurs where both the first loading condition mentioned above and the fourteenth / fifteenth / sixteenth filling condition are met, then the state transition shall be performed based on the fourteenth / fifteenth / sixteenth filling condition.

[0306] In at least one embodiment of the data loading method provided in this disclosure, a hardware-level automatic filling implementation scheme is proposed. This scheme can automatically complete the filling operation during the tensor data copying process (e.g., loading from memory to the cache), without relying on additional software configuration or complex user instruction configuration. This significantly reduces programming complexity and control overhead, while greatly improving the bandwidth and efficiency during data memory access, thereby improving the hardware utilization of the computing unit and enhancing hardware performance. Furthermore, the data loading method provided in this disclosure supports multiple different filling modes, which can flexibly adapt to diverse computing needs. This scheme provides efficient and low-latency data loading support for application scenarios sensitive to data layout, such as deep learning inference, image processing, and high-performance computing, significantly improving the overall system performance.

[0307] At least one embodiment of this disclosure also provides a data loading method. Figure 6 This is a schematic flowchart illustrating a data loading method provided in at least one embodiment of the present disclosure.

[0308] like Figure 6 As shown, at least one embodiment of this disclosure provides a data loading method including steps S201-S202.

[0309] For example, in step S201, a data loading instruction is received.

[0310] For example, the data loading instruction includes the filling information of the tensor to be processed, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor. The filling information is used to indicate the filling mode when filling the specified edge of the tensor to be processed and the filling parameters related to the filling mode. The tensor to be processed and the original tensor include at least the height dimension, the width dimension, and the number of channels dimension.

[0311] For a description of the tensors to be processed and the original tensors, please refer to the description of the data loading method mentioned above. Repeated descriptions will not be repeated here.

[0312] For example, a data load instruction can be a machine instruction, or it can be a microinstruction. For instance, a data load instruction can be a Load instruction.

[0313] In step S202, after parsing the data loading instruction, the execution unit executes the data loading instruction.

[0314] For example, after the processor receives a data load instruction, it parses the data load instruction, such as decoding the data load instruction, generates microinstructions, and sends the microinstructions to the instruction allocation unit; the instruction allocation unit sends the microinstructions to the corresponding scheduling queue according to the microinstruction type; in response to the microinstructions, after the input parameters are prepared, the execution unit executes the relevant operations of the data load instruction.

[0315] For example, step S202 may include: determining multiple fill requests for loading fill data based on fill information, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed, wherein the fill data is determined based on fill mode and fill parameters; sequentially sending multiple fill requests, and writing the sub-data returned by each fill request into a buffer to load the fill data into the buffer.

[0316] For example, the fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, which is determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

[0317] Regarding "determining multiple filling requests for loading filling data based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed", please refer to the relevant content of the aforementioned step S20, which will not be repeated here.

[0318] For details on "sending multiple fill requests sequentially and writing the sub-data returned by each fill request into the cache to load the fill data into the cache", please refer to the relevant content of step S30 above, which will not be repeated here.

[0319] For example, step S202 may further include: determining multiple tensor loading requests for loading the tensor to be processed into the buffer based on the shape information of the tensor to be processed, the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor, and the shape information of the original tensor; sequentially sending multiple tensor loading requests, and writing the sub-data returned by each tensor loading request into the buffer to load the tensor to be processed into the buffer. Here, the tensor loading requests are divided along the width or height dimension of the original tensor, and the sub-data returned by each tensor loading request is located in memory. The sub-data returned by multiple tensor loading requests comes from the original tensor and is stored contiguously in memory.

[0320] In at least one embodiment of the data loading method provided in this disclosure, a hardware-level automatic filling implementation scheme is proposed. This scheme can automatically complete the filling operation during the tensor data copying process (e.g., loading from memory to the cache), without relying on additional software configuration or complex user instruction configuration. This significantly reduces programming complexity and control overhead, while greatly improving the bandwidth and efficiency during data memory access, thereby improving the hardware utilization of the computing unit and enhancing hardware performance. Furthermore, the data loading method provided in this disclosure supports multiple different filling modes, which can flexibly adapt to diverse computing needs. This scheme provides efficient and low-latency data loading support for application scenarios sensitive to data layout, such as deep learning inference, image processing, and high-performance computing, significantly improving the overall system performance.

[0321] Figure 7 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. For example... Figure 7As shown, the electronic device 300 is, for example, suitable for implementing the data loading method or data storage method provided in the embodiments of this disclosure. It should be noted that... Figure 7 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0322] like Figure 7 As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0323] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the data loading method according to any of the above embodiments, or one or more steps of the data storage method according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the data loading method can be found in the relevant descriptions in the embodiments of the data loading method described above, and a detailed description of the processing procedure of the data storage method can also be found in the relevant descriptions in the embodiments of the data storage method described above.

[0324] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0325] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are interconnected via a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0326] Typically, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 308 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 7 An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, a processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processor (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0327] Figure 8 This is a schematic structural diagram of a processor provided for at least one embodiment of the present disclosure. For example... Figure 8 As shown, the processor 400 includes an instruction parsing unit 401 and an execution unit 402.

[0328] For example, instruction parsing unit 401 is used to receive and parse data loading instructions.

[0329] For example, the data loading instruction includes the filling information of the tensor to be processed, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor. The filling information is used to indicate the filling mode when filling the specified edge of the tensor to be processed and the filling parameters related to the filling mode. The tensor to be processed and the original tensor include at least the height dimension, the width dimension, and the number of channels dimension.

[0330] For a description of the tensors to be processed and the original tensors, please refer to the description of the data loading method mentioned above. Repeated descriptions will not be repeated here.

[0331] For example, a data load instruction can be a machine instruction, or it can be a microinstruction. For instance, a data load instruction can be a Load instruction.

[0332] For example, execution unit 402 executes the data load instruction after the instruction parsing unit parses the data load instruction.

[0333] For example, when executing a data loading instruction, the execution unit 402 includes the following operations: determining multiple filling requests for loading filling data based on filling information, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed, wherein the filling data is determined based on the filling mode and filling parameters; sequentially sending multiple filling requests, and writing the sub-data returned by each filling request into a buffer to load the filling data into the buffer.

[0334] For example, the fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, which is determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data with different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

[0335] Specifically, when upper-layer software based on the processor (such as AI applications, HPC applications, and scientific computing applications) sends data loading instructions for computation to the processor (such as CPU or GPU) through a unified encapsulated function library, the data loading instructions can carry the padding information, shape information, and starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor as input parameters. When the processor receives the data loading instructions, the instruction parsing unit 401 parses the data loading instructions to obtain the padding information, shape information, and starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor as input parameters. The processor then schedules the arithmetic unit to execute the data loading task for the input parameters.

[0336] For example, after parsing the data load instruction, the processor can store the input parameters in the data load instruction into a register or memory. When the execution unit 402 performs calculation processing, it can obtain the input parameters from the register or memory.

[0337] For the specific process of executing the data loading instruction using the execution unit 402, please refer to steps S20-S30 in the data loading method described above. Furthermore, you can also refer to steps S40-S50 in the aforementioned method. Repeated steps will not be repeated here.

[0338] The processor provided in at least one embodiment of this disclosure can achieve similar technical effects to the aforementioned data loading method, and the repetitions will not be repeated.

[0339] Figure 9 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. For example, such as Figure 9As shown, storage medium 500 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 501 can be stored non-transitoryly. For example, when computer-readable instructions 501 are executed by a processor, one or more steps in the data loading method described above can be performed. For example, when computer-readable instructions 501 are executed by a processor, one or more steps in the data storage method described above can be performed.

[0340] For example, the storage medium 500 can be used in an electronic device 300, such as the storage medium 500 including the storage device 308 in the electronic device 300.

[0341] For example, a storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and a processor may execute these instructions to perform various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0342] For example, the storage medium may include a memory card for a smartphone, a cache component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0343] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0344] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0345] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0346] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0347] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0348] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0349] The following points should be noted regarding this disclosure:

[0350] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0351] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0352] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data loading method, comprising: Obtain the filling information of the tensor to be processed, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor. The filling information is used to indicate the filling mode when filling the specified edge of the tensor to be processed and the filling parameters related to the filling mode. The tensor to be processed and the original tensor include at least the height dimension, the width dimension, and the number of channels dimension. Based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, multiple filling requests for loading filling data are determined, wherein the filling data is determined based on the filling mode and the filling parameters; The multiple fill requests are sent sequentially, and the sub-data returned by each fill request is written to the cache area to load the fill data into the cache area. The fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data that have different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

2. The data loading method according to claim 1, wherein, Each fill request includes a data read address indicating the starting position for reading data from the memory, a data write address indicating the starting position for writing data to the buffer, and the length of data loaded by the fill request. The step of determining multiple filling requests for loading filling data based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed includes: In response to the first filling mode, the data read address for each filling request is determined based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed. In response to the second fill mode, determine the sub-data returned for each fill request; Based on the filling information and the shape information of the tensor to be processed, the data length loaded for each filling request is determined; Based on the filling information and the length of data loaded for each filling request, the data write address for each filling request is determined.

3. The data loading method according to claim 2, wherein, In response to the first filling mode, the data read address for each filling request is determined based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, including: In response to the first fill mode, perform the following operations: Based on the filling parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, determine the initial coordinates of each filling request; Based on the shape information of the original tensor and the initial coordinates of each fill request, the data read address of each fill request is determined.

4. The data loading method according to claim 2, wherein, The response to the second fill mode determines the sub-data returned for each fill request, including: In response to the second fill mode, determine the sub-data returned for each fill request, wherein the sub-data is a predetermined value.

5. The data loading method according to claim 2, wherein, The step of determining the data length loaded for each filling request based on the filling information and the shape information of the tensor to be processed includes: In response to the first filling mode, the length of the data loaded for each filling request is determined to be the size of the tensor to be processed in the channel number dimension.

6. The data loading method according to claim 3, wherein, The first filling mode includes a first boundary filling sub-mode, which indicates that data originating from the tensor to be processed is filled at a specified edge of the tensor to be processed, and the specified edge is the boundary of the tensor to be processed. The step of determining the initial coordinates of each fill request based on the fill parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed includes: In response to the first boundary padding sub-mode, the initial coordinates of each padding request are determined based on the padding parameters and the target boundary pixel coordinates of the tensor to be processed. The target boundary pixel coordinates of the tensor to be processed are determined based on the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed.

7. The data loading method according to claim 6, wherein, The step of determining the data write address for each fill request based on the fill information and the data length loaded for each fill request includes: In response to the first boundary padding sub-pattern, for each row of the tensor to be processed, the following operations are performed: Based on the length of data loaded in each previous fill request, and combined with the shape information of the tensor to be processed, determine the data write address of the first fill request in the current row; Based on the data write address of the first fill request in the current row, determine the data write address of each fill request after the first fill request in the current row.

8. The data loading method according to claim 6, wherein, The fill parameter includes a first quantity, which indicates the number of fill requests that need to be generated in the width dimension. The step of responding to the first boundary padding sub-mode by determining the initial coordinates of each padding request based on the padding parameters and the target boundary pixel coordinates of the tensor to be processed includes: In response to the first boundary padding sub-pattern, for each row of the tensor to be processed, the following operations are performed: Based on the target boundary pixel coordinates of the tensor to be processed in the current row, determine the initial coordinates of the first fill request in the current row; In response to the fact that the number of fill requests generated in the current row has not reached the first number, the initial coordinates of the next fill request are updated based on the initial coordinates of the current fill request.

9. The data loading method according to claim 8, wherein, The step of updating the initial coordinates of the next fill request based on the initial coordinates of the current fill request includes: Set the initial coordinates of the next fill request to the initial coordinates of the current fill request.

10. The data loading method according to claim 8, wherein, The step of updating the initial coordinates of the next fill request based on the initial coordinates of the current fill request includes: The initial coordinates of the next fill request are set to the coordinates adjacent to the initial coordinates of the current fill request in the width dimension.

11. The data loading method according to claim 7, wherein, The step of determining the data write address of each subsequent fill request based on the data write address of the first fill request of the current row includes: Based on the data write address of the first fill request in the current row, and combined with the data length loaded by each preceding fill request in the current row, determine the data write address of each fill request after the first fill request in the current row.

12. The data loading method according to claim 10, further comprising: For each row of the tensor to be processed, perform the following operations: In response to the fact that the number of fill requests generated for the current row has not reached the first number, the data obtained from the current fill request is stored in a register; In response to the number of fill requests generated for the current row reaching the first number, the data in the register is aggregated to obtain the target data for writing to the cache.

13. The data loading method according to claim 12, wherein, The step of determining the data write address of each subsequent fill request based on the data write address of the first fill request of the current row includes: The data write address of the first fill request in the current row is used as the data write address of each fill request after the first fill request in the current row, and the data write address is used to write the target data.

14. The data loading method according to claim 4, wherein, The second fill mode includes a second boundary fill sub-mode, which indicates that a predetermined value is filled at a specified edge of the tensor to be processed, and the specified edge is the boundary of the tensor to be processed. The fill information also includes fill boundary information, which indicates in which boundary direction of the tensor to be processed is the fill performed. The fill parameters include a first fill size and a second fill size, where the first fill size indicates the number of pixels to be filled in the width dimension, and the second fill size indicates the number of pixels to be filled in the height dimension. The step of determining the data length loaded for each filling request based on the filling information and the shape information of the tensor to be processed includes: In response to the second boundary padding sub-pattern, perform the following operations: In response to the fill boundary information indicating that the left or right boundary of the tensor to be processed is to be filled, the product of the first fill size and the size of the tensor to be processed in the channel number dimension is determined as the data length of the first fill request to be loaded for the left or right boundary fill. In response to the fill boundary information indicating that the upper or lower boundary of the tensor to be processed is to be filled, the data length of the second fill request to be loaded for upper or lower boundary fill is determined based on the sum of the first fill size and the size of the tensor to be processed in the width dimension, the second fill size, and the size of the tensor to be processed in the channel number dimension.

15. The data loading method according to claim 14, wherein, The step of determining the data write address for each fill request based on the fill information and the data length loaded for each fill request includes: In response to the second boundary padding sub-pattern, perform the following operations: Based on the filling boundary information, the data length loaded in each first filling request, and the shape information of the tensor to be processed, the data write address for each first filling request is determined. In response to the filling boundary information indicating that the upper boundary of the tensor to be processed is filled, the data write address of the corresponding second filling request is determined to be the starting address of the cache area; In response to the filling boundary information indicating that the lower boundary of the tensor to be processed is filled, the data write address of the corresponding second filling request is determined based on the starting address and the data length of each prior request.

16. The data loading method according to claim 4, wherein, The second fill mode includes a second internal fill sub-mode, which indicates that a predetermined value is filled at a specified edge of the tensor to be processed, and the specified edge is the internal pixel boundary of the tensor to be processed. The step of determining the data length loaded for each filling request based on the filling information and the shape information of the tensor to be processed includes: In response to the second internal padding sub-mode, the length of the data loaded for each padding request is determined to be the size of the tensor to be processed in the channel number dimension.

17. The data loading method according to claim 3, wherein, The padding parameter includes a second quantity, which indicates the number of pixels that need to be padded between pixels within the tensor to be processed. The first padding mode includes a first internal padding sub-mode, which indicates that data from the tensor to be processed is padded at specified edges of the tensor to be processed, where the specified edges are the internal pixel boundaries of the tensor to be processed. The step of determining the initial coordinates of each fill request based on the fill parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed includes: In response to the first internal padding sub-mode, the initial coordinates of each padding request are determined based on the second quantity and the internal pixel coordinates of the tensor to be processed. The internal pixel coordinates of the tensor to be processed are determined based on the shape information of the tensor to be processed and the starting coordinates of the tensor to be processed.

18. The data loading method according to claim 16 or 17, wherein, The step of determining the data write address for each fill request based on the fill information and the data length loaded for each fill request includes: In response to either the first inner padding sub-pattern or the second inner padding sub-pattern, for each row of the tensor to be processed, the following operations are performed: The data write address of the first fill request in the current row is determined based on the length of the data loaded by the first fill request in the current row. Based on the data length loaded by each fill request preceding the current row, determine the data write address for each fill request after the first fill request of the current row.

19. The data loading method according to claim 1, further comprising: Based on the shape information of the tensor to be processed, the starting coordinates of the tensor to be processed in the coordinate system defined by the original tensor, and the shape information of the original tensor, multiple tensor loading requests are determined for loading the tensor to be processed into the buffer. Specifically, tensor loading requests are divided along the width or height dimension of the original tensor. The sub-data returned by each tensor loading request is located in memory. The sub-data returned by the multiple tensor loading requests come from the original tensor and are stored contiguously in memory.

20. A data loading method, comprising: A data loading instruction is received, wherein the data loading instruction includes filling information of the tensor to be processed as an input parameter, shape information of the tensor to be processed, and starting coordinates of the tensor to be processed in a coordinate system determined by the original tensor, wherein the filling information is used to indicate the filling mode when filling the specified edge of the tensor to be processed and the filling parameters related to the filling mode, and the tensor to be processed and the original tensor include at least a height dimension, a width dimension, and a channel number dimension; After parsing the data loading instruction, the execution unit executes the data loading instruction. The execution of the data loading instruction using the execution unit includes: Based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, multiple filling requests for loading filling data are determined, wherein the filling data is determined based on the filling mode and the filling parameters; The multiple fill requests are sent sequentially, and the sub-data returned by each fill request is written to the cache area to load the fill data into the cache area. The fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data that have different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

21. A processor, comprising an instruction parsing unit and an execution unit, wherein, The instruction parsing unit is used to receive and parse data loading instructions, wherein the data loading instructions include the filling information of the tensor to be processed as input parameters, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed in the coordinate system determined by the original tensor, wherein the filling information is used to indicate the filling mode when filling the specified edge of the tensor to be processed and the filling parameters related to the filling mode, and the tensor to be processed and the original tensor include at least the height dimension, the width dimension, and the number of channels dimension; After the instruction parsing unit parses the data loading instruction, the execution unit executes the data loading instruction. When the execution unit executes the data loading instruction, it includes performing the following operations: Based on the filling information, the shape information of the tensor to be processed, and the starting coordinates of the tensor to be processed, multiple filling requests for loading filling data are determined, wherein the filling data is determined based on the filling mode and the filling parameters; The multiple fill requests are sent sequentially, and the sub-data returned by each fill request is written to the cache area to load the fill data into the cache area. The fill mode includes a first fill mode and a second fill mode. The first fill mode indicates that the sub-data returned by each fill request comes from the tensor to be processed. The sub-data returned by each fill request is the data corresponding to a pixel in the tensor to be processed, determined based on the fill parameters and the fill mode. The data corresponding to the pixel includes all data that have different channel number dimension coordinates but the same coordinates in other dimensions. The second fill mode indicates that the sub-data returned by the fill request is not located in memory.

22. An electronic device, comprising: At least one processor; At least one memory, including one or more computer program modules, The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the data loading method according to any one of claims 1-20.

23. A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein, When the computer instructions are executed by at least one processor, the data loading method according to any one of claims 1-20 is performed.

Images