Data processing method and device, processor, electronic device, and storage medium

Abstract

A data processing method and device, a processor, an electronic device, and a storage medium. The data processing method comprises: performing a padding operation on an output gradient matrix loaded from a memory according to a padding mode of a forward propagation process to obtain a first gradient matrix and cache the first gradient matrix to a cache component of a processor, wherein the padding value filled in the padding operation is 0; reading the first gradient matrix from the cache component, and performing a backward propagation operation of average pooling on the first gradient matrix to obtain an input gradient matrix, wherein, during the backward propagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix, the shape size of the first kernel is the same as that of a convolution kernel of the forward propagation process, and both are RXS, the element value of each element in the first kernel is 1 / RXS, and R and S are positive integers. The data processing method uses calculation instead of the original judgment jump, reduces a large number of branch instructions, and greatly improves the processing performance.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, a data processing apparatus, a processor, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] During the training phase, the loss function affects the output feature map of the pooling layer. L / The gradient of Y needs to be backpropagated to the input feature map. L / X. For example, AveragePool performs uniform scattering, distributing the gradient evenly across the window size and writing it to all locations involved in the forward averaging. This process achieves upsampling and sparse / uniform distribution of the gradient tensor size, ensuring that the chain rule can continue to propagate to previous layers.

[0003] If the input feature map size is not divisible by the pooling window (e.g., input size 7×7, window size 2×2), the sliding window will "go out of bounds" when it reaches the edge, causing the output size to become smaller. In this case, some padding value (e.g., 0) needs to be added to the edges of the input feature map, i.e., a padding operation, to ensure that the output size meets the design requirements.

[0004] During backpropagation, the pooling operator needs to "restore" the gradient of the output tensor to the corresponding position in the input tensor. Currently, for backpropagation processes with padding, the calculation process needs to determine whether it is in a padded region, which introduces additional jump instructions, resulting in poor performance of the pooling operator. Summary of the Invention

[0005] This invention application provides at least one embodiment of a data processing method, comprising: performing a padding operation on an output gradient matrix loaded from memory according to a padding pattern of a forward propagation process to obtain a first gradient matrix and caching it in a cache unit of a processor, wherein the padding value of the padding operation is 0; reading the first gradient matrix from the cache unit and performing a backpropagation operation of average pooling on the first gradient matrix to obtain an input gradient matrix, wherein during the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix, the shape and size of the first kernel are the same as the shape and size of the convolution kernel of the forward propagation process and are both R×S, and the element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

[0006] For example, in a data processing method provided in at least one embodiment of this application, the filling mode indicates the filling position and filling size. Based on the filling mode of the forward propagation process, a filling operation is performed on the output gradient matrix loaded from memory to obtain a first gradient matrix and cache it in the processor's cache unit. This includes: determining the size of the first gradient matrix based on the filling position, the filling size, and the size of the output gradient matrix; determining the cache space size occupied by the first gradient matrix in the cache unit based on the size of the first gradient matrix; allocating a cache space with a capacity equal to the cache space size in the cache unit to cache the first gradient matrix; writing 0 to the region in the cache space where the filling operation is applied; and loading the output gradient matrix from memory to the corresponding position in the cache space according to the position of the output gradient matrix in the first gradient matrix.

[0007] For example, in the data processing method provided in at least one embodiment of this application, the sliding window convolution operation includes sliding the first kernel sequentially from left to right and from top to bottom on the first gradient matrix with a stride of 1. At each sliding position, the first kernel performs a corresponding element-wise multiplication and accumulation operation with the elements in the target window region. The target window region is the corresponding window region in the first gradient matrix determined by the current sliding position of the first kernel, and the result of the multiplication and accumulation operation at each sliding position is used as the element at the corresponding position in the input gradient matrix.

[0008] For example, in a data processing method provided in at least one embodiment of this application, the processor includes a graphics processor, the graphics processor includes multiple streaming processor clusters, each streaming processor cluster includes multiple computing units, each computing unit includes multiple execution units, each computing unit also includes shared memory shared by the multiple execution units, the cache component includes the shared memory, and the data processing method further includes: the multiple execution units performing backpropagation operations of average pooling of multiple first gradient matrices in parallel, or the multiple execution units jointly performing backpropagation operations of average pooling of a first gradient matrix, and each execution unit performing a portion of the backpropagation operations of average pooling of the first gradient matrix.

[0009] For example, in a data processing method provided in at least one embodiment of this application, before performing a filling operation on the output gradient matrix loaded from memory according to the filling pattern of the forward propagation process to obtain a first gradient matrix and cache it in the cache unit of the processor, the data processing method further includes: determining the size of the first gradient matrix, and determining the size of the cache space occupied by the first gradient matrix in the cache unit based on the size of the first gradient matrix; determining the number Q of the loaded output gradient matrices according to the relationship between the size of the cache space and the capacity of the shared memory, where Q is a positive integer and less than or equal to L, and L is a positive integer representing the number of the plurality of execution units.

[0010] For example, in the data processing method provided in at least one embodiment of the present invention, Q first gradient matrices obtained by filling Q output gradient matrices are loaded into the cache component, and L execution units execute the backpropagation operation of average pooling of the Q first gradient matrices in parallel.

[0011] For example, in the data processing method provided in at least one embodiment of the present invention, in response to Q equaling L, the L execution units perform the back propagation operation of the average pooling of the L first gradient matrices in parallel, and each execution unit independently performs the back propagation operation of the average pooling of the first gradient matrix to obtain the corresponding input gradient matrix.

[0012] For example, in the data processing method provided in at least one embodiment of this application, in response to Q equaling 1, the first gradient matrix is ​​split into L sub-tensors, and the average pooling backpropagation operation of each sub-tensor is performed in parallel by the L execution units.

[0013] For example, in a data processing method provided in at least one embodiment of this invention, splitting a first gradient matrix into L sub-tensors includes: dividing the first gradient matrix into the L sub-tensors; wherein, performing a backpropagation operation of average pooling on the first gradient matrix to obtain an input gradient matrix includes: for each sub-tensor, performing a sliding window convolution operation on the sub-tensor using a first kernel to obtain the part of the input gradient matrix corresponding to the sub-tensor.

[0014] For example, in a data processing method provided in at least one embodiment of this application, determining the number Q of the loaded output gradient matrices based on the relationship between the size of the cache space and the capacity of the shared memory includes: determining the number Q of the loaded output gradient matrices to be L in response to the shared memory capacity being greater than or equal to L times the size of the cache space; and determining the number Q of the loaded output gradient matrices to be 1 in response to the shared memory capacity being less than L times the size of the cache space but greater than or equal to the size of the cache space.

[0015] This invention provides at least one embodiment of a data processing apparatus, comprising: a loading module configured to perform a filling operation on an output gradient matrix loaded from memory according to a filling mode of a forward propagation process, to obtain a first gradient matrix and cache it in a cache unit of a processor, wherein the filling value of the filling operation is 0; and a processing module configured to read the first gradient matrix from the cache unit and perform a backpropagation operation of average pooling on the first gradient matrix to obtain an input gradient matrix, wherein during the backpropagation operation, a first kernel is used to perform a sliding window convolution operation on the first gradient matrix to obtain the input gradient matrix, the shape and size of the first kernel are the same as the shape and size of the convolution kernel in the forward propagation process and are both R×S, and the element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

[0016] This invention application provides at least one embodiment of a processor, including memory, a cache unit, a computing core, and a data loading module. The data loading module is configured to perform a padding operation on an output gradient matrix loaded from the memory according to a padding pattern of a forward propagation process, to obtain a first gradient matrix and cache it in the cache unit, wherein the padding value is 0. The computing core is configured to read the first gradient matrix from the cache unit and perform a backpropagation operation with average pooling on the first gradient matrix to obtain an input gradient matrix. During the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first core to obtain the input gradient matrix. The shape and size of the first core are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first core is 1 / R×S, where R and S are positive integers.

[0017] At least one embodiment of this application provides an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to implement the data processing method according to any embodiment of this application.

[0018] This application provides at least one embodiment of a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the data processing method according to any embodiment of this application.

[0019] The data processing method provided in at least one embodiment of this invention uses calculation instead of the original judgment jump, reducing a large number of branch instructions and significantly improving performance; furthermore, data can be directly read from the on-chip cache during processing, without having to read it repeatedly from off-chip, improving memory access speed and further improving performance. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0021] Figure 1A This is a schematic diagram of the backpropagation process of a pooling operator;

[0022] Figure 1B A schematic diagram of the forward propagation process is shown;

[0023] Figure 2 This is a schematic diagram of the architecture of a general-purpose graphics processing unit (GPGPU).

[0024] Figure 3 A schematic flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0025] Figure 4 A schematic diagram of hardware allocation for a backpropagation operation provided in an embodiment of this disclosure;

[0026] Figure 5 A schematic diagram of hardware allocation for a backpropagation operation provided in another embodiment of this disclosure;

[0027] Figure 6 A schematic block diagram of a data processing apparatus provided in at least one embodiment of this disclosure;

[0028] Figure 7 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure;

[0029] Figure 8 A schematic block diagram of an electronic device provided in one embodiment of this disclosure;

[0030] Figure 9 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0033] Figure 1A This is a schematic diagram of the backpropagation process of a pooling operator.

[0034] like Figure 1A As shown, the output gradient is represented by a shape and size of W×H, where W and H are positive integers. During forward pooling, the padding size in the width direction is pad_w, and the padding size in the height direction is pad_h, where pad_w and pad_h are both positive integers. For ease of understanding, in... Figure 1A The output gradient shows the filled region. Figure 1A In the diagram, the gray area represents the filled region added during the forward pooling process, and the white area represents the effective region, which is the aforementioned portion of the output gradient. The input gradient corresponding to the output gradient is obtained through backpropagation gradient calculation, and its shape and size are represented as input_w × input_h, where input_w and input_h are both positive integers.

[0035] During the backpropagation of the pooling operator, when calculating the input gradient, it is necessary to determine whether the data is within the valid region (non-filled region) of the output gradient. Specifically, when determining each element of the input gradient, it is necessary to first determine the n points in the output gradient that will affect that element, where n = pool_w × pool_w, and pool_w represents the size of the pooling window, such as 2 or 3. Then, based on the positions (e.g., coordinates) of these n points, it is determined which points belong to the valid region and which belong to the filled region. Elements located in the valid region participate in the calculation of the elements of the input gradient, while elements located in the filled region do not participate in the calculation.

[0036] Figure 1B A schematic diagram of the forward propagation process is shown.

[0037] like Figure 1B As shown, during the forward propagation process, the input feature map ( Figure 1B Fill the area around the 4x4 rectangle enclosed by the bolded border. Figure 1B In the example, both pad_h and pad_w are 1, and the padding elements are represented by gray boxes. Assuming the convolution kernel size is 2×2, then... Figure 1B The four elements at positions 1 on the left side of the image are convolved with the convolution kernel to obtain the element at position 1' in the output feature map.

[0038] In the backpropagation gradient calculation process, in order to obtain the backpropagation element value marked 1 in the input feature map during backpropagation, it is necessary to first determine the four points in the output feature map that will affect the element. Specifically, apart from the position marked 1' in the output feature map, the other three points do not belong to the output feature map (effective region). Therefore, the element located at the position marked 1' in the output feature map participates in the backpropagation calculation, while other points do not participate in the calculation. However, this process needs to be determined through a conditional judgment instruction.

[0039] In practical implementations, to determine whether the current pixel is within a valid region, the processor must insert an `if` statement or a ternary expression, leading to branch jumps. Referring to the process above, each of the H×W elements in the input gradient requires a conditional check, and each conditional check necessitates executing multiple instructions, significantly impacting performance. In other words, the padding operation, intended to fill in the edges of the input gradient, introduces the extra workload of "conditional checks and jumps" during backpropagation, increasing memory access overhead, computational load, control flow overhead, and overall performance degradation.

[0040] Furthermore, the current backpropagation computation requires loading elements from the current input gradient from memory to compute the elements in the output gradient needed for the computation. This method also introduces repeated data loading, and the latency of loading data from memory is relatively long, which further reduces the overall performance.

[0041] Figure 2 This is a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0042] like Figure 2 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0043] like Figure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Execution Units, each containing an Arithmetic Logic Unit (ALU), a floating-point unit, etc. Execution Units are used to perform specific computational tasks. Furthermore, a Computation Unit also includes registers (e.g., ...). Figure 2 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. Shared memory in a computing unit is used to share data between execution units within that computing unit.

[0044] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 2(Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0045] In each computing unit, the thread beam scheduling / distribution module ( Figure 2 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple execution units within the computation unit can run the thread bundles. Depending on the number of execution units in the computation unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory execution instructions are issued to shared memory within the computation unit or further issued to intermediate-level caches, global caches, or global memory for read and write operations, etc.

[0046] processor (e.g.) Figure 2 During the backpropagation of the pooling operator in the general-purpose graphics processor (GPCB) shown, it is necessary to determine whether the data is within the valid region of the output gradient for each element of the input gradient. This determination needs to be repeated, resulting in poor operator performance. In addition, the process also requires directly reading data from off-chip memory to the storage area of ​​the execution unit (such as shared memory). The memory access bandwidth is lower than that of on-chip memory access, which further reduces performance.

[0047] This disclosure provides at least one embodiment of a data processing method, processor, electronic device, and non-transitory computer-readable storage medium. The data processing method includes: performing a padding operation on an output gradient matrix loaded from memory according to a padding pattern of a forward propagation process to obtain a first gradient matrix and caching it in a cache unit of the processor, wherein the padding value is 0; reading the first gradient matrix from the cache unit and performing a backpropagation operation with average pooling on the first gradient matrix to obtain an input gradient matrix, wherein during the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix, the shape and size of the first kernel being the same as the shape and size of the convolution kernel in the forward propagation process and both being R×S, and the element value of each element in the first kernel being 1 / R×S, where R and S are positive integers.

[0048] In at least one embodiment, the data processing method uses computation instead of the original decision-making jump, reducing a large number of branch instructions and significantly improving performance; furthermore, data can be read directly from the on-chip cache during processing, eliminating the need to repeatedly read from off-chip, thus improving memory access speed and further enhancing performance.

[0049] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0050] Figure 3 This is a schematic flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0051] like Figure 3 As shown, at least one embodiment of the present disclosure provides a data processing method including steps S10-S20.

[0052] In step S10, according to the filling mode of the forward propagation process, the output gradient matrix loaded from memory is filled to obtain the first gradient matrix and cached in the processor's cache unit.

[0053] For example, the fill mode indicates the fill position and fill size, and the fill operation fills with a fill value of 0.

[0054] For example, the padding pattern can include parameters indicating the padding location, such as which edges of the output gradient matrix to pad on, for example, padding on the top, bottom, left, and right edges, or padding only on the top and left edges. The padding pattern can also include parameters indicating the padding size, such as pad_w and pad_h.

[0055] In at least one embodiment of this disclosure, the output gradient matrix is ​​filled according to the filling pattern of the forward propagation process. For example, if the forward propagation process is as follows: Figure 1B As shown, padding is added around the four edges, and both pad_w and pad_h are 1. The same operation is then performed on the output gradient matrix, specifically by adding a ring of padding elements with a value of 0 around the output gradient matrix.

[0056] For example, in some embodiments, the output gradient matrix loaded from memory is padded according to the padding pattern of the forward propagation process to obtain a first gradient matrix and cached in the processor's cache unit. This may include: determining the size of the first gradient matrix based on the padding location, padding size, and the size of the output gradient matrix; determining the cache space size occupied by the first gradient matrix in the cache unit based on the size of the first gradient matrix; allocating a cache space with a capacity equal to the cache space size in the cache unit to cache the first gradient matrix; writing 0 to the region in the cache space to which the padding operation is applied; and loading the output gradient matrix from memory to the corresponding position in the cache space according to the position of the output gradient matrix in the first gradient matrix.

[0057] Specifically, in this embodiment, the size of the first gradient matrix is ​​first determined based on the padding position, padding size, and the size of the output gradient matrix. Then, based on the size of the first gradient matrix, the size of the cache space occupied by the first gradient matrix in the caching component is determined. For example, the size of the cache space can be equal to the product of the size of the first gradient matrix and the bit width of its elements. Next, a cache space with a capacity equal to the size of the cache space is allocated in the caching component to cache the first gradient matrix. Then, a zero-writing operation is performed on the region to be filled in the cache space, and the data of the output gradient matrix is ​​then filled into the middle of this region. This loading method can minimize redundant loading and improve loading performance.

[0058] For example, in other embodiments, after allocating a cache space of the specified size, the cache space can be filled with 0s, and then the output gradient matrix can be loaded to the corresponding position.

[0059] In step S20, the first gradient matrix is ​​read from the cache component, and the backpropagation operation of average pooling is performed on the first gradient matrix to obtain the input gradient matrix.

[0060] The forward propagation process of average pooling is as follows: for each pooling window, take the average value of all elements as the output.

[0061] During the backpropagation operation of average pooling, a sliding window convolution operation is performed on the first gradient matrix using the first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×C. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

[0062] For example, if the kernel size used in the forward propagation is 3×3, then the shape size of the first kernel is also 3×3, and each element has a value of 1 / 9. For example, if the kernel size used in the forward propagation is 2×2, then the shape size of the first kernel is also 2×2, and each element has a value of 0.25.

[0063] For example, the sliding window convolution operation includes sliding the first kernel sequentially from left to right and from top to bottom on the first gradient matrix with a stride of 1. At each sliding position, the first kernel performs element-wise multiplication and accumulation operations with the corresponding elements in the target window region. The target window region is the window region in the first gradient matrix determined by the current sliding position of the first kernel, and the result of the multiplication and accumulation operations at each sliding position is used as the element at the corresponding position in the input gradient matrix.

[0064] The specific process of sliding window convolution operation can be referred to as the conventional convolution operation process, the difference being that the first kernel is used as the convolution kernel for the convolution operation.

[0065] For example, for the first sliding window convolution operation, the top-left element of the first kernel is matched with the top-left element of the first gradient matrix. The region in the first gradient matrix corresponding to the first kernel is determined as the target window region. Then, the multiplication and accumulation operation of the corresponding elements in the first kernel and the target window region is performed to obtain the output result as the top-left element of the input gradient matrix.

[0066] Then, slide the first kernel one step to the right and continue the above process. That is, take the window area determined by the first kernel in the first gradient matrix at the current sliding position as the target window area, perform the multiplication and accumulation calculation of the corresponding elements in the first kernel and the newly determined target window area, and obtain the output result as the adjacent right element of the top left element in the input gradient matrix.

[0067] The subsequent process is similar and will not be repeated here.

[0068] Thus, the input gradient matrix corresponding to the output gradient matrix is ​​obtained through the above process.

[0069] In at least one embodiment of this disclosure, the input gradient matrix is ​​obtained through the above process, eliminating the need for further judgment processes and thus eliminating a large number of branch instructions. Although the elements located in the filling region participate in the calculation, their filling value is 0, so they do not affect the final calculation result. Moreover, although the amount of computation is increased, the calculation only requires one instruction. This method can eliminate a large number of branch instructions, each of which contains multiple instructions. By replacing the judgment with calculation, a large number of jump judgment processes can be eliminated, effectively improving performance. In addition, this disclosure loads all the data into the cache component first, and then retrieves the data from the cache component when performing the backpropagation operation, avoiding repeated loading of data from memory and effectively utilizing the high-speed bandwidth of the on-chip cache, improving data access performance and reducing access overhead.

[0070] For example, the output gradient tensor includes at least one output gradient matrix, each of size W×H, where W and H are positive integers. For example, the output gradient tensor can be a 3-dimensional tensor, such as [C,H,W], which includes output gradient matrices with C channels; or it can be a 4-dimensional tensor, such as [N,C,W,H], where N represents a batch, and the output gradient tensor includes N batches, each batch including output gradient matrices with C channels, thus the output gradient tensor can be considered to include N×C output gradient matrices. This disclosure does not impose specific limitations on the shape of the output gradient tensor.

[0071] For example, in some embodiments, the processor includes a graphics processor. (See reference) Figure 2The architecture diagram shown indicates that the graphics processor may include multiple streaming processor clusters, each streaming processor cluster includes multiple computing units, and each computing unit includes multiple execution units.

[0072] Each computing unit also includes shared memory shared by multiple execution units, and the cache component includes shared memory.

[0073] For example, in some embodiments, the data processing method further includes: multiple execution units performing backpropagation operations of average pooling of multiple first gradient matrices in parallel, or multiple execution units jointly performing backpropagation operations of average pooling of a first gradient matrix, with each execution unit performing a portion of the backpropagation operations of average pooling of a first gradient matrix.

[0074] For example, multiple first gradient matrices are obtained by performing the same padding operation on multiple output gradient matrices from the same output gradient tensor. Therefore, these multiple first gradient matrices have the same shape and size.

[0075] For example, in some embodiments, each execution unit independently executes the backpropagation operation of the average pooling of a first gradient matrix, so that multiple execution units can execute the backpropagation operation of the average pooling of multiple first gradient matrices in parallel, thereby improving hardware execution efficiency.

[0076] For example, in other embodiments, multiple execution units jointly perform a backpropagation operation of average pooling of a first gradient matrix, but each execution unit performs a portion of the backpropagation operation of average pooling of a first gradient matrix. Although this approach introduces some redundant computation, it can still improve the overall hardware execution efficiency.

[0077] Furthermore, since multiple execution units share the shared memory in the computing unit, the first gradient matrix can be loaded into the shared memory, which can be accessed by multiple execution units without having to load data from memory again, thus greatly improving memory access performance and reducing redundant data loading.

[0078] For example, in some embodiments, before step S10, the data processing method provided in at least one embodiment of this disclosure further includes: determining the size of the first gradient matrix, and determining the size of the cache space occupied by the first gradient matrix in the cache component based on the size of the first gradient matrix; determining the number Q of the loaded output gradient matrices according to the relationship between the size of the cache space and the capacity of the shared memory, where Q is a positive integer and less than or equal to L, and L is a positive integer and represents the number of multiple execution units.

[0079] For example, in some embodiments, determining the number Q of the loaded output gradient matrices based on the relationship between the cache size and the shared memory capacity may include: determining the number Q of the loaded output gradient matrices to be L in response to the shared memory capacity being greater than or equal to L times the cache size; and determining the number Q of the loaded output gradient matrices to be 1 in response to the shared memory capacity being less than L times the cache size but greater than or equal to the cache size.

[0080] In this embodiment, to reuse hardware resources and ensure that no execution units idle and waste resources during the backpropagation operation of the average pooling of the output gradient tensor, all execution units participate in the backpropagation operation of the average pooling of the output gradient tensor, maximizing the use of hardware resources, improving resource utilization, and enhancing processing performance. Of course, in other embodiments, some execution units may be configured not to perform the backpropagation operation of the average pooling of the output gradient tensor, i.e., Q is less than L but greater than 1. This disclosure does not impose specific limitations on this.

[0081] For example, in some embodiments, Q first gradient matrices obtained by filling Q output gradient matrices are loaded into a cache unit, and the backpropagation operation of average pooling of the Q first gradient matrices is executed in parallel by L execution units.

[0082] For example, in response to Q equaling L, L execution units perform the backpropagation operation of average pooling of the first gradient matrix in parallel, while each execution unit independently performs the backpropagation operation of average pooling of the first gradient matrix and obtains the corresponding input gradient matrix.

[0083] Figure 4 This is a schematic diagram of the hardware allocation for a backpropagation operation provided in an embodiment of this disclosure.

[0084] For example, four output gradient matrices are stored in memory, namely... Figure 4 The output gradient matrix consists of three matrices: 0, 1, 2, and 3, each with a size of 4×4. The fill mode indicates that the fill position is the four sides of the output gradient matrix, and the fill size is 1. The size of the first gradient matrix is ​​5×5.

[0085] Based on the shape and size of the first gradient matrix, determine the size1 of the cache space occupied by the first gradient matrix in the shared cache.

[0086] like Figure 4 As shown, if L=4, and the capacity of the shared cache is greater than or equal to 4×size1, it means that the shared cache can hold 4 first gradient matrices at the same time. In this case, 4 first gradient matrices can be loaded into the shared cache at the same time.

[0087] For example, during the process of loading each output gradient matrix into the shared buffer, such as Figure 4 As shown, adding a ring of zeros around the output gradient matrix yields the first gradient matrix corresponding to each output gradient matrix, which is then cached in a shared cache. For example... Figure 4 The first gradient matrix 0, first gradient matrix 1, first gradient matrix 2 and first gradient matrix 3 are in the matrix.

[0088] like Figure 4 As shown, the computation unit includes four execution units: execution unit 0, execution unit 1, execution unit 2, and execution unit 3. In this embodiment, the four execution units in the computation unit are configured to perform the backpropagation operation of the average pooling of four first gradient matrices in parallel. Specifically, each execution unit independently performs the backpropagation operation of the average pooling of one first gradient matrix to obtain the corresponding input gradient matrix. The backpropagation operation of the average pooling of each first gradient matrix can be referred to the relevant content of step S20, and will not be repeated here.

[0089] Therefore, this disclosure not only eliminates a large number of loop instruction jumps, but also independently allocates backpropagation operations of different average pooling to different execution units, minimizing uneven splitting and redundant calculations, and improving hardware resource utilization.

[0090] For example, in other embodiments, in response to Q equaling 1, the 1 first gradient matrix is ​​split into L sub-tensors, and the average pooling backpropagation operation of the L sub-tensors is performed in parallel by L execution units.

[0091] For example, in some embodiments, splitting a first gradient matrix into L sub-tensors may include: dividing the first gradient matrix into L equal sub-tensors.

[0092] Since the execution unit can only execute the next task after all execution units have completed their tasks, the first gradient matrix is ​​divided into L sub-tensors. The average pooling backpropagation operation of each sub-tensor is executed in parallel by L execution units. This can make the L execution units finish their tasks at the same time as much as possible, reduce or avoid waiting time, and improve computational efficiency.

[0093] For example, in some embodiments, if the size of the first gradient matrix cannot be evenly divided, the size of each sub-tensor can be distributed as evenly as possible so that the L execution units can complete the computation task simultaneously or with the least time difference.

[0094] For example, in this embodiment, performing a backpropagation operation of average pooling on the first gradient matrix to obtain the input gradient matrix may include: for each sub-tensor, performing a sliding window convolution operation on the sub-tensor using the first kernel to obtain the portion of the input gradient matrix corresponding to the sub-tensor. The specific sliding window convolution process can be found in the aforementioned related content; repeated details will not be elaborated further.

[0095] Figure 5 A schematic diagram of the hardware allocation for the backpropagation operation provided in another embodiment of this disclosure.

[0096] For example, multiple output gradient matrices are stored in memory, each with a size of 4×4. The padding mode indicates that the padding location is the four sides of the output gradient matrix, indicating a padding size of 1, and the first gradient matrix has a size of 5×5.

[0097] Based on the shape and size of the first gradient matrix, determine the size2 of the cache space occupied by the first gradient matrix in the shared cache.

[0098] like Figure 5 As shown, if L=4, and the capacity of the shared cache is less than 4×size2 but greater than or equal to size2, it means that the shared cache cannot hold 4 first gradient matrices at the same time, but can hold 1 first gradient matrix. In this case, 1 output gradient matrix can be loaded into the shared cache.

[0099] For example, during the process of loading the output gradient matrix 0 into the shared buffer, such as Figure 5 As shown, a ring of zeros is added around the output gradient matrix 0 to obtain the first gradient matrix 0 corresponding to the output gradient matrix 0 and cache it in the shared cache.

[0100] like Figure 5 As shown, the computation unit includes four execution units: execution unit 0, execution unit 1, execution unit 2, and execution unit 3. In this embodiment, the four execution units in the computation unit are configured to jointly perform the backpropagation operation of the average pooling of the first gradient matrix 0.

[0101] Specifically, the first gradient matrix 0 consists of four sub-tensors, each with a size of 3×3. A single execution unit independently performs the backpropagation operation of average pooling for one sub-tensor. The four execution units perform the backpropagation operation of average pooling for all four sub-tensors in parallel to obtain the input gradient matrix corresponding to the output gradient matrix 0. The backpropagation operation of average pooling for each sub-tensor can be found in step S20; the process is similar and will not be repeated here.

[0102] In this embodiment, although the boundary portion of the subtensor may result in some redundant data loading during the backpropagation operation of average pooling, this embodiment still eliminates a large number of loop instruction jumps, and the data is always read from the shared cache, making full use of the on-chip shared cache space. Without having to read from off-chip memory, memory access overhead can still be reduced and hardware performance improved.

[0103] For example, if the shared memory in computing unit 1 cannot completely store a first gradient matrix, the existing technology is adopted, that is, the required data is loaded from memory one by one into the execution unit, and the back propagation operation of the average pooling of the output gradient matrix is ​​performed in the manner of "judgment + jump + calculation", which will not be elaborated here.

[0104] At least one embodiment of this disclosure also provides a data processing apparatus. Figure 6 This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0105] like Figure 6 As shown, the data processing device 100 includes a loading module 101 and a processing module 102.

[0106] Loading module 101 is configured to perform a filling operation on the output gradient matrix loaded from memory according to the filling mode of the forward propagation process, to obtain the first gradient matrix and cache it in the processor's cache unit, wherein the filling value of the filling operation is 0.

[0107] The processing module 102 is configured to read the first gradient matrix from the cache component, perform backpropagation operation of average pooling on the first gradient matrix, and obtain the input gradient matrix.

[0108] During the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using the first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

[0109] For example, loading module 101 and processing module 102 include code and programs stored in memory. Processing module 102 is implemented as a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities. The processing unit can be a general-purpose processor, or a microcontroller, microprocessor, digital signal processor, dedicated image processing chip, or field-programmable logic array, etc. Loading module 101 and processing module 102 execute the code and programs to implement some or all of the functions of loading module 101 and processing module 102 as described above. For example, loading module 101 and processing module 102 can be a circuit board or a combination of multiple circuit boards for implementing the functions described above. In the embodiments of this application, the circuit board or the combination of multiple circuit boards may include: (1) one or more processors; (2) one or more non-temporary memories connected to the processor; and (3) processor-executable firmware stored in memory.

[0110] It should be noted that loading module 101 can be used to implement Figure 3 The step S10 shown; the processing module 102 can be used to implement Figure 3 The steps are shown in step S20. For a detailed description of the functions that the loading module 101 can perform, please refer to the description of step S10 in the embodiments of the data processing method described above. For a detailed description of the functions that the processing module 102 can perform, please refer to the description of step S20 in the embodiments of the data processing method described above. Repeated descriptions will not be repeated here. Furthermore, the data processing device 100 can achieve similar technical effects to the aforementioned data processing method, and will not be repeated here.

[0111] It should be noted that in at least one embodiment of this disclosure, the data processing device 100 may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0112] For example, the data processing device 100 may be implemented in hardware, software, or a combination of hardware and software, and this disclosure does not impose any specific limitations on it.

[0113] The data processing apparatus provided in at least one embodiment of this disclosure can achieve similar technical effects to the data processing methods described above, and will not be repeated here.

[0114] The data processing method and data processing apparatus provided in at least one embodiment of this disclosure can be applied to different systems or devices, such as those used in... Figure 8The illustrated electronic device 300 can be a terminal, such as a mobile phone, tablet computer, laptop computer, AR device, VR device, vehicle terminal, etc., or it can be a server. The data processing method provided in at least one embodiment of this disclosure can be applied to scenarios involving backpropagation computation with average pooling in the electronic device 300, such as tensor computation units, involving high-performance computing (HPC) and artificial intelligence (AI). Of course, this disclosure is not limited to this; any scenario, device, or apparatus involving backpropagation computation with average pooling can employ the data processing method or data processing apparatus provided in at least one embodiment of this disclosure.

[0115] In some embodiments, the data processing apparatus provided in at least one embodiment of this disclosure can be a chip, such as a system-on-a-chip (SoC). A SoC includes a processor, which can be a single-core or multi-core processor, memory, and I / O interfaces. The processor can load data and applications from memory and then process the data, such as performing backpropagation calculations for average pooling.

[0116] It should be noted that the output gradient matrix and input gradient matrix of the data processing method or data processing apparatus provided in at least one embodiment of this disclosure may have different physical meanings depending on the application scenario. For example, the data processing method provided in at least one embodiment of this disclosure can be applied to fields such as speech processing, image processing, text processing, and video processing.

[0117] For example, in the field of speech processing, the output gradient matrix and the input gradient matrix can be the input and output parameters of backpropagation calculation involving average pooling in tasks such as feature extraction, speech enhancement, and speech recognition.

[0118] For example, in the field of image processing, the output gradient matrix and the input gradient matrix can be the input and output parameters of backpropagation calculation involving average pooling in tasks such as image recognition, feature extraction, image segmentation, object detection, image classification, and scene reconstruction.

[0119] For example, in the field of text processing, the output gradient matrix and the input gradient matrix can be the input and output parameters of backpropagation calculation involving average pooling in tasks such as text classification, sentiment analysis, and text generation.

[0120] For example, in the field of video processing, the output gradient matrix and the input gradient matrix can be the relevant parameters in the field of image processing as described above, or the input and output parameters of backpropagation calculation involving average pooling, such as optical flow operators (used to estimate motion between video frames) and target tracking operators (used to track specific targets in video).

[0121] Of course, this disclosure is not limited to this. For other application scenarios or fields, as long as backpropagation calculation of average pooling is required, the data processing method described in at least one embodiment of this disclosure can be applied, and will not be described in detail here.

[0122] Figure 7 A schematic structural diagram of a processor provided for at least one embodiment of this disclosure.

[0123] like Figure 7 As shown, the processor 200 includes memory 201, cache unit 202, computing core 203 and data loading module 204.

[0124] For example, cache unit 202 may include shared memory. For example, processor 200 may employ the aforementioned... Figure 2 The graphics processor or general-purpose graphics processor architecture shown.

[0125] Memory 201 could be, for example, high-bandwidth memory.

[0126] The data loading module 204 is configured to perform a filling operation on the output gradient matrix loaded from memory according to the filling mode of the forward propagation process, to obtain the first gradient matrix and cache it in the cache component, wherein the filling value of the filling operation is 0.

[0127] The computational core 203 is configured to read the first gradient matrix from the cache component 202, perform backpropagation operation of average pooling on the first gradient matrix, and obtain the input gradient matrix.

[0128] For example, computational core 203 could be Figure 2 The execution unit in the processor 200. For example, in some embodiments, the computing core 203 may be other components in the processor 200 capable of performing calculations, and this disclosure does not impose specific limitations on this.

[0129] During the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using the first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

[0130] The specific operation of the data loading module 204 in performing the step of "filling the output gradient matrix loaded from memory according to the filling mode of the forward propagation process, obtaining the first gradient matrix and caching it to the cache component" can be referred to the relevant description of step S10 in the aforementioned data processing method embodiment. Repeated parts will not be repeated.

[0131] For example, with the processor as the reference Figure 2 The graphics processor architecture shown includes multiple streaming processor clusters, each streaming processor cluster includes multiple computing units, and each computing unit includes multiple execution units.

[0132] For example, each computation unit can be configured to perform backpropagation operations of average pooling of one or more output gradient matrices, and multiple computation units can perform backpropagation operations of average pooling of multiple output gradient matrices in an output gradient tensor in parallel, thereby making full use of the parallel capabilities of the graphics processor and accelerating the processing.

[0133] like Figure 2 As shown, each computing unit also includes shared memory shared by multiple execution units, and the cache component includes the shared memory.

[0134] For example, multiple execution units are configured to perform backpropagation operations of average pooling of multiple first gradient matrices in parallel, or to perform backpropagation operations of average pooling of a single first gradient matrix together, with each execution unit performing a portion of the backpropagation operation of that single first gradient matrix.

[0135] For example, referring to the description of the relevant embodiments in the aforementioned data processing method, taking computing unit 1 as an example, if the shared memory in computing unit 1 can simultaneously hold L first gradient matrices, then L output gradient matrices are loaded from memory, and they are filled according to the forward filling mode to obtain L first gradient matrices cached in shared memory; the L execution units included in computing unit 1 are configured to execute the back propagation operation of the average pooling of L first gradient matrices in parallel, and each execution unit is configured to independently execute the back propagation operation of the average pooling of one first gradient matrix. This method can eliminate a large number of loop jump instructions, avoid repeated calculations, and the task allocation method is reasonable, effectively improving the utilization of hardware resources and efficiently realizing the back propagation operation of the average pooling operator.

[0136] For example, continuing with computation unit 1, if the shared memory in computation unit 1 cannot hold L first gradient matrices simultaneously, but can hold 1 first gradient matrix, then load 1 output gradient matrix from memory and perform filling operations on it according to the forward filling mode to obtain 1 first gradient matrix cached in shared memory; the first gradient matrix is ​​divided into L sub-tensors, and the L execution units included in computation unit 1 are configured to execute the back propagation operation of average pooling of the L sub-tensors in parallel. This method can eliminate a large number of loop jump instructions and efficiently implement the back propagation operation of the average pooling operator.

[0137] For example, if the shared memory in computation unit 1 cannot hold one first gradient matrix, the existing technology is used to load the data directly from memory to the execution unit and perform the back propagation operation of the average pooling of the output gradient matrix using the "judgment + jump + calculation" method.

[0138] For details on the specific process of using computation kernel 203 to perform the backpropagation operation of average pooling for each first gradient matrix / sub-tensor, please refer to step S20 in the data processing method described above. Repeated steps will not be repeated here.

[0139] The processor provided in at least one embodiment of this disclosure can achieve similar technical effects to the aforementioned data processing method, and the repeated parts will not be described again.

[0140] Figure 8 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. For example... Figure 8 As shown, the electronic device 300 is, for example, suitable for implementing the data processing method provided in the embodiments of this disclosure. It should be noted that... Figure 8 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0141] like Figure 8 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0142] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the data processing method according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the data processing method can be found in the relevant descriptions in the embodiments of the above data processing method.

[0143] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0144] For example, the processing device 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0145] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 8 An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, a processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processor (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0146] Figure 9 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure.

[0147] For example, such as Figure 9As shown, storage medium 400 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 401 can be stored non-transitory. For example, when the computer-readable instructions 401 are executed by a processor, one or more steps in the data processing method described above can be performed.

[0148] For example, the storage medium 400 can be used in an electronic device 300, such as the storage medium 400 including the storage device 308 in the electronic device 300.

[0149] For example, a storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and a processor may execute these instructions to perform various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0150] For example, the storage medium may include a memory card for a smartphone, a cache component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0152] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0153] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0154] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0155] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0156] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0157] The following points should be noted regarding this disclosure:

[0158] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0159] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0160] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data processing method, characterized in that, include: According to the filling pattern of the forward propagation process, the output gradient matrix loaded from memory is filled to obtain the first gradient matrix and cached in the processor's cache unit. The filling value of the filling operation is 0. A cache space with a capacity determined based on the size of the first gradient matrix is ​​requested in the cache unit to cache the first gradient matrix. The first gradient matrix is ​​read from the cache component, and a backpropagation operation with average pooling is performed on the first gradient matrix to obtain the input gradient matrix. In the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

2. The data processing method according to claim 1, characterized in that, The fill pattern indicates the fill location and fill size. Based on the padding pattern of the forward propagation process, the output gradient matrix loaded from memory is padded to obtain the first gradient matrix, which is then cached in the processor's cache unit, including: The size of the first gradient matrix is ​​determined based on the fill position, the fill size, and the size of the output gradient matrix; Based on the size of the first gradient matrix, determine the size of the cache space occupied by the first gradient matrix in the cache component; The cache component allocates a cache space with a capacity equal to the size of the cache space to cache the first gradient matrix; Write 0 to the region in the cache space to which the fill operation is applied; According to the position of the output gradient matrix in the first gradient matrix, the output gradient matrix is ​​loaded from the memory to the corresponding position in the cache space.

3. The data processing method according to claim 1, characterized in that, The sliding window convolution operation involves sliding the first kernel sequentially from left to right and from top to bottom on the first gradient matrix with a stride of 1. At each sliding position, the first kernel performs element-wise multiplication and accumulation operations with the elements in the target window region, wherein the target window region is the corresponding window region in the first gradient matrix determined by the current sliding position of the first kernel, and the result of the multiplication and accumulation operation at each sliding position is used as the element at the corresponding position in the input gradient matrix.

4. The data processing method according to claim 1, characterized in that, The processor includes a graphics processing unit (GPU), which includes multiple streaming processor clusters. Each streaming processor cluster includes multiple computing units, and each computing unit includes multiple execution units. Each computing unit also includes shared memory shared by the plurality of execution units, and the cache component includes the shared memory. The data processing method further includes: The backpropagation operation of average pooling of multiple first gradient matrices is performed in parallel by the multiple execution units, or... The multiple execution units jointly perform a backpropagation operation of the average pooling of a first gradient matrix, and each execution unit performs a portion of the backpropagation operation of the average pooling of the first gradient matrix.

5. The data processing method according to claim 4, characterized in that, Before filling the output gradient matrix loaded from memory according to the filling pattern of the forward propagation process to obtain the first gradient matrix and caching it in the processor's cache, the data processing method further includes: Determine the size of the first gradient matrix, and determine the size of the cache space occupied by the first gradient matrix in the cache component based on the size of the first gradient matrix; Based on the relationship between the size of the cache space and the capacity of the shared memory, the number Q of the loaded output gradient matrices is determined, where Q is a positive integer and less than or equal to L, and L is a positive integer representing the number of the plurality of execution units.

6. The data processing method according to claim 5, characterized in that, The Q first gradient matrices obtained by filling the Q output gradient matrices are loaded into the cache unit, and the average pooling backpropagation operation of the Q first gradient matrices is executed in parallel by L execution units.

7. The data processing method according to claim 6, characterized in that, In response to Q equaling L, the L execution units perform the backpropagation operation of the average pooling of the first gradient matrix in parallel. Each execution unit independently performs the backpropagation operation of the average pooling of the first gradient matrix and obtains the corresponding input gradient matrix.

8. The data processing method according to claim 6, characterized in that, In response to Q equaling 1, the Q first gradient matrices are split into L sub-tensors, and the average pooling backpropagation operation of each sub-tensor is performed in parallel by the L execution units.

9. The data processing method according to claim 8, characterized in that, The Q first gradient matrices are split into L sub-tensors, including: Divide the Q first gradient matrices into the L sub-tensors; The first gradient matrix is ​​subjected to backpropagation with average pooling to obtain the input gradient matrix, which includes: For each sub-tensor, the sliding window convolution operation is performed on the sub-tensor using the first kernel to obtain the portion of the input gradient matrix corresponding to the sub-tensor.

10. The data processing method according to claim 6, characterized in that, Based on the relationship between the cache space size and the shared memory capacity, the number Q of the loaded output gradient matrices is determined, including: In response to the shared memory having a capacity greater than or equal to L times the size of the cache space, the number of output gradient matrices Q loaded is determined to be L; In response to the shared memory having a capacity less than L times the size of the cache space but greater than or equal to the size of the cache space, the number of output gradient matrices Q loaded is determined to be 1.

11. A data processing apparatus, characterized in that, include: The loading module is configured to perform a filling operation on the output gradient matrix loaded from memory according to the filling mode of the forward propagation process, to obtain a first gradient matrix and cache it in the processor's cache unit, wherein the filling operation fills with a filling value of 0, and the cache unit requests a cache space with a capacity determined based on the size of the first gradient matrix to cache the first gradient matrix. The processing module is configured to read the first gradient matrix from the cache component, perform a backpropagation operation of average pooling on the first gradient matrix, and obtain the input gradient matrix. In the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

12. A processor, characterized in that, Includes memory, cache components, computing cores, and data loading modules. The data loading module is configured to perform a filling operation on the output gradient matrix loaded from the memory according to the filling mode of the forward propagation process, to obtain a first gradient matrix and cache it in the cache component, wherein the filling operation fills with a filling value of 0, and a cache space with a capacity determined based on the size of the first gradient matrix is ​​requested in the cache component to cache the first gradient matrix. The computational kernel is configured to read the first gradient matrix from the cache component, perform a backpropagation operation of average pooling on the first gradient matrix, and obtain the input gradient matrix. In the backpropagation operation, a sliding window convolution operation is performed on the first gradient matrix using a first kernel to obtain the input gradient matrix. The shape and size of the first kernel are the same as those of the convolution kernel in the forward propagation process, both being R×S. The element value of each element in the first kernel is 1 / R×S, where R and S are positive integers.

13. An electronic device, characterized in that, include: Memory stores computer-executable instructions non-transiently; The processor is configured to run computer-executable instructions. The computer-executable instructions are executed by the processor to implement the data processing method according to any one of claims 1-10.

14. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they implement the data processing method according to any one of claims 1-10.

Images