Data processing method and apparatus, electronic device, and storage medium

Abstract

A data processing method and device, an electronic device, and a storage medium are used to store a pooling operation result of input data in a first register according to a storage granularity of the first register. The data processing method includes: obtaining input data; performing first processing on the input data to obtain X groups of preprocessed data; performing a pooling operation on a plurality of elements in M channels of each preprocessed sampling group to obtain at least one pooling operation result of each channel in the M channels of each preprocessed sampling group as output data; and sequentially storing N output sampling groups of each output channel in N storage groups in a plurality of storage groups. The data processing method realizes efficient transmission of data for a pooling operation in a register, and improves computing performance.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, a data processing apparatus, an electronic device, and a storage medium. Background Technology

[0002] Global pooling layers are widely used in various convolutional neural networks to replace fully connected layers. Compared to fully connected layers, global pooling layers rely only on the input data itself during computation, without requiring additional training weight parameters. This effectively reduces computational cost and prevents overfitting. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a data processing method for storing the pooling operation result of input data in a first register according to the storage granularity of a first register, wherein the first register includes multiple storage groups, each of the multiple storage groups includes M storage subgroups, and M is a positive integer. The method includes: acquiring the input data, wherein the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, and N is a positive integer; performing a first processing on the input data to obtain X sets of preprocessed data, wherein each set of preprocessed data in the X sets of preprocessed data includes multiple preprocessed sampling groups, each of the multiple preprocessed sampling groups includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements, the number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, and X is a positive integer; and performing a first processing on each preprocessed sampling group... Pooling operations are performed on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group, which is used as output data. The X groups of preprocessed data correspond to X groups of output data, each group of output data includes M output channels, and each of the M output channels includes N output sampling groups. The N output sampling groups in each output channel are sequentially stored in the N storage groups of the plurality of storage groups. The M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are sequentially stored in the M consecutive storage subgroups of the Kth storage group, where 1 ≤ K ≤ N and K is an integer.

[0004] For example, in the data processing method provided in at least one embodiment of this disclosure, the number of output channels of the X groups of output data is the same as the number of channels included in each sampling group of the input data, and the number of output sampling groups included in each output channel of each group of output data is the same as the number of sampling groups of the input data.

[0005] For example, in the data processing method provided in at least one embodiment of this disclosure, the step of performing pooling operations on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group includes: performing pooling operations on multiple elements in the M channels of each preprocessed sampling group in at least one second register to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group.

[0006] For example, in the data processing method provided in at least one embodiment of this disclosure, the size of the data corresponding to each channel of the input data includes a first size. The first processing of the input data to obtain X sets of preprocessed data includes: fixing the number of channels in each sampling group, splicing the input data along the direction of the first size according to the correspondence of the positions of multiple channels in multiple sampling groups to obtain first updated data; dividing the first updated data into the X sets of preprocessed data, wherein the first updated data includes Y first updated sampling groups, where Y is a positive integer, each first updated sampling group includes multiple channels, and the number of channels in each first updated sampling group is the same as the number of channels in each sampling group.

[0007] For example, in the data processing method provided in at least one embodiment of this disclosure, the size of the data corresponding to each channel in the first updated data includes a first update size, the input data includes N sampling groups, the first update size is N / Y times the first size, the storage granularity of the second register includes a third size, and the first update size is an integer multiple of the third size.

[0008] For example, in the data processing method provided in at least one embodiment of this disclosure, the first size is the height of the data corresponding to each channel in the input data, the first update size is the height of the data corresponding to each channel in the first update data, and the third size is the height of the data corresponding to each register channel in the second register, wherein the height refers to the number of elements along the column direction in the array formed by the two-dimensional arrangement of multiple elements; or, the first size is the width of the data corresponding to each channel in the input data, the first update size is the width of the data corresponding to each channel in the first update data, and the third size is the width of the data corresponding to each register channel in the second register, wherein the width refers to the number of elements along the row direction in the array formed by the two-dimensional arrangement of multiple elements.

[0009] For example, in the data processing method provided in at least one embodiment of this disclosure, each of the at least one second register includes M register channels, and the step of dividing the first updated data into the X groups of preprocessed data includes: selecting the data corresponding to each of the M channels in the Y first update sampling groups as a group, dividing the first updated data into the X groups of second updated data, wherein each group of second updated data in the X groups of second updated data includes Y second update sampling groups, each second update sampling group includes M channels, and the size of the data corresponding to each channel in the second updated data includes a second update size; fixing the number of channels in each of the Y second update sampling groups, and according to the correspondence of the positions of multiple channels in the multiple second update sampling groups, splitting each group of second updated data in the X groups of second updated data along the direction of the second update size to obtain the X groups of preprocessed data.

[0010] For example, in the data processing method provided in at least one embodiment of this disclosure, the size of the data corresponding to each channel in the first updated data includes a first update size, and the second update size is the same as the first update size. The first update size is the height of the data corresponding to each channel in the first updated data, and the second update size is the height of the data corresponding to each channel in the second updated data. The height refers to the number of elements along the column direction in an array formed by two-dimensional arrangement of multiple elements; or, the first update size is the width of the data corresponding to each channel in the first updated data, and the second update size is the width of the data corresponding to each channel in the second updated data. The width refers to the number of elements along the row direction in an array formed by two-dimensional arrangement of multiple elements.

[0011] For example, in the data processing method provided in at least one embodiment of this disclosure, each of the at least one second register includes M register channels. The step of performing pooling operations on multiple elements in the M channels of each preprocessing sampling group to obtain at least one pooling operation result for each of the M channels of each preprocessing sampling group includes: storing the data corresponding to the M channels of each preprocessing sampling group in the at least one second register according to a one-to-one correspondence between the M channels of each preprocessing sampling group and the M register channels of each second register, and according to a correspondence between the size of the data corresponding to the M channels of each preprocessing sampling group and the storage granularity of the second register; performing pooling operations on multiple elements in each register channel stored in the at least one second register to obtain at least one pooling operation result for each register channel, wherein the at least one pooling operation result for each register channel is at least one pooling operation result corresponding to at least one channel of the input data.

[0012] For example, in a data processing method provided in at least one embodiment of this disclosure, the pooling operation includes summing and calculating the average value.

[0013] For example, in the data processing method provided in at least one embodiment of this disclosure, the storage granularity of the second register includes a third size, the size of the data corresponding to the M channels in each preprocessing sampling group includes a fourth size, and the correspondence between the size of the data corresponding to the M channels in each preprocessing sampling group and the storage granularity of the second register includes the correspondence between the fourth size and the third size, wherein the fourth size is not greater than the sum of the third sizes of the at least one second register.

[0014] For example, in the data processing method provided in at least one embodiment of this disclosure, the third dimension is the height of the data corresponding to each register channel in the second register, and the fourth dimension is the height of the data corresponding to the M channels in each preprocessing sampling group, wherein the height refers to the number of elements along the column direction in the array formed by the two-dimensional arrangement of multiple elements; or, the third dimension is the width of the data corresponding to each register channel in the second register, and the fourth dimension is the width of the data corresponding to the M channels in each preprocessing sampling group, wherein the width refers to the number of elements along the row direction in the array formed by the two-dimensional arrangement of multiple elements.

[0015] For example, in the data processing method provided in at least one embodiment of this disclosure, each of the N output sampling groups in each output channel includes a pooling operation result, and the step of storing the N output sampling groups in each output channel in the N storage groups in the plurality of storage groups in turn includes: storing the N pooling operation results in the N output sampling groups in each output channel in the N storage groups in turn.

[0016] For example, at least one embodiment of the data processing method provided in this disclosure further includes: storing the X groups of output data in memory.

[0017] For example, in at least one embodiment of the data processing method provided in this disclosure, storing the X groups of output data in memory includes: acquiring the X groups of output data, wherein the X groups of output data include multiple pooling operation results stored in X storage groups, the multiple pooling operation results including the first pooling operation result to the Pth pooling operation result, where P is a positive integer; calculating the Qth address offset of the Qth pooling operation result in memory using the storage location correspondence between the first register and the memory, where 1≤Q≤P and Q is an integer; and storing the Qth pooling operation result in memory at the memory address corresponding to the Qth address offset using a first storage instruction based on the Qth address offset.

[0018] At least one embodiment of this disclosure also provides a data processing apparatus for storing the pooling operation result of input data in a first register according to the storage granularity of a first register, wherein the first register includes multiple storage groups, each of the multiple storage groups includes M storage subgroups, and M is a positive integer. The apparatus includes: an acquisition module configured to acquire the input data, wherein the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, and N is a positive integer; a processing module configured to perform a first processing on the input data to obtain X sets of preprocessed data, wherein each set of preprocessed data in the X sets of preprocessed data includes multiple preprocessed sampling groups, each of the multiple preprocessed sampling groups includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements, the number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, and X is a positive integer; and a calculation module configured to calculate the M channels of each preprocessed sampling group. Pooling operations are performed on multiple elements to obtain at least one pooling operation result for each of the M channels in each preprocessed sampling group, which is used as output data. The X groups of preprocessed data correspond to X groups of output data. Each group of output data includes M output channels, each of the M output channels includes N output sampling groups, and each of the N output sampling groups includes a pooling operation result. A storage module is configured to sequentially store the N output sampling groups in each output channel into N storage groups. The M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are sequentially stored in M ​​consecutive storage subgroups of the Kth storage group, where 1 ≤ K ≤ N and K is an integer.

[0019] For example, in the data processing apparatus provided in at least one embodiment of this disclosure, the computing module is further configured to: perform pooling operations on multiple elements in the M channels of each preprocessed sampling group in at least one second register to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group.

[0020] For example, in a data processing apparatus provided in at least one embodiment of this disclosure, the storage module is further configured to store the X groups of output data in memory.

[0021] At least one embodiment of this disclosure also provides an electronic device. The electronic device includes: a processor; and a memory including one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data processing method provided in any embodiment of this disclosure.

[0022] At least one embodiment of this disclosure also provides a storage medium storing non-transitory computer-readable instructions that, when executed by a computer, implement the data processing method provided in any embodiment of this disclosure. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure, and are not intended to limit this disclosure.

[0024] Figure 1A This is a schematic diagram illustrating the input data generation process of a global pooling layer.

[0025] Figure 1B A schematic diagram illustrating an example of an input data format;

[0026] Figure 2 An exemplary flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0027] Figure 3 A schematic diagram illustrating an example of the data format of input data provided in at least one embodiment of this disclosure;

[0028] Figure 4 A schematic diagram illustrating an example of the data format for preprocessed data provided in at least one embodiment of this disclosure;

[0029] Figure 5 A schematic diagram illustrating the first register storage granularity provided for at least one embodiment of this disclosure;

[0030] Figure 6 A schematic diagram illustrating an example of a second register storage granularity provided for at least one embodiment of this disclosure;

[0031] Figure 7 for Figure 2 An exemplary flowchart of step S120;

[0032] Figure 8 A schematic diagram illustrating an example of the data format of the first update data provided in at least one embodiment of this disclosure;

[0033] Figure 9 for Figure 2 An exemplary flowchart of step S130;

[0034] Figure 10A A schematic diagram illustrating the storage method of multiple pooling operation results in a first register, provided in at least one embodiment of this disclosure;

[0035] Figure 10B A schematic diagram illustrating an example of how multiple pooling operation results are stored in a first register, as provided in at least one embodiment of this disclosure;

[0036] Figure 11 for Figure 2 An exemplary flowchart of step S150;

[0037] Figure 12 This is a schematic diagram illustrating an example of storing multiple pooling operation results in a first register into memory, provided for at least one embodiment of this disclosure;

[0038] Figure 13 A schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure;

[0039] Figure 14 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure;

[0040] Figure 15 A schematic block diagram of another electronic device provided for at least one embodiment of this disclosure; and

[0041] Figure 16 This is a schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0043] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0044] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of the embodiments of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0045] For neural networks, the deeper the network, the more information it can capture, and the richer its features become. However, increasing network depth can lead to gradient explosion and vanishing gradient problems, resulting in poorer optimization performance and lower accuracy on both training and test data. Residual neural networks (ResNet) and other network architectures can enable deeper networks to achieve better training results.

[0046] For example, global pooling layers are widely used in various convolutional neural networks such as ResNet. Convolutional neural networks achieve classification in the last layer through fully connected layers. However, fully connected layers have a large number of parameters, which increases the computational cost of training and testing, reducing computational speed; furthermore, a large number of parameters can easily lead to overfitting. Replacing fully connected layers with global pooling layers can reduce the number of parameters, effectively suppress overfitting, and allow for more flexible input data sizes. For example, global pooling includes global average pooling (GAP) and global max pooling. During forward propagation, for the feature maps output by the convolutional layers, the global max pooling layer can calculate the maximum value of each feature map and output the vector composed of all maximum values; the global average pooling layer can calculate the average value of each feature map and output the vector composed of all average values.

[0047] Figure 1AThis is a schematic diagram illustrating the input data generation process of a global pooling layer.

[0048] For example, such as Figure 1A As shown, convolutional layers are used to extract features from two-dimensional data such as images and videos. Depending on the specific task (classification, regression, image segmentation, etc.), different types of network structures are used for convolutional layers. For example, for deep neural networks such as ResNet50, the features extracted by the convolutional layers can be input into a non-linear activation function layer (such as a BN_Add_ReLU layer) for non-linear processing such as data normalization, resulting in the input data for the global pooling layer.

[0049] For example, such as Figure 1A As shown, the data format (also known as "shape") of the input data is [N,C,H,W], where N is the number of sample groups, C is the number of channels in each sample group, H is the height of the input data in each channel, and W is the width of the input data in each channel.

[0050] Figure 1B This is a schematic diagram illustrating an example of an input data format.

[0051] For example, such as Figure 1B As shown, the input data includes multiple sampling groups, and each sampling group includes multiple channels. In each channel, multiple data elements are arranged in a two-dimensional array. The height H of the input data in each channel is the number of elements along the column direction of the array, and the width W of the input data in each channel is the number of elements along the row direction of the array.

[0052] For example, Figure 1B The input data includes 2 sampling groups, i.e., the number of sampling groups N=2; each sampling group includes 16 channels, i.e., the number of channels in each sampling group C=16; the number of elements in the data element array along the column direction in each channel is 5, i.e., the height of the input data in each channel H=5; the number of elements in the data element array along the row direction in each channel is 4, i.e., the width of the input data in each channel W=4.

[0053] For example, taking ResNet50 as an example, the input data format can be [N, 2048, 7, 7], that is, the input data includes 2048 channels, each channel being a 7×7 two-dimensional array composed of data elements; the global average pooling layer can calculate the average value of the 7×7 two-dimensional array in each channel as the output (for the global max pooling layer, it can calculate the maximum value of the 7×7 two-dimensional array in each channel as the output). Since each channel only outputs an average value (or maximum value) after the global pooling operation, the output data format of the global pooling layer is [N, 2048], that is, the output data format of the global pooling layer is [N, C], keeping the N and C dimensions of the input data unchanged.

[0054] For example, in neural networks such as ResNet50, for global pooling layers and the previous layer (e.g., Figure 1A Data transfer between non-linear activation function layers (in the context of global pooling) uses registers as the data transfer medium to improve transmission performance. For example, for BF16 data types, each register stores data in a fixed 2×32 data storage format. If the output data of the global pooling layer is in the format [N,C], each register will store 2 sample groups × 32 channels of output data at a time, and then store the output data in memory in the format of 2 sample groups × 32 channels for subsequent calculations. For example, performing subsequent calculations requires 16 parallel computing units, each computing unit calculating 32 channels of output data at a time. Therefore, to perform one calculation, 16 parallel computing units process 16×32=512 channels of output data at once. However, the data processing granularity of the computing units used to perform the operations of the previous layer (e.g., BN_Add_ReLU layer) is much smaller than 512. This inconsistency in data processing granularity prevents the global pooling layer and the non-linear activation function layer from directly transferring data in registers.

[0055] Furthermore, due to hardware limitations such as the register data storage structure, for each sample group N of input data, the non-linear activation function layer (e.g., the BN_Add_ReLU layer) processes the data by first fixing the channel direction and then processing all H×W channels, while the global pooling layer processes the input data by first fixing H×W channels and then processing all channels. This inconsistency in data processing order also prevents data from being directly transferred through registers. The non-linear activation function layer must first store the calculation results in memory, and then the global pooling layer loads the input data from memory, significantly reducing data transfer efficiency.

[0056] At least one embodiment of this disclosure provides a data processing method for storing the pooling operation result of input data in a first register according to the storage granularity of a first register. The first register includes multiple storage groups, each of the multiple storage groups including M storage subgroups, where M is a positive integer. The method includes: acquiring input data, wherein the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, where N is a positive integer; performing a first processing on the input data to obtain X sets of preprocessed data, wherein each of the X sets of preprocessed data includes multiple preprocessed sampling groups, each of the multiple preprocessed sampling groups includes M channels, the data corresponding to each channel of each preprocessed sampling group includes multiple elements, the number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, where X is a positive integer; and performing a pooling operation on the multiple elements in the M channels of each preprocessed sampling group. The process obtains at least one pooling operation result for each of the M channels in each preprocessed sampling group as output data. Here, X groups of preprocessed data correspond to X groups of output data. Each group of output data in the X groups includes M output channels. Each of the M output channels includes N output sampling groups. Each of the N output sampling groups includes a pooling operation result. The N output sampling groups in each output channel are stored sequentially in N storage groups. The M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are stored sequentially in M ​​consecutive storage subgroups of the Kth storage group, where 1≤K≤N and K is an integer.

[0057] At least one embodiment of this disclosure also provides a data processing apparatus, an electronic device, and a storage medium.

[0058] The method, apparatus, device, and storage medium provided in at least one embodiment of this disclosure change the way pooling operation output data is stored in registers, ensuring the consistency of data processing granularity of output data in memory with the data processing granularity in the previous layer network, and ensuring the consistency of data processing order in the pooling operation with the data processing order in the previous layer network, thereby realizing efficient transmission of data used for pooling operations in registers and improving computing performance.

[0059] At least one embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals will be used to refer to the same elements described in different drawings.

[0060] Figure 2 An exemplary flowchart of a data processing method provided in at least one embodiment of this disclosure.

[0061] For example, such as Figure 2 As shown, at least one embodiment of this disclosure provides a data processing method for performing pooling operations on input data and storing the pooling operation results in a first register according to the storage granularity of a first register. For example, the data processing method includes the following steps S110 to S140.

[0062] Step S110: Obtain input data;

[0063] Step S120: Perform a first processing on the input data to obtain X sets of preprocessed data;

[0064] Step S130: Perform pooling operations on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group as output data, where M is a positive integer;

[0065] Step S140: Store the N output sampling groups in each output channel in N storage groups in a sequence.

[0066] In some examples, the data processing method may also include step S150:

[0067] Step S150: Store the X group of output data in memory.

[0068] For example, in step S110, the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, where N is a positive integer. For example, the data format of the input data is [N1,C1,H1,W1], where N1 is the number of sampling groups (i.e., N1 = N), C1 is the number of channels in each sampling group, H1 is the height of the input data in each channel, and W1 is the width of the input data in each channel.

[0069] Figure 3 This is a schematic diagram illustrating an example of the data format for input data provided in at least one embodiment of this disclosure. For example, in Figure 3 In the example shown, the input data is in the format [16,2048,7,7]. The input data includes 16 sampling groups, i.e., N1 = N = 16; each sampling group includes 2048 channels, i.e., C1 = 2048; and the data corresponding to each channel includes 7×7 elements.

[0070] For example, such as Figure 3 As shown, in each channel, multiple elements are arranged in a two-dimensional array, and the k-th channel of the i-th sampling group contains the element array E. 1k =[E 11 -k,…,E 17 -k; ...; E71 -k,…,E 77 [-k], where E represents the elements of the input data, where k = 1, 2, ..., 2048 and i = 1, 2, ..., 16. The element array E 1k The number of elements along the column direction is 7, meaning the height H1 of the input data for each channel is 7; element array E 1k The number of elements along the row direction is 7, which means that the width of the input data in each channel is W1 = 7.

[0071] It should be noted that the input data N1, C1, H1, and W1 can also be selected with other different values ​​according to actual needs, and the embodiments disclosed herein do not impose any restrictions on this.

[0072] For example, in step S120, the first processing of the input data can be a process of transforming the data format of the input data. This transformation process only changes the data format without changing the value of each element. For example, the input data can be processed into X sets of preprocessed data after the first processing. Each set of preprocessed data in the X sets includes multiple preprocessed sampling groups. Each preprocessed sampling group includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements.

[0073] For example, the data format for each preprocessed data set is [N4, C4, H4, W4], where N4 is the number of preprocessed sample groups in each preprocessed data set, C4 is the number of channels in each preprocessed sample group (i.e., C4 = M), H4 is the height of the preprocessed data in each channel, and W4 is the width of the preprocessed data in each channel. It should be noted that the number of channels in each sample group of the input data differs from the number of channels in each preprocessed sample group.

[0074] Figure 4 This is a schematic diagram illustrating an example of a data format for preprocessed data provided in at least one embodiment of this disclosure. For example, Figure 4 The 1024 sets of preprocessed data shown are Figure 3 The input data shown is obtained through the first processing step. Figure 4 In the example shown, the data format of each preprocessed data set is [4,2,28,7]. Each preprocessed data set includes 4 preprocessed sampling groups, i.e., N4 = 4; each preprocessed sampling group includes 2 channels C4-1 and C4-2, i.e., C4 = M = 2; the data corresponding to each channel includes 28 × 7 elements.

[0075] For example, such as Figure 4 As shown, in each channel, multiple elements are arranged in a two-dimensional array. For one of the 1024 preprocessed data sets, the k-th channel of the i-th preprocessed sampling group contains the element array E. 4k=[E' 11 -k,…,E' 17 -k;…;E' B1 -k,…,E' B7 -k], where E' represents an element of the preprocessed data, where k = 1 or 2, i = 1, 2, 3, 4, and B = 28. Element array E 4k The number of elements along the column direction is 28, meaning the height H4 of the preprocessed data for each channel is 28; element array E 4k The number of elements along the row direction is 7, which means that the width of the preprocessed data in each channel is W4 = 7.

[0076] It should be noted that, depending on the data structure of the input data and the specific processing method of the first processing, the N4, C4, H4, W4 and the number of groups of the preprocessed data can also be other different values, and the embodiments of this disclosure do not limit this.

[0077] In step S130, pooling operations are performed on multiple elements of each channel in each preprocessed sampling group to obtain at least one pooling operation result for each channel. For example, X groups of preprocessed data are pooled to obtain X groups of output data. Each group of output data in the X groups includes M output channels, and each of the M output channels includes N output sampling groups.

[0078] It should be noted that, in the data processing method provided in at least one embodiment of this disclosure, the number of output channels of X groups of output data is the same as the number of channels included in each sampling group of input data, and the number of output sampling groups included in each output channel of each group of output data is the same as the number of sampling groups of input data.

[0079] For example, each sampling group of the input data includes X×M channels, and the total number of channels in the X groups of output data is also X×M; the input data includes N sampling groups, and each output channel of the output data also includes N output sampling groups. Therefore, in the data processing method provided in at least one embodiment of this disclosure, the total number of channels in the output data is the same as the number of channels in each sampling group of the input data (both are X×M), and the number of output sampling groups in the output data is also the same as the number of sampling groups in the input data (both are N).

[0080] It should be noted that the pooling operation can be a global average pooling operation (e.g., summing and averaging) or a global max pooling operation, or other pooling operation types. The embodiments of this disclosure do not limit this.

[0081] For example, with Figure 4 Taking the 1024 sets of preprocessed data shown as an example, for multiple elements [E'] of the k-th channel of the i-th preprocessed sampling group... 11-k,…,E' 17-k ;…;E' B1-k ,…,E' B7-k Perform pooling operations (e.g., summation and averaging) to obtain at least one pooling result for the k-th channel.

[0082] For example, 1024 sets of preprocessed data are pooled to obtain 1024 sets of output data (i.e., X = 1024); each set of output data includes 2 output channels (i.e., M = 2), and each of the 2 output channels includes 16 output sampling groups (i.e., N = 16).

[0083] For example, each sample group of the input data includes 2048 channels (X×M=1024×2=2048), and the total number of channels in the 1024 sets of output data is also 2048 (X×M=1024×2=2048); the input data includes 16 sample groups (N=16), and each output channel of the output data also includes 16 output sample groups (N=16). Therefore, in the above example, the total number of channels in the output data is the same as the number of channels in each sample group of the input data (both are 2048), and the number of output sample groups in the output data is also the same as the number of sample groups in the input data (both are 16).

[0084] In step S140, the X groups of output data are stored in the first register according to the storage granularity of the first register. For example, the storage granularity of the first register includes the number of storage groups and the number of storage subgroups (also called "register channels").

[0085] Figure 5 This is a schematic diagram illustrating a first register storage granularity provided for at least one embodiment of the present disclosure. For example, such as... Figure 5 As shown, the first register includes multiple storage groups, and each storage group includes M storage subgroups. For example, N storage groups can be selected from the multiple storage groups, and the N output sampling groups of each output channel of each group of output data can be stored sequentially in the N storage groups. The M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are stored sequentially in the M consecutive storage subgroups of the Kth storage group, where 1≤K≤N and K is an integer.

[0086] For example, in some examples, each of the N output sampling groups in each output channel may include a pooling operation result, that is, the N output sampling groups in each output channel include N pooling operation results; therefore, the N pooling operation results in the N output sampling groups in each output channel can be stored sequentially in N storage groups.

[0087] For example, such as Figure 5 As shown, each storage group also includes at least one interval group, and the M storage subgroups of different storage groups are separated by at least one interval group. For example, the interval groups can be set according to actual hardware requirements, and no data is stored in the interval groups.

[0088] It should be noted that if the number of storage groups in the first register is greater than or equal to N, then N sets of output data can be stored in one first register; if the number of storage groups in the first register is less than N, then N sets of output data can be stored in multiple first registers.

[0089] For example, with Figure 4 The example shown is 1024 sets of output data (i.e., X = 1024) obtained from 1024 sets of preprocessed data. For example, each set of output data includes 2 output channels (i.e., M = 2). 16 storage groups (N = 16) can be selected from multiple storage groups. The 16 output sampling groups in each output channel of each set of output data are stored sequentially in the 16 storage groups. The two pooling operation results corresponding to the two output channels in the Kth storage group of the 16 storage groups are stored sequentially in two consecutive storage subgroups of the Kth storage group (e.g., storage subgroup K-1 and storage subgroup K-2), where 1 ≤ K ≤ 16 and K is an integer.

[0090] In step S150, according to the correspondence between the storage locations of the first register and memory, the X sets of output data stored in the first register are stored in memory to perform subsequent operations (e.g., calculations in the next layer of the global pooling layer).

[0091] The data processing method provided in this embodiment first fixes the direction of channel C and then processes all H×W. Therefore, the data format of the output data generated from the input data [N,C] (where N is the number of sampling groups and C is the number of channels) becomes [C,N] (where N can be the number of output sampling groups and C is the number of output channels). That is, the two dimensions N and C are interchanged, thereby ensuring the consistency of the data processing order in the pooling operation process with the data processing order in the previous layer (e.g., the nonlinear activation function layer).

[0092] For example, each set of output data in the X sets of output data stored in the first register has a storage structure of M (output channels / number of storage subgroups) × N (output sampling groups / number of selected storage groups), which is a [C,N] storage structure (where C is the number of output channels / number of storage subgroups M, and N is the number of output sampling groups / number of selected storage groups N). The output data is stored in memory according to this storage structure in step S150. In subsequent operations in memory (e.g., calculations in the layer following the global pooling layer), parallel computation is performed according to channel C. Since M is much smaller than N, the data processing granularity of parallel computation is reduced, thereby ensuring the consistency of data processing granularity between the pooling layer (e.g., the global pooling layer) and the previous layer (e.g., the nonlinear activation function layer).

[0093] The data processing method provided in at least one embodiment of this disclosure changes the way pooling operation output data is stored in registers, ensuring the consistency of data processing granularity in memory with data processing granularity in the previous layer network, and ensuring the consistency of data processing order in pooling operation with data processing order in the previous layer network, thereby achieving efficient transmission of data used for pooling operation in registers and improving computing performance.

[0094] For example, in some examples, in step S130, pooling operations can be performed on multiple elements in the M channels of each preprocessed sample group of each preprocessed data in at least one second register to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sample group. At least one second register is used to store each preprocessed sample group to be pooled, and the computing module (e.g., a computing unit in a chip) can directly read data from at least one second register to perform the pooling operation.

[0095] It should be noted that the pooling operation on multiple elements in the M channels of each preprocessed sampling group in step S130 may not be performed in at least one second register. Other processing hardware (e.g., cache, etc.) can be selected according to actual needs. The embodiments of this disclosure do not limit this.

[0096] For example, each preprocessed sample group needs to be stored in at least one second register according to the storage granularity of the second register. The storage granularity of the second register is, for example, [C3, H3, W3], where C3 is the number of register channels of the second register, H3 is the height of the storage space in each register channel, and W3 is the width of the storage space in each register channel.

[0097] For example, the number of register channels C3 in the second register is the same as the number of channels C4 in each preprocessing sampling group, i.e., C3 = C4 = M; each register channel has a two-dimensional storage space for storing multiple elements in the data, H3 is the number of elements that can be stored in the two-dimensional storage space along the column direction, and W3 is the number of elements that can be stored in the two-dimensional storage space along the row direction.

[0098] Figure 6 This is a schematic diagram illustrating an example of a second register storage granularity provided for at least one embodiment of this disclosure. For example, Figure 6 The example shown uses the second register, for instance, to store... Figure 4 Each preprocessed sample group in the example. For example, such as Figure 6 As shown, each preprocessed sample group is stored, for example, in the example second register in BF16 format. The storage granularity of the example second register is, for example, [2, 4, 8]. That is, the example second register includes two register channels C3-0 and C3-1 (and...). Figure 4 In the example, the number of channels C4 in each preprocessing sampling group is the same (i.e., C3 = C4 = M = 2), the height of the storage space in each register channel is 4 (i.e., H3 = 4), and the width of the storage space in each register channel is 8 (i.e., W3 = 8).

[0099] For example, such as Figure 6 As shown, the two-dimensional storage space in the example second register register channels C3-0 and C3-1 contains subspace arrays of storable elements, namely S0 = [S 11 ,…,S 17 ;…;S 71 ,…,S 77 ] and S1 = [S' 11 ,…,S' 17 ;…;S' 71 ,…,S' 77 ], where S represents the subspace of the storage channel C3-0 that can store elements, and S' represents the subspace of the storage channel C3-1 that can store elements.

[0100] It should be noted that, Figure 6 The second register shown is only an example. The second register can also be selected with other different storage granularities (or other formats, such as FP64, FP32, FP16, etc.) according to actual needs. The embodiments of this disclosure do not limit this.

[0101] For example, in order to obtain preprocessed data that can be stored in at least one second register at the granularity of the second register, the input data can be subjected to first processing such as data format transformation, so that each preprocessed sample group can be stored in at least one second register to perform pooling operations.

[0102] Figure 7 For Figure 2 an exemplary flowchart of step S120 in

[0103] For example, the size of the data corresponding to each channel of the input data includes a first size, and the first size can be the height H1 of the data corresponding to each channel in the input data, or the first size can also be the width W1 of the data corresponding to each channel in the input data. For example, in some examples, in order to perform a first processing on the input data to obtain X sets of preprocessed data, Figure 2 step S120 in

[0104] can include the following steps S121 to S122.

[0105] Step S121: Keep the number of channels in each sampling group fixed, and splice the input data along the direction of the first size according to the corresponding relationship of the positions of the multiple channels in the multiple sampling groups to obtain a first updated data;

[0106] In step S121, the elements in the channels at the corresponding positions in the multiple sampling groups are spliced along the height H1 direction (or the width W1 direction), and the data format of the spliced first updated data changes. The data format of the first updated data is, for example, [N2, C2, H2, W2].

[0107] For example, the first updated data includes Y first updated sampling groups, and the number of first updated sampling groups is N2 (i.e., N2 = Y), where Y is a positive integer; each first updated sampling group includes multiple channels, and the number of channels is C2. For example, the number of channels C2 in each first updated sampling group is the same as the number of channels C1 in each sampling group, i.e., C2 = C1; the number of first updated sampling groups N2 decreases, i.e., N2 < N1; the height H2 of the input data in each channel of the first updated data increases, i.e., H2 > H1 (or the W2 of the input data in each channel increases, i.e., W2 > W1). It should be noted that this splicing process only changes the data format and does not change the value of each element.

[0108] For example, the size of the data corresponding to each channel in the first updated data includes a first updated size, and the first updated size can be the height H2 of the data corresponding to each channel in the first updated data, or the first updated size can also be the width W2 of the data corresponding to each channel in the first updated data.

[0109] It should be noted that, in order to ensure that the total number of data elements remains unchanged, the factor by which the first update sampling group decreases relative to the input data sampling group needs to be the same factor by which the first update size increases relative to the first size. For example, if the input data includes N sampling groups (i.e., N1 = N), the first update size H2 (or W2) is N / Y times the first size H1 (or W1), that is... (or ).

[0110] It should be noted that, since the purpose of concatenating the input data into the first update data is to obtain preprocessed data that matches the storage granularity of the second register, the concatenated first update size H2 (or W2) needs to be an integer multiple of the height H3 (or width W3) of the storage space in each register channel of the second register. For example, the storage granularity of the second register includes a third size, which can be the height H3 of the storage space in each register channel, or the width W3 of the storage space in each register channel; the first update size H2 (or W2) must be an integer multiple of the third size H3 (or W3).

[0111] For example, if the first dimension is the height H1 of the data corresponding to each channel in the input data, then the first update dimension is the height H2 of the data corresponding to each channel in the first update data, and the third dimension is the height H3 of the data corresponding to each register channel in the second register, where height refers to the number of elements along the column direction in the array formed by the two-dimensional arrangement of multiple elements; or, if the first dimension is the width W1 of the data corresponding to each channel in the input data, then the first update dimension is the width W2 of the data corresponding to each channel in the first update data, and the third dimension is the width W3 of the data corresponding to each register channel in the second register, where width refers to the number of elements along the row direction in the array formed by the two-dimensional arrangement of multiple elements.

[0112] Figure 8 A schematic diagram illustrating an example of the data format of the first update data provided for at least one embodiment of this disclosure. Figure 8 The first updated data shown is, for example, Figure 3 The input data shown is obtained through the processing in step S121. That is, the number of channels C1 in each sampling group is fixed, and the input data is spliced ​​along the direction of the first dimension (the height H1 direction in this example) according to the correspondence of the positions of multiple channels in multiple sampling groups to obtain the first updated data.

[0113] For example, in Figure 8In the example shown, the data format of the first updated data is [2,2048,56,7]. The first updated data includes two first updated sampling groups, i.e., N2 = Y = 2; each first updated sampling group includes 2048 channels (the same number of channels C1 in each sampling group of the input data), i.e., C2 = C1 = 2048; the data corresponding to each channel includes 56 × 7 elements, and the data element corresponding to each channel is obtained by concatenating the elements in the corresponding channel of the corresponding sampling group of the input data along the direction of the first dimension (e.g., Figure 8 In the example, the first dimension is the height H1).

[0114] For example, such as Figure 8 As shown, in each channel, multiple elements are arranged in a two-dimensional array. The k-th channel of the i-th first update sampling group contains the element array E. 2k =[E” 11 -k,…,E” 17-k ;…;E” A1-k ,…,E” A7-k ], where E represents the first element to update the data, where k = 1, 2, ..., 2048, i = 1 or 2, and A = 56. Element array E 2k The number of elements along the column direction is 56, meaning the height H2 of the input data for each channel is 56; element array E 2k The number of elements along the row direction is 7, which means that the width of the input data in each channel is W1 = 7.

[0115] For example, such as Figure 3 and Figure 8 As shown, the input data includes 16 sampling groups (i.e., N1 = N = 16), the first update data includes 2 first update sampling groups (i.e., N2 = Y = 2), and the first update size H2 (H2 = 56) is 16 / 2 times the first size H1 (H1 = 7), that is...

[0116] For example, by Figure 8 The preprocessed data obtained from further processing of the first updated data shown needs to be stored in at least one... Figure 5 In the exemplary second register shown. For example, as Figure 5 As shown, the third size H3 of the second register is 4; as Figure 8 As shown, the first update size H2 of the first updated data is 56; the first update size H2 (or W2) is an integer multiple of the third size H3 (or W3), that is, H2 / H3 = 56 / 4 = 14.

[0117] It should be noted that, depending on the data structure of the input data and the specific processing method for converting the input data into the first updated data (the specific concatenation method is not limited to...), the specific concatenation method may vary. Figure 8The examples in the document, and the specific processing methods are not limited to splicing, splitting, dividing, etc., the N2, C2, H2, and W2 of the first updated data can be other different values, and the embodiments of this disclosure do not limit this.

[0118] In step S122, the first updated data is further divided into X groups of preprocessed data. Each preprocessed sample group of each group of preprocessed data can be stored in at least one second register. For example, each second register in at least one second register includes M register channels, that is, the number of register channels M included in each second register is the same as the number of channels M included in each preprocessed sample group. For example, in order to match the storage granularity of each preprocessed sample group with that of the second register, on the one hand, the number of channels M of each preprocessed sample group needs to be the same as the number of register channels M of the second register; on the other hand, the size of the data corresponding to the M channels of each preprocessed sample group also needs to match the size of the storage space in each register channel of the second register.

[0119] For example, the size of the data corresponding to the M channels in each preprocessed sampling group includes a fourth size, which can be the height H4 of the data corresponding to the M channels in each preprocessed sampling group, or the width W4 of the data corresponding to the M channels in each preprocessed sampling group. For example, the correspondence between the size of the data corresponding to the M channels in each preprocessed sampling group and the storage granularity of the second register includes the correspondence between the fourth size H4 (or W4) and the third size H3 (or W3). For example, in some examples, the fourth size H4 (or W4) is not greater than the sum of the third sizes H3 (or W3) of at least one second register, such that each preprocessed sampling group can be stored in at least one second register.

[0120] For example, if the third dimension is the height H3 of the data corresponding to each register channel in the second register, then the fourth dimension is the height H4 of the data corresponding to the M channels in each preprocessed sampling group; or, if the third dimension is the width W3 of the data corresponding to each register channel in the second register, then the fourth dimension is the width W4 of the data corresponding to the M channels in each preprocessed sampling group.

[0121] For example, in step S122, specifically, in order to divide the first updated data into X groups of preprocessed data, the data corresponding to each M channels in Y first updated sampling groups can be selected as a group, and the first updated data can be divided into X groups of second updated data. For example, each group of second updated data in the X groups includes Y second updated sampling groups, that is, the number of second updated sampling groups is the same as the number of first updated sampling groups; each second updated sampling group includes M channels, that is, the number of channels included in each second updated sampling group is the same as the number of channels M in each preprocessed sampling group.

[0122] For example, the size of the data corresponding to each channel in the second updated data includes a second updated size H2' (or W2'), which is the same as the first updated size H2 (or W2), i.e., H2' = H2, or W2' = W2. For example, if the first updated size is the height H4 of the data corresponding to each channel in the first updated data, then the second updated size is the height H2' of the data corresponding to each channel in the second updated data; or, if the first updated size is the width W2 of the data corresponding to each channel in the first updated data, then the second updated size is the width W2' of the data corresponding to each channel in the second updated data.

[0123] For example, further, the number of channels M in each of the Y second update sampling groups remains unchanged, and according to the correspondence of the positions of multiple channels in the multiple second update sampling groups, each group of second update data in the X groups is split along the direction of the second update size H2' to obtain X groups of preprocessed data.

[0124] For example, the number of channels M in each preprocessed sampling group is the same as the number of registered channels M in the second register; due to the further division of the second update size H2' (or W2') of the second update data, the fourth size H4 (or W4) corresponding to each preprocessed sampling group matches the third size H3 (or W3) of at least one second register. For example, in some examples, the fourth size H4 (or W4) obtained by division is not greater than the sum of the third sizes H3 (or W3) of at least one second register.

[0125] For example, with Figure 8 The first updated data shown is divided into Figure 4 Taking the 1024 sets of preprocessed data shown as an example, Figure 8 The first update data shown includes two first update sampling groups (i.e., N2 = Y = 2), each first update sampling group includes 2048 channels (i.e., C2 = 2048), and the first update size of the first update data is H2 = 56.

[0126] For example, in order to divide the first update data into 1024 groups of preprocessed data, we can first select the data corresponding to every 2 channels in the 2 first update sampling groups as a group and divide the first update data into 1024 groups of second update data.

[0127] For example, the data format of each of the 1024 sets of second update data is [2,2,56,7]. Each set of second update data includes two second update sampling groups, meaning the number of second update sampling groups is the same as the number of first update sampling groups (i.e., N2' = N2 = 2); each second update sampling group includes two channels, meaning the number of channels in each second update sampling group is the same as the number of channels M in each preprocessing sampling group (i.e., C2' = C4 = 2).

[0128] For example, the size of the data corresponding to each channel in the second updated data includes the second update size H2'. For example, in Figure 4 and Figure 8 In the example, the fourth dimension is height H4, the first updated dimension is height H2, and the second updated dimension is height H2'; the second updated dimension H2' is the same as the first updated dimension H2, that is, H2' = H2 = 56.

[0129] For example, further, the number of channels 2 in each second update sampling group is kept constant, and according to the correspondence of the positions of multiple channels in multiple second update sampling groups, each group of second update data in the 1024 groups of second update data is split along the direction of the second update size H2' to obtain 1024 groups of preprocessed data.

[0130] For example, such as Figure 4 As shown, each preprocessed sampling group obtained by partitioning includes 2 channels (i.e., C4 = M = 2); due to the further partitioning of the second update size H2' (H2' = 56) of the second update data, as... Figure 4 As shown, the fourth size H4 obtained by partitioning is 28, and the fourth size H4 obtained by partitioning is not greater than the sum of the third sizes H3 of at least one second register. For example, through the above partitioning process, the first update data is divided into 1024 groups of preprocessed data, each group of preprocessed data includes 4 preprocessed sampling groups (N4 = N2 × (H2 / H4) = 2 × (56 / 28) = 4), and each preprocessed sampling group contains 2 channels (C4 = M = 2).

[0131] For example, regarding the above process of dividing the first updated data into preprocessed data, it should be noted that:

[0132] (1) Depending on the data structure of the first updated data and the data structure of the preprocessed data (e.g., depending on the matching of the storage granularity of the preprocessed data and the second register), N2', C2', H2', and W2' of the second updated data can also be other different values; in addition, for the specific processing method of converting the first updated data into the second updated data, besides the data splitting method described in the above example, other processing methods (not limited to splicing, splitting, dividing, etc.) can also be selected according to actual needs, and the embodiments of this disclosure do not limit this;

[0133] (2) Regarding the specific processing method for converting the second updated data into preprocessed data, in addition to the data partitioning method described in the above examples, other processing methods (not limited to splicing, splitting, partitioning, etc.) can also be selected according to actual needs. The embodiments of this disclosure do not limit this.

[0134] (3) Depending on the degree of matching between the storage granularity of the first updated data and the second register, the first updated data can also be directly divided into preprocessed data without first splitting the first updated data into second updated data and then dividing the second updated data into preprocessed data; or, further, if the data format of the first updated data matches the storage granularity of the second register, splicing, splitting, and division can be skipped, and the first updated data can be directly stored as preprocessed data in at least one second register; the specific selection can be made according to actual needs, and the embodiments of this disclosure do not limit this.

[0135] For example, in some examples, it can be Figure 4 The i-th preprocessed sample group in the example is stored in at least one Figure 6 In the exemplary second register shown. For example, as Figure 4 and Figure 6 As shown, the number of channels in the i-th preprocessing sampling group is C4 = M = 2, and the number of register channels in the second register is C3 = M = 2, i.e., C4 = C3 = M = 2; the fourth size of the i-th preprocessing sampling group is H4 = 28, the third size of the second register is H3 = 4, and the fourth size H4 is not greater than the sum of the third sizes H3 of 7 second registers (i.e., H4 ≤ 7 × H3). Therefore, the data corresponding to each of the two channels in the i-th preprocessing sampling group can be stored in 7 second registers, i.e., the data corresponding to the two channels can be stored in a total of 2 × 7 second registers.

[0136] It should be noted that if the data format of the input data matches the storage granularity of the second register, the first processing in step S120 can be skipped, and the input data can be directly stored as preprocessed data in at least one second register. In addition, besides the data splicing, splitting, dividing or grouping methods in the above examples, other transformation processing methods can be selected for the first processing of the input data according to actual needs, and this disclosure embodiment does not limit this.

[0137] Figure 9 for Figure 2 An exemplary flowchart of step S130.

[0138] For example, in some examples, in order to perform pooling operations on multiple elements in the M channels of each preprocessed sample group, Figure 2 Step S130 may include the following steps S131 to S132.

[0139] Step S131: According to the one-to-one correspondence between the M channels of each preprocessing sampling group and the M register channels of each second register, and according to the correspondence between the size of the data corresponding to the M channels of each preprocessing sampling group and the storage granularity of the second register, store the data corresponding to the M channels of each preprocessing sampling group in at least one second register.

[0140] Step S132: Perform pooling operations on multiple elements in each register channel stored in at least one second register to obtain at least one pooling operation result for each register channel.

[0141] In step S131, since the number of register channels M included in each second register is the same as the number of channels M included in each preprocessing sampling group, the data corresponding to the M channels of each preprocessing sampling group can be stored one-to-one in the M register channels of each second register; and according to the correspondence between the size of the data corresponding to the M channels of each preprocessing sampling group and the storage granularity of the second register (for example, the correspondence between the height H4 and width W4 of the data corresponding to each channel in each preprocessing sampling group and the height H3 and width W3 of the data corresponding to the register channels of the second register), the data corresponding to the M channels of each preprocessing sampling group is stored in at least one second register.

[0142] In step S132, pooling operations are performed on multiple elements in each register channel stored in at least one second register to obtain at least one pooling operation result for each register channel; the at least one pooling operation result for each register channel is at least one pooling operation result corresponding to at least one channel of the input data.

[0143] For example, since the data corresponding to each channel of the input data includes H1×W1 elements, the array consisting of H4×W4 elements in each channel of each preprocessing sampling group can be split into at least one H1×W1 element array to calculate the pooling operation result of the H1×W1 elements corresponding to each channel of the input data.

[0144] For example, similarly, to Figure 4 The i-th preprocessed sample group in the example is stored in at least one Figure 6 Taking the exemplary second register as an example, since the number of register channels C3 included in each second register and the number of channels C4 included in the i-th preprocessing sampling group are both 2 (C4=C3=M=2), and the fourth size H4 is not greater than the sum of the third size H3 of the 7 second registers (i.e. H4≤7×H3), the data corresponding to each of the 2 channels in the i-th preprocessing sampling group can be stored in 7 second registers, that is, the data corresponding to the 2 channels can be stored in a total of 2×7 second registers.

[0145] For example, the data corresponding to the k-th channel in the i-th preprocessed sampling group (i.e., the element array E) 4k =[E' 11 -k,…,E' 17 -k;…;E' B1 -k,…,E' B7-k [Here, k = 1 or 2, B = 28]. Since each channel of the input data corresponds to 7×7 elements, a 28×7 element array E can be created. 4k The data is split into four 7×7 element arrays and stored in the k-th register channel of the seven second registers. To calculate the pooling operation result of the 7×7 elements corresponding to each channel of the input data, the following allocation method can be used:

[0146] (1) Store the first 7×7 element array in the 4×8 storage space of the first second register and the first 3×8 storage space of the second second register;

[0147] (2) Store the second 7×7 element array in the last 1×8 storage space of the second register, the 4×8 storage space of the third second register, and the first 2×8 storage space of the fourth second register;

[0148] (3) Store the third 7×7 element array in the last 2×8 storage space of the fourth second register, the 4×8 storage space of the fifth second register, and the first 1×8 storage space of the sixth second register;

[0149] (4) Store the fourth 7×7 element array in the last 3×8 storage space of the sixth second register and the 4×8 storage space of the seventh second register.

[0150] For example, pooling operations can be performed on the four 7×7 element arrays stored in the kth register channel of the seven second registers to obtain the four pooling operation results of the kth register channel; the four pooling operation results of the kth register channel are the four pooling operation results corresponding to the four channels of the input data.

[0151] For example, in some examples, the pooling operation can be a global average pooling operation, which can use hardware units (e.g., a general matrix multiplication reduction buffer (GRB)) to perform a summation operation of every 7×7 elements in the second register. Multiplying the sum of every 7×7 elements by 1 / 7×7 gives the average value of every 7×7 elements, and then one of the average values ​​can be used as the pooling operation result corresponding to one of the input data channels.

[0152] It should be noted that the above pooling operation processing method for preprocessed data is only an example. Other pooling operation processing methods can be selected according to different input data formats, pooling operation processing hardware (different from the second register), storage granularity of the second register, and actual needs. The embodiments disclosed herein do not limit this.

[0153] For example, in some examples, X sets of preprocessed data are pooled to obtain X sets of output data. These X sets of output data include M output channels, and each of the M output channels includes N output sample groups. For instance, each of the N output sample groups in each output channel can include one pooling result; that is, each of the N output sample groups in each output channel includes N pooling results, meaning each set of output data includes M×N pooling results, where X, M, and N are positive integers.

[0154] For example, Figure 4In the example, the 1024 sets of preprocessed data are pooled to obtain 1024 sets of output data (X = 1024). The data format of each set of output data is [2, 16]. Each set of output data includes 2 output channels, and each output channel includes 16 output sample groups. For example, each output sample group can include one pooling operation result. That is, the 16 output sample groups in each output channel include 16 pooling operation results, which means that each set of output data includes 2 × 16 pooling operation results. For example, as described above, the 1024 sets of output data include 32748 pooling operation results obtained from the pooling operation of the 1024 sets of preprocessed data. That is, 1024 (number of sets of preprocessed data) × 4 (number of preprocessed sampling groups per set of preprocessed data) × 2 (number of channels per preprocessed sampling group) × 4 (number of 7×7 element arrays corresponding to each channel) = 32748. In other words, X (number of sets of output data) × M (number of output channels per set of output data) × N (number of output sampling groups per output channel) = 1024 × 2 × 16 = 32748.

[0155] Figure 10A This is a schematic diagram illustrating the storage method of multiple pooling operation results in a first register, provided for at least one embodiment of this disclosure.

[0156] For example, such as Figure 10A As shown, the first register includes multiple storage groups, each of which includes M storage subgroups. Each of the X groups of output data includes M output channels, and each of the M output channels includes N output sample groups. For example, N storage groups can be selected from the multiple storage groups, and the N output sample groups of each output channel of each group of output data can be stored sequentially in the N storage groups. For example, in some examples, each of the N output sample groups in each output channel may include a pooling operation result, that is, the N output sample groups in each output channel include N pooling operation results; therefore, the N pooling operation results in the N output sample groups in each output channel can be stored sequentially in the N storage groups.

[0157] For example, each storage group can store M pooling operation results corresponding to M output channels. Therefore, the M pooling operation results corresponding to M output channels in the Kth storage group out of N storage groups can be sequentially stored in M ​​consecutive storage subgroups of the Kth storage group, where 1≤K≤N and K is an integer. That is, there is a one-to-one correspondence between storage groups and output sampling groups, and a one-to-one correspondence between storage subgroups and output channels.

[0158] For example, such as Figure 10AAs shown, taking the second set of output data as an example, the N pooling operation results of the N output sampling groups in each output channel of the second set of output data can be stored in the first storage group, the second storage group, ... the Nth storage group in turn; taking the second storage group as an example, the M pooling operation results corresponding to the M output channels in the second storage group can be stored in the storage subgroup 2-1, the storage subgroup 2-2, ... the storage subgroup 2-M in turn.

[0159] Therefore, a storage group can correspond to an output sampling group, and a storage subgroup can correspond to an output channel; such as Figure 10A As shown, the example's first register stores the output data in a storage structure with M output channels × N output sampling groups, which is also a [C,N] storage structure.

[0160] Figure 10B This is a schematic diagram illustrating an example of how multiple pooling operation results are stored in a first register, as provided in at least one embodiment of this disclosure.

[0161] For example, such as Figure 10B As shown, the X groups of output data are stored in the first register in BF16 format. Taking only the first two subgroups of the eight storage groups in the first register as an example, the eight storage groups are N0, N1...N7, and the first two subgroups of each storage group are C0 and C1. For example, the eight pooling operation results of the eight output sampling groups in each output channel of the j-th group of output data can be stored sequentially in the first storage group, the second storage group...the eighth storage group, where 1≤j≤X and j is an integer; taking the first storage group N0 as an example, the two pooling operation results corresponding to the two output channels in the first storage group N0 can be stored sequentially in two consecutive storage subgroups C0 and C1.

[0162] It should be noted that, Figure 10B The storage method of multiple pooling operation results in the first register shown is only an example. Other storage methods can be selected according to actual needs. The embodiments of this disclosure do not limit this.

[0163] Figure 11 for Figure 2 An exemplary flowchart of step S150.

[0164] For example, in order to store the output data of group X in memory for subsequent operations (such as computations in the layer following the global pooling layer), Figure 2 Step S150 may include the following steps S151 to S153.

[0165] Step S151: Obtain X sets of output data;

[0166] Step S152: Calculate the Qth address offset in memory of the Qth pooling operation result among multiple pooling operation results using the correspondence between the storage locations of the first register and memory, where 1≤Q≤P and Q is an integer;

[0167] Step S153: Based on the Q-th address offset, use the first storage instruction to store the Q-th pooling operation result into the memory address corresponding to the Q-th address offset.

[0168] In step S151, the output data of group X is passed Figure 2 The X sets of output data obtained in steps S110 to S140 are detailed in the following section. Figures 2 to 10B The description in the text will not be repeated here.

[0169] In step S152, the X sets of output data include multiple pooling operation results stored in X memory groups. These multiple pooling operation results include the first pooling operation result to the Pth pooling operation result, where P is a positive integer. For example, the storage location of the P pooling operation results in the first register is as follows: Figure 10A As shown, see details. Figure 10A The description in the text will not be repeated here. For example, by using the correspondence between the storage locations of the first register and memory, the Q-th address offset in memory that should be stored in the first register for the Q-th pooling operation result can be calculated, where 1≤Q≤P and Q is an integer.

[0170] In step S153, the Q-th address offset can be used as a mapping coordinate, and the Q-th pooling operation result can be stored in the memory address corresponding to the Q-th address offset using the first storage instruction.

[0171] Figure 12 This is a schematic diagram illustrating an example of storing multiple pooling operation results in a first register into memory, provided for at least one embodiment of this disclosure. Figure 12 The first register in is, for example, Figure 10B The first register in the system.

[0172] For example, in some examples, the correspondence between the storage locations of the X group of output data in registers and memory is as follows: Figure 12 As shown; using as Figure 12 The correspondence shown can be used to calculate the memory offsets from address 1 to address 16 for storing the 16 pooling operation results N0_C0, N0_C1, N1_C0, N1_C1...N7_C0, N7_C1. Using these 16 address offsets as mapping coordinates, the first storage instruction is used to store the 16 pooling operation results into their corresponding memory addresses.

[0173] For example, the first store instruction can be an ST instruction, which can directly convert FP32 format to BF16 format. Alternatively, the first store instruction can be any other instruction capable of performing format conversion, depending on actual needs; the embodiments disclosed herein do not impose any limitations on this.

[0174] For example, in subsequent operations, by adding a transpose option to the instructions used for subsequent operations, the results of multiple pooling operations in memory can be directly used for subsequent operations (e.g., calculations in the next layer of the global pooling layer), thus eliminating the overhead of other additional processing and ensuring the validity of the [C,N] storage structure in the first register.

[0175] In the data processing method provided in this embodiment, the input data in the [N,C] (where N is the number of sampling groups and C is the number of channels) format is processed to produce output data in the [C,N] (where N can be the number of output data groups and C can be the number of channels) format, that is, the two dimensions N and C are interchanged. For example, with Figure 10B Taking the example shown, the first register of the example stores the output data in a storage structure of 2C×32N (BF16 format), which is the [C,N] storage structure. That is, each register will store 2C×32N of data in memory each time.

[0176] For example, since the output data adopts the [C,N] storage structure (2C×32N), and 16 parallel computing units are needed to perform subsequent calculations, each computing unit only calculates 16×2=32 channels of output data at a time, breaking the limitation of 16×32=512 channels that need to be calculated when using the [N,C] storage structure (32C×2N), ensuring the consistency of data processing granularity between the pooling layer (e.g., global pooling layer) and the previous layer (e.g., nonlinear activation function layer).

[0177] Furthermore, the data processing method provided in this embodiment processes data in the following order: first, fix the direction of channel C, and then process all H×W (see details). Figure 2 In the middle steps S110 to S140 and Figures 2 to 10B (as described in the text) ensures that the data processing order during pooling operations is consistent with the data processing order in the previous layer (e.g., the non-linear activation function layer).

[0178] Therefore, based on the data processing method provided in the embodiments of this disclosure, the pooling layer (e.g., a global pooling layer) and the previous layer (e.g., a nonlinear activation function layer) can directly transfer data in the first register. The data processing method provided in at least one embodiment of this disclosure changes the way pooling operation output data is stored in the register, ensuring the consistency between the data processing granularity of the output data in memory and the data processing granularity in the previous layer network, and ensuring the consistency between the data processing order during the pooling operation and the data processing order in the previous layer network. This achieves efficient transfer of data used for pooling operations in the register, improving computational performance.

[0179] Figure 13 This is a schematic block diagram of a data processing apparatus provided for at least one embodiment of the present disclosure.

[0180] For example, such as Figure 13 As shown, the data processing device 200 is used to store the pooling operation result of the input data in a first register according to the storage granularity of the first register. For example, the first register includes multiple storage groups, and each storage group includes M storage subgroups, where M is a positive integer. For example, the data processing device 200 includes an acquisition module 210, a processing module 220, a calculation module 230, and a storage module 240.

[0181] For example, the acquisition module 210 is configured to acquire input data. For example, the input data includes N sample groups, each of the N sample groups includes multiple channels, and each channel of the input data corresponds to multiple elements, where N is a positive integer. That is, the acquisition module 210 can be configured to perform, for example... Figure 2 The step S110 is shown.

[0182] For example, processing module 220 is configured to perform a first processing on the input data to obtain X sets of preprocessed data. For example, each set of preprocessed data in the X sets includes multiple preprocessed sampling groups, each preprocessed sampling group includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements. The number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, where X is a positive integer. That is, processing module 220 can be configured to perform, for example... Figure 2 The step S120 shown.

[0183] For example, the calculation module 230 is configured to perform pooling operations on multiple elements in the M channels of each preprocessed sampling group, obtaining at least one pooling operation result for each channel in the M channels of each preprocessed sampling group, as output data. For example, X groups of preprocessed data correspond to X groups of output data, each group of output data in the X groups includes M output channels, each of the M output channels includes N output sampling groups, and each of the N output sampling groups includes a pooling operation result. That is, the calculation module 230 can be configured to perform, for example... Figure 2 The step S130 shown.

[0184] For example, the calculation module 230 is further configured to perform pooling operations on multiple elements in the M channels of each preprocessed sampling group in at least one second register to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group.

[0185] For example, storage module 240 is configured to sequentially store the N output sample groups from each output channel into N storage groups. For instance, the M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are sequentially stored into M consecutive storage subgroups of the Kth storage group, where 1 ≤ K ≤ N and K is an integer. That is, storage module 240 can be configured to perform, for example... Figure 2 The step S140 shown.

[0186] For example, storage module 240 is also configured to store X sets of output data in memory. That is, storage module 240 can be configured to perform, for example... Figure 2 The step S150 shown.

[0187] Due to the above description, for example Figure 2 The data processing method shown has already described in detail the operation of the data processing device 200, so for the sake of brevity, it will not be repeated here. For relevant details, please refer to Figure 1 above. Figure 12 The description.

[0188] It should be noted that, Figure 13 The various modules described above in the data processing apparatus 200 shown can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, these modules may correspond to dedicated integrated circuits, pure software code, or modules combining software and hardware. As an example, see [reference needed]. Figure 13 The device described may be a PC computer, tablet device, personal digital assistant, smartphone, web application or other device capable of executing program instructions, but is not limited thereto.

[0189] Furthermore, although the data processing apparatus 200 has been divided into modules for performing corresponding processes in the description above, it will be clear to those skilled in the art that the processes performed by each module can also be performed without any specific module division in the apparatus or without clear boundaries between the modules. In addition, the above references... Figure 13 The data processing apparatus 200 described is not limited to the modules described above, but may also include other modules (e.g., writing module, control module, etc.) as needed, or the above modules may be combined.

[0190] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory; the memory includes one or more computer program modules; the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include methods for implementing the data processing methods provided by the embodiments of this disclosure described above.

[0191] Figure 14 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0192] For example, such as Figure 14 As shown, the electronic device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to execute the non-transitory computer-readable instructions, which, when executed by the processor 310, can perform one or more steps of the data processing method described above. The memory 320 and the processor 310 can be interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0193] For example, processor 310 may be a central processing unit (CPU), a digital signal processor (DSP), or other processing unit with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) may be an x86 or ARM architecture. Processor 310 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in the adaptive voltage and frequency regulation device 300 to perform desired functions.

[0194] For example, memory 320 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 310 may run one or more computer program modules to implement various functions of device 300. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0195] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 300 can be referred to the description of the data processing method provided in at least one embodiment of this disclosure above, and will not be repeated here.

[0196] Figure 15 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0197] For example, such as Figure 15 As shown, the electronic device 400 is, for example, suitable for implementing the data processing method provided in the embodiments of this disclosure. It should be noted that... Figure 15 The illustrated electronic device 400 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0198] For example, such as Figure 15As shown, electronic device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 41, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 42 or a program loaded from storage device 48 into random access memory (RAM) 43. RAM 43 also stores various programs and data required for caching the operation of device 400 in a system simulation. Processing unit 41, ROM 42, and RAM 43 are interconnected via bus 44. Input / output (I / O) interface 45 is also connected to bus 44. Typically, the following devices can be connected to I / O interface 45: input devices 46 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 47 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 48 including, for example, magnetic tape, hard disks, etc.; and communication devices 49. Communication device 49 allows electronic device 400 to communicate wirelessly or wiredly with other electronic devices to exchange data.

[0199] Although Figure 15 An electronic device 400 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 400 may alternatively implement or have more or fewer devices.

[0200] For detailed descriptions and technical effects of the electronic device 400, please refer to the relevant description of the data processing method above, which will not be repeated here.

[0201] Figure 16 This is a schematic diagram of a storage medium provided for at least one embodiment of the present disclosure.

[0202] For example, such as Figure 16 As shown, storage medium 500 stores non-transitory computer-readable instructions 510. For example, when the non-transitory computer-readable instructions 510 are executed by a computer, one or more steps in the data processing method described above are performed.

[0203] For example, this storage medium 500 can be applied to Figure 14 In the illustrated electronic device 300, for example, the storage medium 500 can be the memory 320 within the electronic device 300. For example, a description of the storage medium 500 can be found here. Figure 14 The corresponding description of the memory 320 in the illustrated electronic device 300 will not be repeated here.

[0204] The following points need to be clarified regarding this disclosure:

[0205] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0206] (2) Where there is no conflict, features of the same embodiment and different embodiments of this disclosure can be combined with each other.

[0207] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A data processing method for storing the pooling operation result of input data in a first register according to the storage granularity of a first register, wherein, The first register includes multiple memory groups, and each memory group includes M memory subgroups, where M is a positive integer. The method includes: The input data is obtained, wherein the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, where N is a positive integer; The input data is subjected to a first processing to obtain X sets of preprocessed data, wherein each set of preprocessed data in the X sets of preprocessed data includes multiple preprocessed sampling groups, each preprocessed sampling group includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements. The number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, and X is a positive integer. Pooling operations are performed on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group, which is used as output data. Here, the X groups of preprocessed data correspond to X groups of output data, each group of output data in the X groups of output data includes M output channels, and each output channel in the M output channels includes N output sampling groups. The N output sampling groups in each output channel are sequentially stored in the N storage groups of the plurality of storage groups. The M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are sequentially stored in the M consecutive storage subgroups of the Kth storage group, where 1≤K≤N and K is an integer.

2. The data processing method according to claim 1, wherein, The number of output channels in the X groups of output data is the same as the number of channels in each sampling group of the input data, and the number of output sampling groups in each output channel of each group of output data is the same as the number of sampling groups in the input data.

3. The data processing method according to claim 1, wherein, The step of performing pooling operations on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group includes: Pooling operations are performed on multiple elements in the M channels of each preprocessed sampling group in at least one second register to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group.

4. The data processing method according to claim 3, wherein, The size of the data corresponding to each channel of the input data includes a first size. The first processing of the input data to obtain X sets of preprocessed data includes: With the number of channels in each sampling group fixed, the input data is spliced ​​along the direction of the first dimension according to the correspondence of the positions of multiple channels in multiple sampling groups to obtain the first updated data; The first updated data is divided into the X groups of preprocessed data. The first update data includes Y first update sampling groups, where Y is a positive integer. Each first update sampling group includes multiple channels, and the number of channels in each first update sampling group is the same as the number of channels in each sampling group.

5. The data processing method according to claim 4, wherein, The size of the data corresponding to each channel in the first updated data includes a first update size. The input data includes N sampling groups, and the first update size is N / Y times the first size. The storage granularity of the second register includes a third size, and the first update size is an integer multiple of the third size.

6. The data processing method according to claim 5, wherein, The first dimension is the height of the data corresponding to each channel in the input data, the first update dimension is the height of the data corresponding to each channel in the first update data, and the third dimension is the height of the data corresponding to each register channel in the second register. The height refers to the number of elements along the column direction in an array formed by multiple elements arranged in a two-dimensional configuration; or... The first dimension is the width of the data corresponding to each channel in the input data, the first update dimension is the width of the data corresponding to each channel in the first update data, and the third dimension is the width of the data corresponding to each register channel in the second register. The width refers to the number of elements along the row direction in an array formed by two-dimensional arrangement of multiple elements.

7. The data processing method according to claim 4, wherein, Each of the at least one second register includes M register channels. The step of dividing the first updated data into the X groups of preprocessed data includes: Select the data corresponding to each M channels in the Y first update sampling groups as a group, and divide the first update data into X groups of second update data. Each group of second update data in the X groups of second update data includes Y second update sampling groups, each second update sampling group includes M channels, and the size of the data corresponding to each channel in the second update data includes the second update size. The number of channels in each of the Y second update sampling groups is kept constant. According to the correspondence of the positions of multiple channels in the multiple second update sampling groups, each group of second update data in the X groups is split along the direction of the second update size to obtain the X groups of preprocessed data.

8. The data processing method according to claim 7, wherein, The size of the data corresponding to each channel in the first updated data includes a first update size, and the second update size is the same as the first update size. The first update size is the height of the data corresponding to each channel in the first update data, and the second update size is the height of the data corresponding to each channel in the second update data. The height refers to the number of elements along the column direction in an array formed by two-dimensional arrangement of multiple elements. or, The first update size is the width of the data corresponding to each channel in the first update data, and the second update size is the width of the data corresponding to each channel in the second update data. The width refers to the number of elements along the row direction in an array formed by two-dimensional arrangement of multiple elements.

9. The data processing method according to claim 3, wherein, Each of the at least one second register includes M register channels. The step of performing pooling operations on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group includes: According to the one-to-one correspondence between the M channels of each preprocessing sampling group and the M register channels of each second register, and according to the correspondence between the size of the data corresponding to the M channels of each preprocessing sampling group and the storage granularity of the second register, the data corresponding to the M channels of each preprocessing sampling group is stored in at least one second register. A pooling operation is performed on multiple elements in each register channel stored in the at least one second register to obtain at least one pooling operation result for each register channel. Wherein, at least one pooling operation result for each registered channel is at least one pooling operation result corresponding to at least one channel of the input data.

10. The data processing method according to claim 9, wherein, The pooling operation includes summation and averaging.

11. The data processing method according to claim 9, wherein, The storage granularity of the second register includes a third dimension, and the size of the data corresponding to the M channels in each preprocessed sampling group includes a fourth dimension. The correspondence between the size of the data corresponding to the M channels in each preprocessing sampling group and the storage granularity of the second register includes the correspondence between the fourth size and the third size, wherein the fourth size is not greater than the sum of the third sizes of the at least one second register.

12. The data processing method according to claim 11, wherein, The third dimension is the height of the data corresponding to each register channel in the second register, and the fourth dimension is the height of the data corresponding to the M channels in each preprocessing sampling group. The height refers to the number of elements along the column direction in an array formed by the two-dimensional arrangement of multiple elements; or... The third dimension is the width of the data corresponding to each register channel in the second register, and the fourth dimension is the width of the data corresponding to the M channels in each preprocessing sampling group. The width refers to the number of elements along the row direction in an array formed by two-dimensional arrangement of multiple elements.

13. The data processing method according to claim 1, wherein, Each of the N output sampling groups in each output channel includes a pooling operation result. The step of storing the N output sampling groups in each output channel sequentially in the N storage groups includes: The N pooling operation results of the N output sampling groups in each output channel are sequentially stored in the N storage groups of the plurality of storage groups.

14. The data processing method according to claim 1, further comprising: The X group of output data is stored in memory.

15. The data processing method according to claim 14, wherein, The step of storing the X group of output data into memory includes: Obtain the X sets of output data, wherein the X sets of output data include multiple pooling operation results stored in X storage groups, and the multiple pooling operation results include the first pooling operation result to the Pth pooling operation result, where P is a positive integer; The Qth address offset of the Qth pooling operation result in memory is calculated using the correspondence between the storage locations of the first register and the memory, where 1 ≤ Q ≤ P and Q is an integer; Based on the Q-th address offset, the Q-th pooling operation result is stored in the memory address corresponding to the Q-th address offset using the first storage instruction.

16. A data processing apparatus for storing the pooling operation result of input data in a first register according to the storage granularity of a first register, wherein, The first register includes multiple memory groups, and each memory group includes M memory subgroups, where M is a positive integer. The device includes: The acquisition module is configured to acquire the input data, wherein the input data includes N sampling groups, each of the N sampling groups includes multiple channels, and the data corresponding to each channel of the input data includes multiple elements, where N is a positive integer; The processing module is configured to perform a first processing on the input data to obtain X sets of preprocessed data, wherein each set of preprocessed data in the X sets of preprocessed data includes multiple preprocessed sampling groups, each preprocessed sampling group includes M channels, and the data corresponding to each channel of each preprocessed sampling group includes multiple elements. The number of channels in each sampling group of the input data is different from the number of channels in each preprocessed sampling group, and X is a positive integer. The calculation module is configured to perform pooling operations on multiple elements in the M channels of each preprocessed sampling group to obtain at least one pooling operation result for each channel in the M channels of each preprocessed sampling group as output data. The X groups of preprocessed data correspond to X groups of output data. Each group of output data in the X groups of output data includes M output channels. Each output channel in the M output channels includes N output sampling groups. Each output sampling group in the N output sampling groups includes a pooling operation result. The storage module is configured to sequentially store the N output sampling groups in each output channel into the N storage groups, wherein the M pooling operation results corresponding to the M output channels in the Kth storage group of the N storage groups are sequentially stored into the M consecutive storage subgroups of the Kth storage group, where 1≤K≤N and K is an integer.

17. The data processing apparatus according to claim 16, wherein, The computing module is also configured to: Pooling operations are performed on multiple elements in the M channels of each preprocessed sampling group in at least one second register to obtain at least one pooling operation result for each of the M channels of each preprocessed sampling group.

18. The data processing apparatus according to claim 16, wherein, The storage module is also configured to store the X group of output data in memory.

19. An electronic device comprising: processor; Memory, including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the data processing method according to any one of claims 1-15.

20. A storage medium storing non-transitory computer-readable instructions that, when executed by a computer, implement the data processing method according to any one of claims 1-15.

Images