Data processing methods and related products

Abstract

The embodiment of the present application provides a data processing method and related products, the data processing method comprising: an electronic device obtains convolution data of a first convolution layer in a first calculation stage; determines whether the data volume of the convolution data in a first direction can be divided by a first data volume, the first data volume is the granularity of data output by the basic calculation unit required by the Winograd method when the calculation efficiency is the highest; if not, divides the convolution data into N parts in the first direction, the data volume of the first part of the N parts is less than the first data volume, the data volume of the other parts of the N parts except the first part is equal to the first data volume, and N is an integer greater than or equal to 2; performs convolution operations on the first to Nth parts of the N parts in turn. The embodiment of the present application can balance the calculation efficiency and cache space.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and specifically to a data processing method and related products. Background Art

[0002] Classifier is a very important branch in the field of machine learning. Deep learning has become the mainstream algorithm for classifiers. Current deep learning algorithms generally adopt a convolution (conv) + shortcut network structure.

[0003] In order to improve the computational efficiency of deep learning algorithms, the hardware tends to use the Winograd method when calculating conv. When the Winograd method is used to calculate shortcuts, the cache space used by shortcuts will increase. In deep learning algorithms, how to balance computational efficiency and cache space has become an urgent problem to be solved. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and related products, which can minimize the used cache space while maximizing the computing efficiency, thereby balancing the computing efficiency and the cache space.

[0005] A first aspect of an embodiment of the present application provides a data processing method, including:

[0006] Obtaining convolution data of the first convolutional layer in the first calculation stage;

[0007] Determine whether the data volume of the convolution data in the first direction is divisible by a first data volume, where the first data volume is a granularity of data output by a basic computing unit required by the Winograd method when the computing efficiency is the highest;

[0008] If not, the convolution data is divided into N parts in the first direction, the data amount of the first part of the N parts is less than the first data amount, the data amounts of the other parts of the N parts except the first part are equal to the first data amount, and N is an integer greater than or equal to 2;

[0009] Convolution operations are performed on the first to Nth parts of the N parts in sequence.

[0010] A second aspect of an embodiment of the present application provides a data processing device, including:

[0011] An acquisition unit, used to acquire convolution data of the first convolution layer in the first calculation stage;

[0012] a judging unit, configured to judge whether the data volume of the convolution data in the first direction is divisible by a first data volume, wherein the first data volume is a granularity of data output by a basic computing unit required by the Winograd method when the computing efficiency is the highest;

[0013] a splitting unit, configured to, if the judgment result of the judging unit is no, split the convolution data in the first direction into N parts, wherein the data amount of a first part of the N parts is less than the first data amount, and the data amounts of other parts of the N parts except the first part are equal to the first data amount, and N is an integer greater than or equal to 2;

[0014] The operation unit is used to perform convolution operations on the first to Nth parts of the N parts in sequence.

[0015] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the step instructions in the first aspect of the embodiment of the present application.

[0016] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0017] The fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product can be a software installation package.

[0018] In an embodiment of the present application, convolution data of a first convolutional layer in a first calculation stage is obtained; it is determined whether the data volume of the convolution data in a first direction is divisible by a first data volume, where the first data volume is the granularity of data output by a basic calculation unit required by the Winograd method when the calculation efficiency is the highest; if not, the convolution data is divided into N parts in the first direction, the data volume of the first part of the N parts is less than the first data volume, the data volume of the other parts of the N parts except the first part is equal to the first data volume, and N is an integer greater than or equal to 2; and convolution operations are performed on the first to Nth parts of the N parts in sequence. In the data processing method of the embodiment of the present application, if the data volume of the convolution data in the first direction cannot be divided by the first data volume, then during the first convolution operation (convolution operation is performed on the first portion) of the first convolution layer, it is not aligned with the first data volume, and all convolution operations after the first convolution operation in the first convolution layer are aligned with the first data volume, only the computational efficiency of the first convolution operation is lost, but the cache space used can be greatly reduced in the entire operation process, the cache space used by subsequent shortcuts can be reduced, and the cache space used can be minimized while maximizing the computational efficiency, thereby balancing the computational efficiency and the cache space. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of a network structure of a unet provided in an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of a convolution calculation process provided in an embodiment of the present application;

[0022] Figure 3a It is a schematic diagram of the number of layers required to pass through a network and the corresponding calculation method of each layer without using Winograd, provided in an embodiment of the present application;

[0023] Figure 3b It is a schematic diagram of another method for calculating the number of layers required to pass through a network and the corresponding calculation method for each layer without using Winograd provided in an embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of the number of layers required to pass through a network and the corresponding calculation method of each layer when Winograd is used, provided in an embodiment of the present application;

[0025] Figure 5 It is a flowchart of a Winograd accelerated convolution algorithm provided in an embodiment of the present application.

[0026] Figure 6 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0027] Figure 7 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0028] Figure 8 It is a flowchart of another data processing method provided in an embodiment of the present application;

[0029] Fig. 9 It is a schematic diagram of the calculation results of each layer of a network at each step using Winograd provided in an embodiment of the present application;

[0030] Fig.10a It is a schematic diagram of another method for calculating the number of layers required to pass through a network and the corresponding calculation method for each layer without using Winograd provided in an embodiment of the present application;

[0031] Fig.10b This is another schematic diagram of the number of layers required to pass through a network and the corresponding calculation method for each layer when Winograd is used, provided in an embodiment of the present application;

[0032] Fig.10c It is another schematic diagram of the calculation results of each layer of a network at each step using Winograd provided in an embodiment of the present application;

[0033] Fig.11a It is a schematic diagram of another method for calculating the number of layers required to pass through a network and the corresponding calculation method for each layer without using Winograd provided in an embodiment of the present application;

[0034] Fig.11b This is another schematic diagram of the number of layers required to pass through a network and the corresponding calculation method for each layer when Winograd is used, provided in an embodiment of the present application;

[0035] Fig.11c It is another schematic diagram of the calculation results of each layer of a network at each step using Winograd provided in an embodiment of the present application;

[0036] Fig.12 It is a segmented schematic diagram of input data of a network provided in an embodiment of the present application;

[0037] Fig.13 is a schematic diagram of a cache space used in a computing process provided by an embodiment of the present application;

[0038] Fig.14 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0039] Fig.15 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0042] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0043] The electronic devices involved in the embodiments of the present application may include various devices or computers with data processing capabilities, such as desktop computers, servers, laptop computers, tablet computers, etc.

[0044] Current deep learning algorithms generally use a convolution (conv) + shortcut network structure. For example, Figure 1 See the (unet) network shown. Figure 1 , Figure 1 Schematic diagram of a network structure of a unet provided in an embodiment of the present application. Figure 1As shown, unet includes convolution operations (conv 3×3, conv 1×1), direct connection operations (including copy and crop), pooling operations (for example, max pool 2×2), upsampling (up-conv 2×2), etc.

[0045] The current method generally uses a 3*3 convolution kernel (conv kernel) to extract local features. The convolution calculation process is as follows: Figure 2 , usually requires input data in each 3*3 window to output 1 data. Generally speaking, when using a 3*3 convolution kernel for convolution operation, input n rows of data, corresponding to output n-2 rows of data, and each two adjacent 3*3 windows will have 2 rows or 2 columns of overlap. The shortcut originated from the residual network (resnet), which can avoid the vanishing derivative problem. Figure 1 The arrows corresponding to copy and crop in the figure represent shortcuts, which are to copy the original data. Figure 1 1 place ( Figure 1 The handwritten 1) is divided into two arrows of different grayscales, which are divided into original data and learning data. Figure 1 2 places ( Figure 1 2) Merge the original data and the learning data.

[0046] At present, artificial intelligence (AI) inference generally uses hardware such as central processing unit (CPU), graphics processing unit (GPU), neural-network processing unit (NPU), etc. On the one hand, the cache space of these hardware is limited and needs to be saved. On the other hand, these hardware tend to use memory blocks of specified size in memory (for example, static random access memory) when calculating, which can improve computing efficiency.

[0047] In order to improve computational efficiency, when calculating convolution, the hardware tends to use the Winograd method, with 4*4 input and 2*2 output, but this will increase the cache space used by the shortcut. For example, Figure 3a ( Figure 3ais the number of layers required to pass through a network and the corresponding calculation method of each layer), assume that the layer with id=1 (layer1) and the layer with id=18 (layer8) need to be merged (concat).

[0048] If winograd is not used, the maximum offset between layer1 and layer18 is 47-5+1=43. If layer1 is to be permanently in the cache, since layer1 generates 47 rows and layer18 only generates 5 rows, layer19 can only merge 5 rows. In addition, the (47-5) row of layer1 needs to wait for layer18 to generate the corresponding row before it can be released when layer19 uses the (47-5) row of layer1 for the merge (concat) operation. Figure 3a To be able to complete all layers and output at least 1 line per layer (1 line is the minimum granularity for calculation), layer1 still needs to generate 1 more line so that layer18 can parameterize the number of subsequent lines. Therefore, layer1 needs to occupy a total of 47-5+1=43 lines of cache space.

[0049] The offset refers to the cache space occupied by the original data (data in layer1) in the concat operation during the shortcut operation. The smaller the offset, the more cache space is saved.

[0050] Concat is to merge two elements in the channel direction. For example, a is w*h*c0 and b is w*h*c1. After concat, it is w*h*(c0+c1).

[0051] Figure 3a In the figure, Kernel stride: kernel stride; Kernel h: kernel height; Kernel w: kernel width; Pad: Padding, refers to the amount of extra data added to the left, right, top, and bottom of the image; First h: the number of rows in the first stage; Remain: the layer does not consume all the inputs in this calculation. For example, for cnv3*3, stride = 2, without padding, 4 rows are input and only 1 row can be output. 3 rows of input are used. The remaining rows that can be used in the next calculation are 2. Quantize: quantization operator, used for format conversion.

[0052] If Winograd is used, convolution can only be performed after 4 lines are input. When 3 lines are input, it is still necessary to wait. Among these 3 lines, 2 lines are from the overlap with the previous convolution, and 1 line is from the new input. It is temporarily reserved and will be calculated when another line is input. At this time, the maximum offset between layer 1 and layer 18 is 87-10+1=78. Figure 4 As shown, using winograd increases the cache space by (78-43) / 43=81% compared to not using winograd.

[0053] On the one hand, in order to improve the computing efficiency of the hardware, it is necessary to use Winograd to accumulate 4*4 input data before calculating again, or even higher, such as accumulating 16*16 before calculating again; on the other hand, in order to reduce the interaction between hardware and memory (for example, double-data-rate synchronous dynamic random access memory DDR), it is necessary to cache the data in the hardware cache as much as possible. For example, the cache can be static random access memory (SRAM), but the direct connection (shortcut) leads to a contradiction between the two.

[0054] When shortcutting, the number of rows that need to be retained is: the number of rows in the gap between the two layers of concat + the granularity. On the one hand, the larger the granularity, the higher the computing efficiency. On the other hand, a large granularity will increase the number of rows stored in the cache, requiring more cache space.

[0055] The Winograd method can also be called the Winograd algorithm. The Winograd method is an acceleration algorithm for convolution operations. Since the time complexity of multiplication and addition in hardware implementation is different, the time required for multiplication operations is usually much longer than the time required for addition. Therefore, the core idea of ​​the Winograd method is to use fewer operations to achieve the purpose of accelerating convolution operations. For ease of understanding, the Winograd method is explained below by way of example.

[0056] Input = [z0 z1 z2 z3] T (1)

[0057] Output output = [y0 y1] T (2)

[0058] Convolution kernel Filter = [x0 x1 x2] T (3)

[0059] Convolution operation is performed on a 2×3 matrix and a 3×1 matrix using the conventional convolution algorithm as follows:

[0060]

[0061] The accelerated convolution algorithm based on Winograd is as follows:

[0062]

[0063] Among them, m1=(z0-z2)x0, m4=(z1-z3)x2,

[0064] That is to say, for a 2×3 matrix and a 3×1 matrix, the traditional convolution operation requires 6 multiplications and 4 additions, while the Winograd method only requires 4 multiplications and 4 additions, thereby improving the computational efficiency. The above formula (5) is extended to the general matrix as follows:

[0065] O=A T [(GI)⊙(B T F)] (6)

[0066] The symbol ⊙ represents element-wise matrix multiplication (EWMM), I is the input data, F is the convolution kernel data, O is the output data, and A, B, and G are constant matrices. Taking the above two-dimensional convolution F(2,3) as an example, formula (6) can be expressed as:

[0067] O=A T [(GIG T )⊙(B T FB)]A (7)

[0068] Based on this, the process of Winograd accelerated convolution algorithm is as follows Figure 5 As shown in the figure, it is mainly divided into four steps: input transformation, filter transformation, matrix multiplication and output transformation. Correspondingly, when using GPU for Winograd-based accelerated convolution operations, there are also four tasks: input transformation task, filter transformation task, matrix multiplication task and output transformation task, among which,

[0069] Input transformation: The input window is transformed into a window of size (m+r-1)×(m+r-1), where there are (r-1) rows of overlapping elements between adjacent windows.

[0070] Filter Transformation or Kernel Transformation: The filter is also transformed to the same size as the transformed input window, i.e. (m+r-1)(m+r-1).

[0071] EWMM: Perform element-wise multiplication and addition of the transformed input windows in the Winograd method, where the elements in the matrix are real numbers in the Winograd method.

[0072] Inverse transform or output transform: The result after element-wise multiplication is inversely transformed to produce convolution result windows of size m×m, where there are no overlapping windows in the output tensor.

[0073] The embodiments of the present application provide a data processing method and related products, which can minimize the used cache space while maximizing the computing efficiency, thereby balancing the computing efficiency and the cache space.

[0074] The basic idea of ​​the embodiments of the present application is:

[0075] 1. When each layer is calculated for the first time, if the data of the layer in a certain direction (for example, the height direction or the width direction) is not an integer multiple of the output unit with the highest computational efficiency, it will not be aligned to the unit with the highest computational efficiency (4*4 or 16*16), so as to reduce the cache used by the shortcut.

[0076] 2. In subsequent calculations (other calculations after the first calculation, except the final boundary), all are aligned to the unit with the highest computational efficiency. This way, only one calculation that does not meet the highest efficiency is wasted, which can reduce the cache used by the shortcut.

[0077] 3. The actual size of the first calculation should be calculated based on the shortcut.

[0078] 4. Perform 1 to 3 above in both width and height directions to optimize the components of the cache used by the shortcut.

[0079] See also Figure 6 , Figure 6 is a flow chart of a data processing method provided in an embodiment of the present application. Figure 6 As shown, the method may include the following steps.

[0080] 601. The electronic device obtains convolution data of a first convolution layer in a first calculation stage.

[0081] The data processing method of the embodiment of the present application can be applied to a first network structure, which may include at least one convolutional layer. The first convolutional layer may be any one of the at least one convolutional layer. The first network structure may include at least one convolutional layer and at least one merging layer. The merging layer, which may also be called a concat layer, is used to merge the data of two layers, and a shortcut is a specific form of a merging layer.

[0082] Since the raw data input into the first network structure is often relatively large, if it is input at one time, it will occupy a large storage space. Generally, the raw data is divided into multiple calculation stages and input into the first network structure respectively, and the first calculation stage, the second calculation stage, ... are performed in sequence. The first calculation stage is the first calculation stage of the multiple calculation stages.

[0083] 602. The electronic device determines whether the data amount of the convolution data in the first direction is divisible by the first data amount.

[0084] The first data volume is the granularity of data output by the basic computing unit required by the Winograd method when the computing efficiency is the highest.

[0085] In the embodiment of the present application, the convolution data may be two-dimensional data or three-dimensional data. When the convolution data is two-dimensional data, the first direction may be a width direction or a height direction; when the convolution data is three-dimensional data, the first direction may be a width direction or a height direction or a depth direction.

[0086] The basic computing unit may be an integer multiple of 4*4. For example, the basic computing unit may be 4*4, 16*16, etc. For example, when the basic computing unit is 4*4, the data output by the basic computing unit required by the winograd method (when the convolution kernel is 3*3) when the computing efficiency is the highest is 2*2, and the granularity in the first direction is 2.

[0087] 603, if not, the electronic device divides the convolution data into N parts in the first direction.

[0088] The data amount of the first copy among the N copies is less than the first data amount, the data amount of the other copies among the N copies except the first copy is equal to the first data amount, and N is an integer greater than or equal to 2.

[0089] 604, the electronic device performs convolution operations on the first to the Nth copies of the N copies in sequence.

[0090] In the embodiment of the present application, since the data volume of the first portion of the N portions is smaller than the first data volume, when performing convolution operation on the first portion, although it is not aligned with the first data volume, the computational efficiency is not the highest, but the used cache space can be reduced. When performing convolution operation on the second portion to the Nth portion, since they are all aligned with the first data volume, the computational efficiency is higher.

[0091] Among them, in other calculation stages after the first calculation stage, the size of the convolution result of the first convolution layer after any convolution operation in the first direction is equal to the first data amount.

[0092] Fig. 9 In the example, setp1 to step10 correspond to the first calculation stage, and setp11 to step18 correspond to the second calculation stage. Fig. 9 It can be seen that N=10 in Example 1. Each convolution layer is not aligned with the first data volume (first data volume=2) only in the first convolution operation in the first stage, which has little impact on the calculation efficiency. In the second calculation stage, each convolution operation of each convolution layer is aligned with the first data volume (first data volume=2), and the size of its output result in the first direction is equal to the first data volume, and the calculation efficiency is the highest.

[0093] In the data processing method of the embodiment of the present application, if the data volume of the convolution data in the first direction cannot be divided by the first data volume, then during the first convolution operation (convolution operation is performed on the first portion) of the first convolution layer, it is not aligned with the first data volume, and all convolution operations after the first convolution operation in the first convolution layer are aligned with the first data volume, only the computational efficiency of the first convolution operation is lost, but the cache space used can be greatly reduced in the entire operation process, the cache space used by subsequent shortcuts can be reduced, and the cache space used can be minimized while maximizing the computational efficiency, thereby balancing the computational efficiency and the cache space.

[0094] Optionally, after executing step 602, the following steps may also be executed.

[0095] (11) when the data amount of the convolution data in the first direction is divisible by the first data amount, the electronic device divides the convolution data into M parts in the first direction, where M is an integer greater than or equal to 1;

[0096] (12) The electronic device performs convolution operations on the first to the Mth parts of the M parts in sequence.

[0097] In the embodiment of the present application, if the amount of convolution data in the first direction is divisible by the first data amount, the convolution data can be divided into M parts in the first direction to achieve the highest computational efficiency.

[0098] Optionally, step 604 may specifically include the following steps:

[0099] (21) when the electronic device performs a convolution operation on the first portion of the N portions, outputting the convolution result of the first portion;

[0100] (22) When the electronic device performs a convolution operation on the second of the N parts, it performs a convolution operation based on the data output by the previous layer of the first convolution layer, the overlapping data of the convolution operation on the second part and the convolution operation on the previous part of the second part, and outputs the convolution result of the second part, where the second part is any of the other parts; the size of the convolution result of the second part in the first direction is equal to the first data amount.

[0101] In the embodiments of this application, please refer to Fig. 9 Provide explanation. Fig. 9 In the example of the first convolution layer, layer5, the amount of convolution data in the first direction of layer5 in the first calculation stage is equal to 5, which cannot be divided by the first amount of data (the first amount of data = 2). 5 can be divided into 3 parts, namely 1 (the first part), 2 (the second part), and 2 (the third part). Fig. 9 It can be seen that layer5 is not aligned with the first data volume when performing convolution operation on the first copy (step 1), but is aligned with the first data volume when performing convolution operation on the second copy (step 2) and the third copy (step 4). Among them, when the convolution operation is performed on the second portion, the data output by layer4 of the previous layer of layer5 in the first direction has a data volume of 2. When layer5 performs a convolution operation on the first portion, the overlapping data when performing a convolution operation on the second portion in the time domain includes 2*2 (when a 3*3 convolution kernel slides a window in a 4*4 data, when the step size is 1, two adjacent sliding windows overlap by 2 rows or 2 columns), and the overlapping data in the first direction is equal to 2. The overlapping data 2=4 of the data output by the previous layer of the first convolution layer + the second portion is convolved with the previous portion (the first portion) of the second portion. These 4 rows of data are used as the input data for the convolution operation of layer5 on the second portion. The input data is convolved with the convolution kernel (3*3) to output 2 rows of data (equal to the first data volume). The computational efficiency of the second convolution operation of layer5 is the highest.

[0102] See also Figure 7 , Figure 7 FIG. 1 is a flow chart of another data processing method provided in an embodiment of the present application. Figure 7 As shown, the method is applied to a first network structure, the first network structure includes P pooling layers, and the method may include the following steps.

[0103] 701. The electronic device obtains input data required for a first calculation stage, and determines a second data amount according to P pooling layers and the first data amount.

[0104] It is necessary to ensure that after the input data is divided into Q parts, the amount of data output by each part after the pooling process of P pooling layers can meet the minimum input requirement of the next convolutional layer for convolution operation. Fig. 9 For example, the number of pooling layers is equal to 3. Each time the pooling is performed, the amount of data is reduced by half. It is necessary to ensure that after three poolings, layer5 can still perform convolution operations. Since the output of layer4 is 2, plus pad 1, it is equal to 3, which just meets the convolution operation requirements of the 3*3 convolution kernel. From this, it can be inferred that in step 1, the second data volume = 2*2 3 =16. It should be noted that Fig. 9 The amount of input data in the first direction (48) is divisible by the second amount of data (16).

[0105] 702. The electronic device determines whether the amount of input data in the first direction is divisible by the second amount of data.

[0106] 703. If the amount of the input data in the first direction cannot be divided by the second amount of data, the electronic device divides the input data into Q parts in the first direction.

[0107] The data volume of the first one of the Q parts is greater than the second data volume and less than twice the second data volume, the data volume of the other parts of the Q parts except the first one is equal to the second data volume, and Q is an integer greater than or equal to 2.

[0108] 704, the electronic device uses the first of the Q copies as input data for the first step of the first layer in the first calculation stage.

[0109] In the embodiment of the present application, the electronic device may sequentially operate on the first to Qth portions of the Q portions. If the first convolutional layer participates in the operation of the first portion of the Q portions, the process of operating the first portion of the Q portions may include the process of performing a convolution operation on the first portion of the first convolutional layer.

[0110] In an embodiment of the present application, the electronic device may perform operations on the first to Qth parts of the Q parts in sequence.

[0111] 705. The electronic device obtains convolution data of the first convolution layer in the first calculation stage.

[0112] 706. The electronic device determines whether the data amount of the convolution data in the first direction is divisible by the first data amount.

[0113] The first data volume is the granularity of data output by the basic computing unit required by the Winograd method when the computing efficiency is the highest.

[0114] 707, if the data amount of the convolution data in the first direction cannot be divided by the first data amount, the electronic device divides the convolution data into N parts in the first direction.

[0115] Among them, the data amount of the first copy among the N copies is less than the first data amount, the data amount of the other copies among the N copies except the first copy is equal to the first data amount, and N is an integer greater than or equal to 2.

[0116] 708. The electronic device performs convolution operations on the first to the Nth copies of the N copies in sequence.

[0117] The specific implementation of steps 705 to 708 can be found in Figure 6 Steps 601 to 604 are shown and will not be described in detail here.

[0118] Optional, Figure 7 The method may further comprise the steps of:

[0119] (31) the electronic device divides the input data into S equal parts in the first direction, where S is an integer greater than or equal to 1, when the amount of the input data in the first direction is divisible by the second amount of data;

[0120] (32) The electronic device uses the first of the S parts as input data for the first step of the first layer in the first calculation stage.

[0121] In an embodiment of the present application, if the amount of input data in the first direction is divisible by the second amount of data, the input data can be divided into S parts in the first direction to achieve the highest computing efficiency and occupy less cache space.

[0122] See also Figure 8 , Figure 8 FIG. 1 is a flow chart of another data processing method provided in an embodiment of the present application. Figure 8 As shown, the method is applied to a first network structure, wherein the first network structure includes P pooling layers and merging layers. The method may include the following steps.

[0123] 801. The electronic device calculates a maximum cache space used when performing operations on a merging layer of a first network structure.

[0124] In the embodiment of the present application, if the first network structure includes multiple merging layers, the cache space used when the multiple merging layers are operated can be calculated separately, and the largest cache space is selected from the cache space used when the multiple merging layers are operated.

[0125] 802. The electronic device obtains a third data amount according to the maximum cache space and the first data amount, and the third data amount is greater than or equal to the sum of the maximum cache space and the first data amount; the amount of input data required for the first calculation stage is equal to the third data amount.

[0126] In the embodiment of the present application, the actual size of the first calculation can be specifically calculated according to the shortcut. The core idea is to make the amount of input data required for the first calculation stage greater than or equal to the minimum amount of data (the third amount of data) required to pass through all layers of the first network structure. Fig. 9 The last shortcut is at layer 19, and the last layer only needs the first amount of data to complete the operation. Therefore, the third amount of data = the maximum cache space + the first amount of data. Fig. 9 As shown, the maximum cache space used by the shortcut is 48-7+2=43, and the third data volume only needs to be greater than or equal to 43. Fig. 9 The amount of input data required for the first calculation stage is 48.

[0127] 803. The electronic device obtains input data required for the first calculation stage, and determines a second data amount according to P pooling layers and the first data amount.

[0128] 804 , the electronic device determines whether the data amount of the input data in the first direction is divisible by the second data amount.

[0129] 805. If the amount of input data in the first direction cannot be divided by the second amount of data, the electronic device divides the input data into Q parts in the first direction.

[0130] 806. The electronic device uses the first of the Q copies as input data for the first step of the first layer in the first calculation stage.

[0131] 807. The electronic device obtains convolution data of the first convolution layer in the first calculation stage.

[0132] 808. The electronic device determines whether the data amount of the convolution data in the first direction is divisible by the first data amount.

[0133] 809. If the amount of the convolution data in the first direction cannot be divided by the first data amount, the electronic device divides the convolution data into N parts in the first direction.

[0134] 810. The electronic device performs convolution operations on the first to the Nth parts of the N parts in sequence.

[0135] The specific implementation of step 803 to step 810 can be found in Figure 7 Steps 701 to 708 are shown and will not be described in detail here.

[0136] In an embodiment of the present application, the amount of input data required for the first computing stage is determined based on the idea that the amount of input data required for the first computing stage can be greater than or equal to the minimum amount of data required to pass through all layers of the first network structure, so that each layer of each computing stage of the first network structure can participate in the calculation.

[0137] Embodiment 1

[0138] Winograd can use 4*4 as the basic calculation unit. The actual size of the first calculation needs to be calculated according to the shortcut. First, you need to search for the largest offset of the shortcut. You can get the value by running the simulation once. For example, if the largest offset of the shortcut is 47. Figure 3b As shown, the input should input 47+1 lines, layer1 outputs 48 lines, layer2 outputs 24 lines, layer3 outputs 12 lines, layer4 outputs 6 lines, layer5 outputs 5 lines, layer6 outputs 4 lines, ..., layer8 outputs 7 lines, ..., layer10 outputs 6 lines, layer11 outputs 5 lines, ..., so that the maximum offset of the shorcut (48-7+1=42) will be relatively small.

[0139] Correspondingly, such as Fig. 9 As shown, due to Fig. 9There are 3 poolings in the process. In order to make the output after 3 poolings have 2 rows, in step 1, the input input is 16 rows, layer 1 outputs 16 rows, layer 2 outputs 8, layer 3 outputs 4, layer 4 outputs 2, the output size of the first actual convolution (conv) calculation of layer 5 is 5% 2=1 (corresponding to the input size of 3 rows), the output size of the first actual convolution (conv) calculation of layer 6 is 2 (because 4% 2=0) (corresponding to the input size of 4 rows), ..., the output size of the first actual convolution (conv) calculation of layer 8 is 7% 2=1, ..., the output size of the first actual convolution (conv) calculation of layer 10 ... The output size calculated by v) is 6% 2+2=2, the output size calculated by the first actual convolution (conv) of layer11 is 5% 2=1,…, the output size calculated by the first actual convolution (conv) of layer13 is 9% 2=1,…, the output size calculated by the first actual convolution (conv) of layer15 is 8% 2+2=2, the output size calculated by the first actual convolution (conv) of layer16 is 7% 2=1,…, the output size calculated by the first actual convolution (conv) of layer18 is 13% 2=1,…, the output size calculated by the first actual convolution (conv) of layer20 is 12% 2+2=2.

[0140] The reason for taking the remainder of 2 is that when Winograd uses 4*4 as the basic calculation unit, the calculation output size is 2 when its calculation efficiency is the highest. Fig. 9 As shown in the figure, the details after expansion in two dimensions, time and layer, are shown. First, in step 1, the input input is 16 lines, layer 1 outputs 16 lines, layer 2 outputs 8 lines, layer 3 outputs 4 lines, layer 4 outputs 2 lines, layer 5 inputs 3 lines in total: 2 lines output by layer 4 + 1 line of pad, layer 5 outputs 1 line, which is also the first actual calculation of layer 5, and layer 6 inputs 2 lines in total: 1 line output by layer 5 + 1 line of pad, which is smaller than the size of the 3*3 convolution kernel and cannot be calculated (when 2 lines are input, convolution operation cannot be performed with the convolution kernel. Only when the input is greater than or equal to 3 lines, convolution operation can be performed).

[0141] In Step 2, the input input is 16 rows, layer1 outputs 16 rows, layer2 outputs 8, layer3 outputs 4, layer4 outputs 2, and layer5 inputs a total of 4 rows: 2 rows output by layer4 + 2 rows of overlap in the last time (2 rows overlapped when layer5 performed convolution operations in step 1 and step 2 respectively). Layer5 outputs 2 rows. This is the second actual calculation of layer5 (except for the first and last times, the input is 4 rows and the output is 2 rows in other times). Layer6 inputs a total of 4 rows: 1 row of the first output of layer5 + 2 rows of the second output of layer5 + 1 row of pad. Layer6 outputs 2 rows. This is also the first calculation of layer6. Layer7 is upsampled and outputs 4 rows, 2 of which continue to be calculated in step 2, and the other 2 rows are left for calculation in step 3. Layer8 inputs a total of 3 rows: 2 rows output by layer7 + 1 row of pad. Layer8 outputs 1 row. This is also the first actual calculation of layer8. Layer9 outputs 1 row. Layer10 inputs a total of 2 rows: 1 row output by layer9 + pad 1 line, calculation cannot be performed (when 2 lines are input, convolution operation cannot be performed with the convolution kernel. Convolution operation can only be performed when the input is greater than or equal to 3 lines).

[0142] In Step 3, calculate the other 2 rows output by layer7, and layer8 inputs a total of 4 rows: 2 rows output by layer7 + 2 rows of overlap in the last time (2 rows overlapped when layer8 performed convolution operations in step 2 and step 3 respectively). Layer8 outputs 2 rows. This is the second actual calculation of layer8. Layer9 outputs 2 rows. Layer10 inputs a total of 4 rows: 1 row output by layer9 for the first time + 2 rows output by layer9 for the second time + 1 row of pad. Layer10 outputs 2 rows. Layer11 inputs a total of 3 rows: 2 rows output by layer10 + 1 row of pad. Layer11 outputs 1 row. This is also the first actual calculation of layer11. Layer12 is upsampled and outputs 2 rows. Layer13 inputs a total of 3 rows: 2 rows output by layer12 + 1 row of pad. Layer13 outputs 1 row. This is also the first actual calculation of layer13. Layer12 is upsampled and outputs 2 rows. Layer13 inputs a total of 3 rows: 2 rows output by layer12 + 1 row of pad. 1 line, layer13 outputs 1 line, which is also the first actual calculation of layer13, layer14 outputs 1 line, and layer15 inputs 2 lines in total: 1 line output by layer14 + 1 line of pad, which cannot be calculated (when 2 lines are input, convolution operation cannot be performed with the convolution kernel. Convolution operation can only be performed when the input is greater than or equal to 3 lines).

[0143] In step 4, the input input is 16 lines, layer1 outputs 16 lines, layer2 outputs 8, layer3 outputs 4, layer4 outputs 2, and layer5 inputs a total of 4 lines: 2 lines output by layer4 + 2 lines of overlap from the last time (2 lines overlapped when layer5 performs convolution operations in step 2 and step 4 respectively). Layer5 outputs 2 lines. Similarly, layer6 outputs 2 lines, which is also the second calculation of layer6. Layer7 is upsampled and outputs 4 lines, 2 of which continue to be calculated in step 4, and the other 2 lines are left for calculation in step 7. Layer8 outputs 2 row, which is also the second actual calculation of layer8, layer9 outputs 2 rows, layer10 outputs 2 rows, which is also the second actual calculation of layer10, layer11 outputs 2 rows, which is also the second actual calculation of layer11, layer12 is upsampled and outputs 4 rows, 2 of which continue to be calculated in step4, and the other 2 rows are left for step5 to calculate, layer13 outputs 2 rows, which is also the second actual calculation of layer13, layer14 outputs 2 rows, and layer15 inputs a total of 4 rows: 1 row of layer14's first output + 2 rows of layer14's second output + pad 1 row, layer15 outputs 2 rows, layer16 inputs a total of 3 rows: 2 rows output by layer15 + 1 row of pad, layer16 outputs 1 row, which is also the first actual calculation of layer16, layer17 is upsampled and outputs 2 rows, layer18 inputs a total of 3 rows: 2 rows output by layer17 + 1 row of pad, layer18 outputs 1 row, which is also the first actual calculation of layer18, layer19 outputs 1 row, layer20 inputs a total of 2 rows: 1 row output by layer19 + 1 row of pad, and calculation cannot be performed (when 2 rows are input, convolution operation cannot be performed with the convolution kernel. Convolution operation can only be performed when the input is greater than or equal to 3 rows).

[0144] In step 5, the other 2 rows output by layer12 are calculated. Layer13 inputs a total of 4 rows: 2 rows output by layer12 + 2 rows of the last overlap (2 rows overlapped when layer13 performed convolution operations in step 4 and step 5 respectively), layer13 outputs 2 rows, layer14 outputs 2 rows, layer15 outputs 2 rows, which is also the second calculation of layer15, layer16 outputs 2 rows, which is also the second calculation of layer16, layer17 is upsampled and outputs 4 rows, 2 of which continue to be calculated in step 5, and the other 2 rows are left for calculation in step 6, layer18 outputs 2 rows, which is also the second calculation of layer18, layer19 outputs 2 rows, which is also the second calculation of layer19, layer20 inputs a total of 4 rows: 1 row of the first output of layer19 + 2 rows of the second output of layer19 + 1 row of pad, layer20 outputs 2 rows, which is also the first calculation of layer20.

[0145] Step 6 and later are the second or third, fourth... calculations, and convolution (conv) inputs a total of 4 lines: 2 lines output by the previous layer + 2 lines of the previous overlap, and outputs 2 lines, which can meet the computational efficiency of winograd. At this time, the shortcut size is: when 48 lines are input, the output starts, so at least 48 lines of space are required, and 13 lines are output at this time, leaving 48-13=35 lines. However, since we use winograd after 3 pools, each input input is 2*2^3=16 to complete a winograd, so the maximum offset=35+16=51 lines, which is only increased by (51-43) / 43=18% compared to not using winograd, and reduced by (78-51) / 78=34.6% compared to winograd without the method of the embodiment of the present application ("the first calculation is not aligned").

[0146] In fact, because there are multiple pools in this embodiment, the input is input 16 lines at a time, and the offset is increased. In fact, if there is no pool, the cache space will be very close to the optimal value. Figure 10a to Figure 10c This demonstrates the situation. Fig.10a It shows that winograd is not used, offset=6, Fig.10b It is explained that after using winograd, if the method of the embodiment of the present application is not used, then offset=12, Fig.10cIt shows that after using winograd and adopting the method of the embodiment of the present application ("first calculation is not aligned"), offset = 6-1+2 = 7, which is very close to the optimal offset (offset = 6). The smaller the offset, the smaller the cache space occupied, and the better the offset.

[0147] Embodiment 2

[0148] If the hardware uses a larger size as the basic computing unit (such as 16*16) to further improve the computing efficiency, the phenomenon of increased cache used by shortcuts will be more serious. For example Fig.11a If the basic calculation unit is not aligned, the cache used by the shortcut is only offset = 3, but if it is aligned according to 16*16, Fig.11b As shown, the cache used by the shortcut requires offset = 48, which is 48 / 3 = 16 times the gap. Similarly, if we do not align the first calculation according to the basic calculation unit, such as Fig.11c As shown, the cache used by the shortcut requires offset=3-1+16=18.

[0149] The shortcut in the above example uses a cache offset of 3-1+16=18, where 16 rows are input because 16 rows are input each time. In fact, we can further subdivide the 16 rows and apply the row logic to the columns. Fig.12 As shown, the entire input is divided into 9 parts. Fig.12 The a part of the image is composed of a 3*3 (3 rows and 3 columns) size. Fig.12 The b part (3 rows and 16n) is composed of one or n 3*16 (3 rows and 16 columns) (the number of squares in the figure does not represent the size), Fig.12 The c part represents the remaining part of the first 3 rows in the width direction, which is composed of a 3*(input width–3–16*n). Fig.12 The d part (16m rows and 3 columns) consists of one or m 16*3 (16 rows and 3 columns) size. Fig.12 The e part (16m rows and 16n columns) consists of one or n*m 16*16 (16 rows and 16 columns), Fig.12 The f part represents the remaining part of the middle 16*m rows in the width direction, which consists of one or m 16*(input width–3–16*n). Fig.12 The g part represents the remaining rows in the first 3 columns, which consists of (inputheight–3–16*m)*3. Fig.12The k part represents the remaining rows in the middle of the 16*n columns, which consists of one or n (input height–3–16*m)*16. Fig.12 The i part represents the remaining rows and 16 columns in the remaining columns, which consists of (input height–3–16*m)*(input width–3–16*n).

[0150] The calculation order is to first calculate 1 a, then calculate n b, then calculate 1 c, {then calculate 1 d, then calculate 1 e, then calculate 1 f}, {} a total of m cycles, then calculate 1 g, then calculate n k, and then calculate 1 i. This approach can further reduce the cache used by shortcuts.

[0151] At this time, if Fig.13 As shown, the cache used by the shortcut consists of the following parts: the dark gray part represents 3-1=2 rows of data, the width of this part of data is the entire input width, and the size is 2*input width; the light gray part represents the 16 rows being calculated, and this part has calculated k+18 columns, and the size is 16*(k+18); the first box represents the space that can be released (concat the data already used), and the size of this part of data is 16*k. It can be seen that in the w direction, the offset of 3-1+16=18 columns of the second box is still retained, and the size of the second box is 16*18. In the h direction, only the necessary offset of 3-1=2 rows is retained, which consists of the third box and the fourth box. One part (the third box) is the remaining 2*(input width-k) in dark gray, and the other part (the fourth box) is the newly generated light gray 2*k. At this time, the total cache used by the shortcut is: the second box + the third box + the fourth box = 2*Input width+16*18. Compared to the cache required by the original 18-line shortcut (18*input width), it has been reduced a lot and is very close to the cache required by the 2-line shortcut that is not aligned with the basic computing unit (2*input width).

[0152] The solutions of the first and second embodiments have the following beneficial effects: reducing the size of the cache used by the shortcut; ensuring that most calculations are aligned to the most efficient calculation unit.

[0153] Optionally, this solution can also be applied to scenarios where multiple shortcuts are nested and where L columns are required to be output at one time. In scenarios where multiple shortcuts are nested, you can trade off the offsets of each shorcut and choose a more appropriate size for the first calculation. In scenarios where L columns are required to be output at one time, you can calculate the size of the first calculation based on the size of each layer when outputting L columns.

[0154] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0155] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0156] In line with the above, see Fig.14 , Fig.14 This is a structural diagram of a data processing device provided in an embodiment of the present application. The data processing device 1400 is applied to an electronic device. The data processing device 1400 may include an acquisition unit 1401, a judgment unit 1402, a segmentation unit 1403 and a calculation unit 1404, wherein:

[0157] An acquiring unit 1401 is used to acquire convolution data of the first convolution layer in the first calculation stage;

[0158] A judging unit 1402 is used to judge whether the data amount of the convolution data in the first direction is divisible by a first data amount, where the first data amount is a granularity of data output when the basic calculation unit required by the Winograd method has the highest calculation efficiency;

[0159] a splitting unit 1403, configured to, if the judgment result of the judging unit 1402 is no, split the convolution data in the first direction into N parts, the data amount of a first part of the N parts being less than the first data amount, and the data amounts of other parts of the N parts except the first part being equal to the first data amount, and N being an integer greater than or equal to 2;

[0160] The operation unit 1404 is used to perform convolution operations on the first to Nth parts of the N parts in sequence.

[0161] Optionally, the splitting unit 1403 is further configured to, when the amount of data of the convolution data in the first direction is divisible by the first data amount, equally split the convolution data into M parts in the first direction, where M is an integer greater than or equal to 1;

[0162] The operation unit 1404 is also used to perform convolution operations on the first to the Mth parts of the M parts in sequence.

[0163] Optionally, the operation unit 1404 performs convolution operations on the first to Nth of the N parts in sequence, including: when performing a convolution operation on the first of the N parts, outputting the convolution result of the first part; when performing a convolution operation on the second of the N parts, performing a convolution operation based on the data output by the previous layer of the first convolution layer, the convolution operation on the second part, and the overlapping data of the convolution operation on the previous part of the second part, and outputting the convolution result of the second part, where the second part is any of the other parts; the size of the convolution result of the second part in the first direction is equal to the first data amount.

[0164] Optionally, in other calculation stages after the first calculation stage, a size of a convolution result of the first convolution layer after any convolution operation in the first direction is equal to the first data amount.

[0165] Optionally, the device is applied to a first network structure, and the first network structure includes P pooling layers;

[0166] The acquiring unit 1401 is further configured to acquire input data required for the first computing stage before acquiring convolution data of the first convolution layer in the first computing stage, and determine a second data amount according to the P pooling layers and the first data amount;

[0167] The determining unit 1402 is further configured to determine whether the amount of the input data in the first direction is divisible by the second amount of data;

[0168] The dividing unit 1403 is further configured to divide the input data into Q parts in the first direction if the judgment result of the judging unit 1402 is no, the data amount of the first part of the Q parts is greater than the second data amount and less than twice the second data amount, the data amounts of the other parts of the Q parts except the first part are equal to the second data amount, and Q is an integer greater than or equal to 2;

[0169] Among them, the first one of the Q parts is the input data of the first step of the first layer in the first calculation stage.

[0170] Optionally, the splitting unit 1403 is further configured to, when the amount of the input data in the first direction is divisible by the second amount of data, equally split the input data into S parts in the first direction, where S is an integer greater than or equal to 1;

[0171] Among them, the first one of the S parts is the input data of the first step of the first layer in the first calculation stage.

[0172] Optionally, the device may further include a calculation unit 1405; the first network structure further includes a merging layer;

[0173] The calculation unit 1405 is used to calculate the maximum cache space used when the merging layer of the first network structure performs operations before the acquisition unit 1401 acquires the input data required for the first calculation stage; obtain a third data amount based on the maximum cache space and the first data amount, and the third data amount is greater than or equal to the sum of the maximum cache space and the first data amount; the amount of input data required for the first calculation stage is equal to the third data amount.

[0174] Among them, the acquisition unit 1401, the judgment unit 1402, the segmentation unit 1403, the operation unit 1404, and the calculation unit 1405 in the embodiment of the present application can be a processor in an electronic device.

[0175] In an embodiment of the present application, if the data volume of the convolution data in the first direction cannot be divided by the first data volume, then during the first convolution operation (convolution operation is performed on the first portion) in the first convolution layer, it is not aligned with the first data volume, and all convolution operations after the first convolution operation in the first convolution layer are aligned with the first data volume, which only loses the computational efficiency of the first convolution operation. However, the cache space used can be greatly reduced in the entire operation process, and the cache space used by subsequent shortcuts can be reduced. The cache space used can be minimized while maximizing the computational efficiency, thereby balancing the computational efficiency and the cache space.

[0176] See also Fig.15 , Fig.15 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig.15 As shown, the electronic device 1500 includes a processor 1501 and a memory 1502, and the processor 1501 and the memory 1502 can be connected to each other via a communication bus 1503. The communication bus 1503 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 1503 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 The memory 1502 is used to store computer programs. The computer programs include program instructions. The processor 1501 is configured to call the program instructions. The program includes instructions for executing Figures 6 to 8 Some or all of the steps in the method shown.

[0177] Processor 1501 may be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.

[0178] The memory 1502 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0179] In addition, the electronic device 1500 may also include common components such as a communication interface and an antenna, which will not be described in detail here.

[0180] In an embodiment of the present application, if the data volume of the convolution data in the first direction cannot be divided by the first data volume, then during the first convolution operation (convolution operation is performed on the first portion) in the first convolution layer, it is not aligned with the first data volume, and all convolution operations after the first convolution operation in the first convolution layer are aligned with the first data volume, which only loses the computational efficiency of the first convolution operation. However, the cache space used can be greatly reduced in the entire operation process, and the cache space used by subsequent shortcuts can be reduced. The cache space used can be minimized while maximizing the computational efficiency, thereby balancing the computational efficiency and the cache space.

[0181] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any data processing method recorded in the above method embodiments.

[0182] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0183] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0185] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0187] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0188] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0189] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method is applied to image data processing of a unet network, and the method comprises: Obtaining convolution data of the first convolutional layer in the first calculation stage; Determining whether the data volume of the convolution data in the first direction is divisible by a first data volume, where the first data volume is the granularity of data output by a basic computing unit required by the Winograd method when the computing efficiency is the highest; the convolution data is two-dimensional data, and the first direction is a width direction or a height direction of an image; If not, the convolution data is divided into N parts in the first direction, the data amount of the first part of the N parts is less than the first data amount, the data amounts of the other parts of the N parts except the first part are equal to the first data amount, and N is an integer greater than or equal to 2; Convolution operations are performed on the first to Nth parts of the N parts in sequence.

2. The method according to claim 1, characterized in that The method further comprises: When the amount of the convolution data in the first direction is divisible by the first data amount, the convolution data is equally divided into M parts in the first direction, where M is an integer greater than or equal to 1; Convolution operations are performed on the first to the Mth parts of the M parts in sequence.

3. The method according to claim 1, characterized in that The sequentially performing convolution operations on the first to the Nth portions of the N portions includes: When performing a convolution operation on a first portion of the N portions, outputting a convolution result of the first portion; When performing a convolution operation on the second of the N parts, a convolution operation is performed based on the data output by the previous layer of the first convolution layer, the overlapping data of the convolution operation performed on the second part and the convolution operation performed on the previous part of the second part, and the convolution result of the second part is output, and the second part is any one of the other parts; the size of the convolution result of the second part in the first direction is equal to the first data amount.

4. The method according to claim 1, characterized in that: In other calculation stages after the first calculation stage, a size of a convolution result of the first convolution layer after any convolution operation in the first direction is equal to the first data amount.

5. The method according to claim 1, characterized in that: The method is applied to a first network structure, the first network structure includes P pooling layers, and before obtaining the convolution data of the first convolution layer in the first calculation stage, the method further includes: Obtain input data required for the first calculation stage, and determine a second data amount according to the P pooling layers and the first data amount; Determining whether the amount of the input data in the first direction is divisible by the second amount of data; If not, the input data is divided into Q parts in the first direction, the data amount of the first part of the Q parts is greater than the second data amount and less than twice the second data amount, the data amount of the other parts of the Q parts except the first part is equal to the second data amount, and Q is an integer greater than or equal to 2; The first of the Q parts is used as input data for the first step of the first layer in the first calculation stage.

6. The method according to claim 5, characterized in that The method further comprises: When the amount of the input data in the first direction is divisible by the second amount of data, the input data is equally divided into S parts in the first direction, where S is an integer greater than or equal to 1; The first of the S parts is used as input data for the first step of the first layer in the first calculation stage.

7. The method according to claim 5, characterized in that The first network structure further includes a merging layer. Before obtaining the input data required for the first calculation stage, the method further includes: Calculating a maximum cache space used when the merging layer of the first network structure performs operations; A third data amount is obtained based on the maximum cache space and the first data amount, and the third data amount is greater than or equal to the sum of the maximum cache space and the first data amount; the amount of input data required for the first calculation stage is equal to the third data amount.

8. A data processing device, characterized in that: The device is applied to image data processing of a unet network, and the device comprises: An acquisition unit, used to acquire convolution data of the first convolution layer in the first calculation stage; a judging unit, configured to judge whether the data volume of the convolution data in the first direction is divisible by a first data volume, wherein the first data volume is the granularity of data output by a basic computing unit required by the Winograd method when the computing efficiency is the highest; the convolution data is two-dimensional data, and the first direction is a width direction or a height direction of an image; a splitting unit, configured to, if the judgment result of the judging unit is no, split the convolution data in the first direction into N parts, wherein the data amount of a first part of the N parts is less than the first data amount, and the data amounts of other parts of the N parts except the first part are equal to the first data amount, and N is an integer greater than or equal to 2; The operation unit is used to perform convolution operations on the first to Nth parts of the N parts in sequence.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to perform the method according to any one of claims 1 to 7.

Images