Pooling layer parallel processing method, computing device, computing equipment, storage medium

Abstract

This disclosure provides a parallel processing method, computing device, computing apparatus, and storage medium for pooling layers. The parallel processing method for pooling layers according to this disclosure includes: obtaining the backpropagation gradient matrix of the pooling layer; splitting the backpropagation gradient matrix to obtain K sub-matrices, where K is an integer greater than 1; performing parallel back pooling operations on the K sub-matrices respectively to obtain K output sub-matrices; and accumulating the results of the K output sub-matrices to obtain an output matrix corresponding to the backpropagation gradient matrix. By utilizing the parallel processing method, computing device, computing apparatus, and storage medium for pooling layers provided in some embodiments of this disclosure, the parallel strategy of the backpropagation operation of the pooling layer can be adjusted by splitting the backpropagation gradient matrix of the pooling layer, avoiding sliding window overlap, improving computational parallelism and accuracy, thereby increasing computational efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to a pooling layer parallel processing method, computing device, computing equipment, and storage medium. Background Technology

[0002] Pooling layers are widely used in various types of neural networks. For example, a convolutional neural network (CNN) typically consists of convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract image features, pooling layers reduce the number of features, and fully connected layers perform non-linear combinations of features and predict categories. Pooling layers are considered an indispensable part of CNNs, providing them with robust robustness. The forward operation of pooling layers is similar to image downsampling; this operation reduces the size of features and the number of subsequent parameters, thus suppressing overfitting. The backward operation of pooling layers refers to backpropagating gradient values ​​during neural network training. In practice, various calculations in the neural network are usually performed in parallel to improve processing efficiency. Summary of the Invention

[0003] Some embodiments of this disclosure provide a parallel processing method, computing device, computing equipment, and storage medium for pooling layers, which are used to perform parallel processing on pooling layers in various types of convolutional neural networks, thereby improving parallelism and increasing computational efficiency.

[0004] According to one aspect of this disclosure, a parallel processing method for pooling layers is provided. The parallel processing method for pooling layers includes: obtaining the backpropagation gradient matrix of the pooling layer; splitting the backpropagation gradient matrix to obtain K sub-matrices, where K is an integer greater than 1; performing parallel back pooling operations on the K sub-matrices respectively to obtain K output sub-matrices; and accumulating the results of the K output sub-matrices to obtain an output matrix corresponding to the backpropagation gradient matrix.

[0005] According to some embodiments of this disclosure, performing back pooling operations according to sliding windows and step sizes, splitting the backpropagation gradient matrix includes: splitting the backpropagation gradient matrix into K sub-matrices in a way that avoids sliding window overlap during the back pooling operation.

[0006] According to some embodiments of this disclosure, K equals 4, and splitting the backpropagation gradient matrix into K sub-matrices includes: splitting the backpropagation gradient matrix into 4 sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

[0007] According to some embodiments of this disclosure, the parallel processing method for the pooling layer further includes: calculating the offset coordinates of each sub-matrix after splitting based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix, wherein the offset coordinates represent the data points in the sub-matrix corresponding to the data points in the output matrix.

[0008] According to some embodiments of this disclosure, the pooling layer is one of the following: a max pooling layer, an average pooling layer, an adaptive max pooling layer, and an adaptive average pooling layer.

[0009] According to some embodiments of this disclosure, when the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the parallel processing method of the pooling layer further includes: calculating the step size and kernel size required for the back pooling operation based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix.

[0010] According to another aspect of this disclosure, a computing apparatus is provided. The computing apparatus includes: an acquisition unit configured to acquire the backpropagation gradient matrix of a pooling layer; a splitting unit configured to split the backpropagation gradient matrix to obtain K sub-matrices, where K is an integer greater than 1; a computation unit configured to perform parallel back pooling operations on the K sub-matrices respectively to obtain K output sub-matrices; and to accumulate the results of the K output sub-matrices to obtain an output matrix corresponding to the backpropagation gradient matrix.

[0011] According to some embodiments of this disclosure, the operation unit performs reverse pooling operation according to the sliding window and the step size, and the splitting unit splits the backpropagation gradient matrix by: splitting the backpropagation gradient matrix into K sub-matrices in a way that avoids sliding window overlap during the reverse pooling operation.

[0012] According to some embodiments of this disclosure, K equals 4, and the splitting unit splits the backpropagation gradient matrix into K sub-matrices by: splitting the backpropagation gradient matrix into 4 sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

[0013] According to some embodiments of this disclosure, the computation unit is further configured to: calculate the offset coordinates of each sub-matrix after splitting based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix, wherein the offset coordinates represent the data points in the sub-matrix corresponding to the data points in the output matrix.

[0014] According to some embodiments of this disclosure, the pooling layer is one of the following: a max pooling layer, an average pooling layer, an adaptive max pooling layer, and an adaptive average pooling layer.

[0015] According to some embodiments of this disclosure, when the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the computation unit is further configured to: calculate the step size and kernel size required for the back pooling operation based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix.

[0016] According to another aspect of this disclosure, a computing device is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable code that, when executed by the processor, performs the pooling layer parallel processing method as described above.

[0017] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which instructions are stored, which, when executed by a processor, implement the pooling layer parallel processing method as described above.

[0018] By utilizing the parallel processing method, computing device, computing equipment, and storage medium for pooling layers provided in some embodiments of this disclosure, it is possible to adjust the parallel strategy of the backpropagation gradient matrix of the pooling layer by splitting it, thereby avoiding window overlap, improving computational parallelism and computational accuracy, and thus increasing computational efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1A A schematic diagram of forward propagation in the average pooling layer is shown;

[0021] Figure 1B A schematic diagram of backpropagation in the average pooling layer is shown;

[0022] Figure 2A A schematic diagram of forward propagation in the max pooling layer is shown;

[0023] Figure 2B A schematic diagram of backpropagation in the max pooling layer is shown;

[0024] Figure 3A and Figure 3B A schematic diagram of atomic accumulation of the pooling layer during backpropagation is shown;

[0025] Figure 4 This diagram illustrates the sliding window overlap that occurs during the reverse pooling operation in the pooling layer.

[0026] Figure 5 A flowchart illustrating a parallel processing method for pooling layers according to some embodiments of the present disclosure is shown;

[0027] Figure 6 A schematic diagram illustrating the splitting of the backpropagation gradient matrix according to some embodiments of the present disclosure is shown;

[0028] Figures 7A-7D A schematic diagram illustrating the reverse pooling operation process according to some embodiments of the present disclosure is shown;

[0029] Figure 8 A schematic block diagram of a computing device according to some embodiments of the present disclosure is shown;

[0030] Figure 9 A schematic block diagram of a computing device according to some embodiments of the present disclosure is shown;

[0031] Figure 10 A schematic diagram of the architecture of an exemplary computing device according to some embodiments of the present disclosure is shown;

[0032] Figure 11 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is shown. Detailed Implementation

[0033] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0034] Furthermore, as shown in this disclosure and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "comprising" or "including" and similar terms mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, without excluding other elements or objects. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0035] This disclosure uses flowcharts to illustrate the steps of a method according to embodiments of this disclosure. It should be understood that the preceding or following steps are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes. It is understood that the technical terms and nouns used herein have meanings well-known to those skilled in the art.

[0036] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0037] Neural Networks (NNs), or simply neural networks, are mathematical models that mimic the behavioral characteristics of animal neural networks to perform distributed parallel information processing. These networks rely on system complexity to adjust the connections between a large number of internal nodes, thereby achieving information processing. Regardless of the type, neural networks share common characteristics such as massively parallel processing, distributed storage, flexible topology, high redundancy, and nonlinear operations. They also possess advantages in computational speed, associative ability, adaptability, fault tolerance, and self-organization. These characteristics and capabilities form the technological foundation for neural networks to simulate intelligent activities and have found important applications in various technological fields. For example, neural networks can be applied to image processing, data compression, video encoding / decoding, and signal processing.

[0038] In the process of performing various calculations in neural networks using parallel processing, such as the parallel calculation of pooling layers, the operation of pooling layers is performed by a set sliding window and step size. During backpropagation, there is a problem of sliding window overlap, which will seriously affect the parallelism of computation, reduce computational efficiency, and hinder the application and development of neural networks.

[0039] Based on this, this disclosure is applied to the field of high-efficiency computing in neural network processing. Specifically, it proposes a parallel processing method for pooling layers, which is used to perform parallel processing on pooling layers in various types of convolutional neural networks, thereby improving parallelism and increasing computational efficiency.

[0040] Next, the calculation process of the pooling layer will be described first with reference to the accompanying drawings to illustrate the sliding window overlap problem that exists in the reverse operation process. Then, the parallel processing method of the pooling layer according to some embodiments of this disclosure will be described in detail.

[0041] In related technologies, pooling layers generally include average pooling layers and max pooling layers, both of which have fixed sliding window sizes and strides. Building upon this, adaptive average pooling layers and adaptive max pooling layers have also been developed. "Adaptive" means that the sliding window size and stride of the pooling layer are variable, obtained through adaptive calculation during the computation process.

[0042] Figure 1A A schematic diagram of forward propagation in the average pooling layer is shown. Figure 1B A schematic diagram of backpropagation in an average pooling layer is shown. As an example, in... Figure 1A In this model, the input matrix is ​​4x4, containing 4x4 data points, and the output matrix is ​​2x2, meaning the amount of data is reduced through average pooling. Specifically, the sliding window size from the input matrix to the output matrix is ​​2x2, with a stride of 2. The forward propagation of the average pooling layer can be a process of downsampling data points by performing forward operations through mean calculation, as described in the reference. Figure 1A The data values ​​in the top-left 2x2 matrix are averaged to obtain data points with a value of 2. Figure 1A The diagram illustrates the correspondence between sliding windows using different filling effects. Backpropagation in average pooling layers can be a process of reverse calculation using the average gradient to achieve gradient backpropagation. For example, during neural network training, the loss value needs to be calculated based on the backpropagated gradient, and the network parameters are adjusted accordingly. (Reference) Figure 1B The data values ​​in the matrix on the right can be the obtained gradient matrix. For average pooling layers, the gradient values ​​are averaged during backpropagation to obtain the data values ​​at the corresponding sliding window positions. Similarly, Figure 1B The text also uses different fill effects to illustrate the correspondence between sliding windows.

[0043] Figure 2A A schematic diagram of forward propagation in the max-pooling layer is shown. Figure 2B A schematic diagram of backpropagation in a max-pooling layer is shown. As an example, in Figure 2A In this model, the input matrix has a size of 4*4, meaning it contains 4*4 data points, and the output matrix has a size of 2*2. This is achieved by reducing the amount of data through max pooling. Specifically, the sliding window size from the input matrix to the output matrix is ​​2*2, with a stride of 2. The forward propagation of the max pooling layer can be described as a process of downsampling data points by performing forward operations using maximum value calculations. (See reference...) Figure 2A The largest data value of 4 in the top-left 2x2 matrix is ​​used as the pooled data value. Figure 1A The diagram illustrates the correspondence between sliding windows using different filling effects. Backpropagation in a max-pooling layer can be a process of performing reverse calculations using the maximum value to achieve gradient backpropagation. For example, during neural network training, the loss value needs to be calculated based on the backpropagated gradient, and the network parameters are adjusted accordingly. (Reference) Figure 2B The data values ​​in the matrix on the right can be the obtained gradient matrix. For max pooling layers, during backpropagation, the gradient values ​​are passed to the position of the maximum value in the forward operation. As an example, in Figure 2A At the first sliding window position in the left-hand matrix (shown as a diagonal stripe fill effect), the data point in the 2nd row and 2nd column has the largest value. Therefore, during backpropagation, the gradient value of 0.8 is added to this position. Similarly, Figure 2B The text also uses different fill effects to illustrate the correspondence between sliding windows.

[0044] Because the sliding window portion of the pooling process may overlap, adjacent data points might be accumulated to the same data point during backpropagation. This necessitates the use of atomic accumulation (ADD) to avoid computational errors. However, atomic accumulation disrupts computational parallelism, resulting in a significant performance penalty compared to regular accumulation. Furthermore, due to minute parallelism variations in computing devices, atomic accumulation can lead to inconsistent accumulation order, causing bit inconsistencies in the results and affecting computational accuracy.

[0045] Figure 3A and Figure 3B This diagram illustrates the atomic accumulation of the pooling layer during backpropagation. Figure 3A and Figure 3B In the example, a 7x7 X matrix (xtensor) is transformed into a 3x3 Y matrix (ytensor) through pooling. Specifically, the sliding window size is 3x3, and the stride is 2. During forward propagation... Figure 3A The first sliding window position of the X matrix (shown as a diagonal stripe fill effect) is pooled to Figure 3A The first data point in the Y matrix on the right side. Next, Figure 3B The X matrix in the diagram shows the position of the second sliding window (shown as a dotted fill effect) pooled to... Figure 3B The second data point in the Y matrix on the right side of the graph. During backpropagation, the corresponding data point in the Y matrix on the right side of the graph will be accumulated into the corresponding sliding window position in the X matrix on the left side of the graph. However, as... Figure 3A and Figure 3BAs shown, there is window overlap during backpropagation. That is, the three data points filled with the character "A" are the overlap of two sliding windows, meaning the data value here will be obtained by data accumulation. In other words, adjacent data are accumulated to the same data point position. To avoid calculation errors, atomic accumulation operations are needed to calculate the three data points filled with the character "A". This will destroy the parallelism of the calculation and reduce the computational efficiency.

[0046] refer to Figure 3A and Figure 3B The effect shown Figure 4 The diagram illustrates the sliding window overlap during the back pooling operation (i.e., back propagation) of the pooling layer. The data points at the grayscale filling positions all exhibit sliding window overlap, which means that the data points at these positions need to be atomically accumulated.

[0047] This disclosure improves parallelism and computational accuracy by adjusting the parallel strategy of the backpropagation operation of the pooling layer, avoiding atomic accumulation in the overlapping part of the backpropagation sliding window of the aforementioned pooling layer, thereby increasing computational efficiency. The solution provided by this disclosure can be applied to training scenarios of various neural networks in artificial intelligence that include pooling layers, and is suitable for most commonly used pooling layer types as well as adaptive pooling layer operations.

[0048] Below, we will combine Figure 5 A parallel processing method for pooling layers according to embodiments of this disclosure will be described. For example... Figure 5 As shown, the parallel processing method for pooling layers according to an embodiment of this disclosure includes steps S101-S104. Specifically, in step S101, the backpropagation gradient matrix of the pooling layer is obtained. This backpropagation gradient matrix refers to the matrix in which the numerical values ​​represent gradient values; for example, it could be... Figure 3A and Figure 3B The Y matrix shown on the right side of the diagram is, as can be understood, only schematically illustrated, and can be a matrix of any dimension. Figure 3A and Figure 3B The dimensions of the X matrix, the sliding window size, and the step size shown are also illustrative. The method according to the embodiments of this disclosure can be applied to pooling layer operations with arbitrary parameters, and is not limited herein.

[0049] Furthermore, according to some embodiments of this disclosure, the pooling layer implemented according to this disclosure can be one of the following: a max pooling layer, an average pooling layer, an adaptive max pooling layer, and an adaptive average pooling layer. In other words, the parallel processing method according to embodiments of this disclosure is applicable to various types of pooling layers, and is not limited thereto.

[0050] Next, in step S102, the backpropagation gradient matrix is ​​split into K sub-matrices, where K is an integer greater than 1. In step S103, parallel backpooling operations are performed on the K sub-matrices to obtain K output sub-matrices.

[0051] According to some embodiments of this disclosure, performing back pooling operations according to sliding windows and step sizes, splitting the backpropagation gradient matrix includes: splitting the backpropagation gradient matrix into K sub-matrices in a way that avoids sliding window overlap during the back pooling operation.

[0052] In the method according to embodiments of this disclosure, the backpropagation gradient matrix is ​​first split, and then the split sub-matrices are backpropagated in parallel. In comparison, in... Figure 3A and Figure 3B In the scenario shown, the backpropagation gradient matrix is ​​calculated globally using parallel backpropagation, i.e., Figure 3A Each data point in the Y matrix is ​​backpropagated in parallel. To avoid atomic accumulation caused by sliding window overlap during parallel processing, the method according to this disclosure splits the backpropagation gradient matrix of the pooling layer, adjusting the parallel strategy of the pooling layer's backpropagation operation, i.e., changing the computation order of parallel operations, thereby avoiding sliding window overlap in parallel computation. The specific process of splitting will be discussed below. Figure 6 Describe it.

[0053] Next, in step S104, the results of the obtained K output sub-matrices are summed to obtain the output matrix corresponding to the backpropagation gradient matrix. Here, the output matrix refers to, for example... Figure 3A and Figure 3B The output matrix obtained by summing the X matrix shown on the left is the output result of the reverse pooling operation of the pooling layer.

[0054] According to some embodiments of this disclosure, K equals 4, and splitting the backpropagation gradient matrix into K sub-matrices includes: splitting the backpropagation gradient matrix into 4 sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

[0055] Figure 6 A schematic diagram illustrating the splitting of the backpropagation gradient matrix according to some embodiments of this disclosure is shown. For example... Figure 6As shown, for a matrix, splitting it according to the rules of odd rows and odd columns (ee), odd rows and even columns (eo), even rows and odd columns (oe), and even rows and even columns (oo) yields four corresponding sub-matrices. For the split sub-matrices, such as the ee matrix, the data points within this ee matrix do not exhibit sliding window overlap during backpropagation. Therefore, there will be no atomic accumulation during parallel processing on a per-matrix basis. Similarly, Figure 6 The eo, oe, and oo matrices shown do not exhibit sliding window overlap during parallel computation.

[0056] Specifically, Figures 7A-7D A schematic diagram illustrating the reverse pooling operation process according to some embodiments of this disclosure is shown. Combined with... Figures 7A-7D The description illustrates the process of avoiding sliding window overlap through the above splitting. Figures 7A-7D In this example, we use a 7x7 X matrix, a 3x3 Y matrix, a 3x3 sliding window, and a stride of 2. It is understood that the method disclosed herein can be similarly applied to other pooling layer parameters, and no limitations are imposed here.

[0057] For a 3x3 backpropagation gradient matrix (i.e., the Y matrix), it is first decomposed into the following parts: Figures 7A-7D The four submatrices shown on the right side of the image have different fill effects within the Y matrix. For example, refer to... Figure 7A We can first perform parallel backward pooling operations on the submatrix of type ee (also called the ee submatrix), that is, perform gradient backpropagation. During this process, if it is an average pooling layer, the gradient mean is backpropagated; if it is a max pooling layer, the gradient value is backpropagated to the position of the maximum value in the forward operation. From Figure 7A It can be seen that these data points belonging to the ee submatrix do not have sliding window overlap during parallel computation, thus preventing atomic accumulation. Similarly, refer to... Figure 7B To perform the gradient backpropagation process of the eo submatrix, refer to... Figure 7C For the gradient backpropagation process of the oe submatrix, refer to... Figure 7D The gradient backpropagation process for the oo submatrix is ​​performed. It is understood that the method of this disclosure does not restrict the processing order of the submatrices obtained from the above splitting. For example, it can be performed in the order of ee, eo, oe, oo, or in other orders. Furthermore, the operations between these submatrices can be performed in parallel; no restrictions are placed here. Finally, the obtained data matrices are summed to obtain the final X matrix.

[0058] According to some embodiments of this disclosure, the pooling layer parallel processing method can process the pooling layers according to... Figure 6 The process shown involves splitting the data into K sub-matrices, which are then subjected to parallel reverse pooling operations to obtain K output sub-matrices. The results of these K output sub-matrices are then summed to obtain the output matrix corresponding to the backpropagation gradient matrix of the input. This process achieves the calculation from the Y matrix to the X matrix. Furthermore, as described above, the processing according to this scheme avoids sliding window overlap in parallel computation, i.e., avoids atomic accumulation of data points, thereby realizing a parallel strategy for adjusting the reverse operation of the pooling layer, improving computational parallelism, accuracy, and efficiency.

[0059] The parallel processing method for pooling layers according to some embodiments of this disclosure may further include: calculating the offset coordinates of each sub-matrix after splitting based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix, wherein the offset coordinates represent the data points in the sub-matrix corresponding to the data points in the output matrix. The offset coordinates calculated in this process are used to establish, for example... Figure 7A The diagram shows the correspondence between data points in the Y matrix and data points in the X matrix. For example, the data point ee located in the upper left corner of the Y matrix corresponds to a 3*3 sliding window in the upper left corner of the X matrix. That is, the gradient value of the data point ee in the upper left corner of the Y matrix is ​​backpropagated to the 3*3 sliding window in the upper left corner of the X matrix.

[0060] When the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the parallel processing method for pooling layers according to some embodiments of this disclosure may further include: calculating the stride and kernel size required for the backpropagation gradient matrix and the output matrix based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix. Compared to conventional pooling layers, the kernel size and stride of the adaptive pooling layer are calculated based on a function and do not require manual setting. Furthermore, the kernel size and stride are dynamically variable, and adjacent pooling windows can overlap.

[0061] The following describes the process of calculating the stride and kernel size required for the reverse pooling operation. As an example, the calculation process can be divided into two cases: the first is when the size of the X matrix is ​​an integer multiple of the size of the Y matrix, and the second is when the size of the X matrix is ​​not an integer multiple of the size of the Y matrix.

[0062] In the first case, the kernels of the adaptive pooling layers have the same size and do not overlap. The formula for calculating adaptive pooling is:

[0063] stride = X size / Y size (1)

[0064] kernel=X size -(Y size -1)*stride (2)

[0065] Where stride represents the step size, X size Represents the size of matrix X, Y size The Y matrix represents the size of the Y matrix, and kernel represents the kernel size. It can be understood that during the backpropagation process of the pooling layer, the Y matrix represents the backpropagation gradient matrix obtained, and the X matrix represents the output matrix obtained after the back pooling operation.

[0066] In the second scenario, the kernel size and stride of the adaptive pooling layer change dynamically. The formula for calculating adaptive pooling is:

[0067] stride i =floor((i+1)*X size / Y size )-floor(i*X size / Y size (3)

[0068] kernel i =cell((i+1)*X size / Y size )-floor(i*X size / Y size (4)

[0069] Next, we will use an adaptive average pooling layer as a specific example to describe the parallel processing method of the pooling layer according to embodiments of this disclosure.

[0070] The first step is to read in the backpropagation gradient matrix (Y matrix), which includes data points representing gradient values ​​to be used for gradient value backpropagation during the backpropagation process; in this example, it is gradient mean backpropagation.

[0071] The second step is to calculate the adaptive pooling sliding window (i.e., calculate the kernel size) and step size. As an example, the adaptive sliding window and step size can be calculated according to the above formulas (1)-(4).

[0072] The third step is to calculate the actual offset coordinates after splitting. For example, in the case where the backpropagation gradient matrix is ​​split into four sub-matrices according to the rules of odd row odd column, odd row even column, even row odd column, and even row even column, the actual corresponding coordinate position range of the four parts ee, eo, oe, and oo in the Y matrix can be calculated according to the sliding window and step size calculated above. The calculation process can be carried out according to the following formulas (5)-(8):

[0073] start_e i =floor((2*i)*X size / Y size (5)

[0074] end_e i =ceil((2*i)i*X size / Y size (6)

[0075] start_o i =floor((2*i+1)*X size / Y size (7)

[0076] end_o i =ceil((2*i+1)i*X size / Y size (8)

[0077] Among them, start_e i This represents the corresponding position of the i-th odd-numbered row, end_e i This represents the corresponding position of the i-th odd-numbered column, start_o i This represents the corresponding position of the i-th even-numbered row, end_o i This represents the corresponding position of the i-th even-numbered column. Therefore, according to the above formulas (5)-(8), the start and end positions of the sliding window in the odd and even cases under each independent dimension can be determined.

[0078] The fourth step is to split the obtained Y matrix into the following parts: Figure 6 The four submatrices shown.

[0079] Fifth, following the reverse pooling operation, each gradient value in the Y matrix is ​​divided by its corresponding sliding window size, and the resulting average gradient value is passed to the corresponding sliding window position in the forward pass. In this process, the sub-matrices obtained from the above split are processed in parallel, thus avoiding sliding window overlap during gradient backpropagation.

[0080] The sixth step is to sum the calculation results to obtain the final X matrix.

[0081] The processing method described above avoids sliding window overlap in parallel computing, i.e., avoids atomic accumulation of data points, thereby realizing a parallel strategy for adjusting the reverse operation of the pooling layer, improving computational parallelism, accuracy, and efficiency.

[0082] According to another aspect of this disclosure, a computing device is also provided, for example, a processor for performing neural network calculations, involving multiple cores to achieve parallel processing. As an example, Figure 8 A schematic block diagram of a computing device according to some embodiments of the present disclosure is shown. Figure 8As shown, the computing device 1000 according to the embodiments of the present disclosure may include an acquisition unit 1010, a splitting unit 1020, and a calculation unit 1030.

[0083] Specifically, the acquisition unit 1010 can be configured to acquire the backpropagation gradient matrix of the pooling layer. The splitting unit 1020 can be configured to split the backpropagation gradient matrix to obtain K sub-matrices, where K is an integer greater than 1. The operation unit 1030 can be configured to perform parallel back pooling operations on the K sub-matrices to obtain K output sub-matrices; and to accumulate the results of the K output sub-matrices to obtain the output matrix corresponding to the backpropagation gradient matrix.

[0084] According to some embodiments of this disclosure, the operation unit 1030 performs reverse pooling operations according to a sliding window and a step size. The splitting unit 1020 splits the backpropagation gradient matrix by splitting it into K sub-matrices in a way that avoids sliding window overlap during the reverse pooling operation.

[0085] According to some embodiments of this disclosure, K equals 4. The splitting unit 1020 splits the backpropagation gradient matrix into K sub-matrices by: splitting the backpropagation gradient matrix into 4 sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

[0086] According to some embodiments of this disclosure, the operation unit 1030 is further configured to: calculate the offset coordinates of each sub-matrix after splitting based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix, wherein the offset coordinates represent the data points in the sub-matrix corresponding to the data points in the output matrix.

[0087] According to some embodiments of this disclosure, the pooling layer is one of the following: a max pooling layer, an average pooling layer, an adaptive max pooling layer, and an adaptive average pooling layer.

[0088] According to some embodiments of this disclosure, when the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the computation unit 1030 can also be configured to: calculate the step size and kernel size required for the back pooling operation based on the dimension of the backpropagation gradient matrix and the dimension of the output matrix.

[0089] Regarding the specific implementation process of parallel pooling layer processing by the computing device according to embodiments of this disclosure, please refer to the parallel pooling layer processing methods according to some embodiments of this disclosure described above in conjunction with the accompanying drawings, which will not be repeated here. The computing device using embodiments of this disclosure can perform similar parallel processing processes and achieve similar technical effects.

[0090] According to another aspect of this disclosure, a computing device is provided. Figure 9 A schematic block diagram of a computing device according to an embodiment of the present disclosure is shown.

[0091] like Figure 9 As shown, the computing device 2000 may include a processor 2010 and a memory 2020. According to some embodiments of this disclosure, the memory 2020 stores executable instructions that, when run by the processor 2010, can perform the pooling layer parallel processing method as described above.

[0092] Processor 2010 can perform various actions and processes according to the program stored in memory 2020. Specifically, processor 2010 can be an integrated circuit with signal processing capabilities. The general-purpose processor can be a microprocessor or any conventional processor. For example, the processor here can refer to a processing device capable of performing neural network calculations.

[0093] Memory 2020 stores computer-executable instructions that, when executed by processor 2010, cause the processor to implement a pooling layer parallel processing method according to some embodiments of this disclosure. Memory 2020 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the memory described herein can be of any suitable type. As an example, by executing the computer-executable instructions in memory 2020, the processor is able to implement a pooling layer parallel processing method, such as the steps of the pooling layer parallel processing method described above in conjunction with the accompanying drawings.

[0094] The pooling layer parallel processing method or computing device according to the embodiments of this disclosure can also be used by means of, Figure 10 The architecture of the exemplary computing device 3000 shown is implemented as follows. Figure 10 As shown, the computing device 3000 may include a bus 3010, one or more central processing units (CPUs) 3020, read-only memory (ROM) 3030, random access memory (RAM) 3040, a communication port 3050 connected to a network, input / output components 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files required for the processing and / or communication of the methods provided in this disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 10 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 10 One or more components in the computing device shown.

[0095] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is also provided. Figure 11 A schematic diagram 4000 of a storage medium according to the present disclosure is shown.

[0096] like Figure 11 As shown, a computer-readable storage medium 4000 stores computer-readable instructions 4010. When the computer-readable instructions 4010 are executed by a processor, the pooling layer parallel processing method described with reference to the above figures can be performed. The computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 4000 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions 4010 stored on the computer-readable storage medium 4000, the pooling layer parallel processing method provided according to the embodiments of this disclosure as described above can be performed.

[0097] This disclosure provides a parallel processing method, computing device, computing equipment, and storage medium for pooling layers, enabling parallel processing of pooling layers in various types of convolutional neural networks, thereby improving parallelism and increasing computational efficiency. The parallel processing method for pooling layers provided in this disclosure can be used to split the backpropagation gradient matrix of the pooling layer, adjust the parallel strategy of the backpropagation operation, avoid sliding window overlap, improve computational parallelism and accuracy, and thus increase computational efficiency. The solution provided in this disclosure can be applied to training scenarios of various neural networks including pooling layers in artificial intelligence, and is suitable for most commonly used pooling layer types and adaptive pooling layer operations.

[0098] Those skilled in the art will understand that the contents disclosed herein can be varied and modified in many ways. For example, the various devices or components described above can be implemented in hardware, or in software, firmware, or a combination of some or all of the three.

[0099] Furthermore, while this disclosure makes various references to certain units in computing devices and computing apparatuses according to embodiments of this disclosure, any number of different units may be used and run on clients and / or servers. The units are merely illustrative, and different aspects of methods, computing devices, and computing apparatuses may use different units.

[0100] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This disclosure is not limited to any particular combination of hardware and software.

[0101] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms such as those defined in a common dictionary should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0102] The foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it. While several exemplary embodiments of the present disclosure have been described, those skilled in the art will readily understand that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined by the claims. It should be understood that the foregoing description is intended to illustrate the present disclosure and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A parallel training method for a neural network model, wherein, The neural network model includes convolutional layers and pooling layers, and the method includes: Generate a backpropagation gradient matrix, wherein the backpropagation gradient matrix serves as the input data for the pooling layer in the neural network model; The backpropagation gradient matrix is ​​split into K sub-matrices, where K is an integer greater than 1; The K sub-matrices are split and reverse pooling is performed in parallel using multiple cores in the computer for parallel processing to obtain K output sub-matrices. The results of the K output sub-matrices are summed to obtain the output matrix corresponding to the backpropagation gradient matrix; and The parameters of the neural network model are trained based on the output matrix.

2. The method according to claim 1, characterized in that, The multiple kernels perform the back pooling operation according to a sliding window and a step size, and the splitting of the backpropagation gradient matrix includes: To avoid window overlap during the backpropagation gradient matrix operation, the backpropagation gradient matrix is ​​split into the K sub-matrices.

3. The method according to claim 2, characterized in that, When K equals 4, splitting the backpropagation gradient matrix into the K sub-matrices includes: The backpropagation gradient matrix is ​​split into four sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: Based on the dimensions of the backpropagation gradient matrix and the output matrix, the offset coordinates of each sub-matrix after splitting are calculated, wherein the offset coordinates represent the data points in the sub-matrix that correspond to the data points in the output matrix.

5. The method according to any one of claims 1-3, characterized in that, The pooling layer is one of the following: max pooling layer, average pooling layer, adaptive max pooling layer, and adaptive average pooling layer.

6. The method according to claim 1, characterized in that, When the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the method further includes: The step size and kernel size required for the reverse pooling operation are calculated based on the dimensions of the backpropagation gradient matrix and the output matrix.

7. A computing device for parallel training of a neural network model, wherein, The neural network model includes convolutional layers and pooling layers, and the computing device includes: The acquisition unit is configured to generate a backpropagation gradient matrix, wherein the backpropagation gradient matrix serves as the input data for the pooling layer in the neural network model. The splitting unit is configured to split the backpropagation gradient matrix to obtain K sub-matrices, where K is an integer greater than 1; The processing unit is configured to use multiple cores in the computer for parallel processing to perform parallel reverse pooling operations on the split K sub-matrices respectively to obtain K output sub-matrices; The results of the K output sub-matrices are summed to obtain the output matrix corresponding to the backpropagation gradient matrix; and The parameters of the neural network model are trained based on the output matrix.

8. The computing device according to claim 7, characterized in that, The computation unit performs the reverse pooling operation according to the sliding window and step size, and the splitting unit splits the backpropagation gradient matrix as follows: To avoid window overlap during the backpropagation gradient matrix operation, the backpropagation gradient matrix is ​​split into the K sub-matrices.

9. The computing device according to claim 8, characterized in that, When K equals 4, the splitting unit splits the backpropagation gradient matrix into the K sub-matrices, including: The backpropagation gradient matrix is ​​split into four sub-matrices according to the rules of odd rows and odd columns, odd rows and even columns, even rows and even columns.

10. The computing device according to any one of claims 7-9, characterized in that, The arithmetic unit is further configured to: Based on the dimensions of the backpropagation gradient matrix and the output matrix, the offset coordinates of each sub-matrix after splitting are calculated, wherein the offset coordinates represent the data points in the sub-matrix that correspond to the data points in the output matrix.

11. The computing device according to any one of claims 7-9, characterized in that, The pooling layer is one of the following: max pooling layer, average pooling layer, adaptive max pooling layer, and adaptive average pooling layer.

12. The computing device according to claim 7, characterized in that, When the pooling layer is an adaptive max pooling layer or an adaptive average pooling layer, the computation unit is further configured to: The step size and kernel size required for the reverse pooling operation are calculated based on the dimensions of the backpropagation gradient matrix and the output matrix.

13. A computing device, comprising: processor; and A memory, wherein computer-readable code is stored in the memory, which, when executed by the processor, performs the method as described in any one of claims 1-6.

14. A non-transitory computer-readable storage medium having instructions stored thereon, characterized in that, When the instructions are executed by the processor, they implement the method as described in any one of claims 1-6.

Images