Processing method and device for executing maximum pooling in neural network
By splitting the pooling window and using the sampling unit, the complexity problem caused by different pooling windows and step sizes in the hardware accelerator is solved, and a simplified hardware design and cost-effective maximum pooling operation are achieved.
Patent Information
- Application Number
- CN202510784039.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-26
AI Technical Summary
In the existing technology, when implementing the maximum pooling layer, neural network hardware accelerators have different pooling windows and step sizes, which leads to complex hardware design logic and high cost.
By splitting the maximum pooling layer with a pooling window greater than 3 into multiple pooling windows less than or equal to 3, and combining it with the sampling unit, the maximum pooling operation of arbitrary size and step length is realized, and the controller is used to control the collaborative work of the pooling unit and the sampling unit.
The logic and implementation complexity of hardware design are reduced. Only simple-sized maximum pooling is required. Operations with pooling windows of any size are supported, simplifying hardware design and reducing costs.
Smart Images
Figure CN120706484A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to neural network technology, and in particular to a processing method and device for performing maximum pooling in a neural network. Background Art
[0002] Max pooling is a commonly used technique in convolutional neural networks (CNNs). Its main purpose is to reduce the dimensionality of the data while retaining important feature information. By using max pooling, the spatial dimension of the input feature map can be reduced, thereby reducing the amount of computation and the number of parameters, speeding up training, and improving the generalization ability of the model. The max pooling operation defines a spatial neighborhood (pooling window, such as a 3*3 window) and then selects the largest element from each window as the output. The stride process gradually reduces the spatial scale of the input representation, making the network invariant to small changes, redundancy, and transformations in the input image. By selecting the maximum value in each window, max pooling can effectively extract important features while ignoring unimportant details.
[0003] Max pooling is widely used in various vision tasks, such as image classification and object detection. By reducing the dimensionality of the data, max pooling helps models better process large-scale image data, improving their efficiency and effectiveness. Furthermore, max pooling helps models become more robust to small changes in the input image, thereby improving their generalization capabilities.
[0004] Both pooling and convolution have a stride parameter (such as 2*2), which indicates the distance the window moves in the width and height directions of the input feature map each time it operates, which affects the size of the output feature map.
[0005] Neural network hardware accelerators need to implement hardware units including maximum pooling layers. However, the pooling windows and step sizes of the maximum pooling layers vary. Considering multiple situations will make the hardware design logic complex and the cost high. Summary of the Invention
[0006] The purpose of the present invention is to overcome the deficiencies of the above-mentioned prior art and provide a processing method and device for performing maximum pooling in a neural network, so as to reduce the hardware design logic and implementation complexity and support maximum pooling operations for pooling windows of any size.
[0007] To achieve the above object, the technical solution of the present invention is:
[0008] In a first aspect, the present invention provides a method for performing maximum pooling in a neural network, wherein kh*kw is set as the height and width of the pooling window to achieve the target, and tkh*tkw is set as the remaining height and width after the split pooling window is completed. Initially, tkh=kh, tkw=kw. The method includes:
[0009] S1, calculate the pooling window size kh'*kw' for the first split:
[0010] S2, perform a maximum pooling operation with a pooling window size of kh'*kw';
[0011] S3, update tkh, tkw and the next pooling window parameter kh'*kw';
[0012] S4, if at least one of tkh and tkw is not 1, perform a maximum pooling operation with a pooling window of kh'*kw' and return to step S3; otherwise, continue to the next step;
[0013] S5, determine whether to add a point extraction operation based on the step size:
[0014] If the step height sh>1 or the width sw>1, then add the sh*sw sampling operation; if sh and sw are both 1, then do not add this operation.
[0015] Optionally, step S1 includes:
[0016] If the pooling window height tkh is 1, the pooling window height kh' after splitting is also 1, otherwise when tkh is an odd number > 1, kh'=3, and when tkh is an even number, kh'=2;
[0017] If the pooling window width tkw is 1, the pooling window height kw' after the split is also 1. Otherwise, when tkw is an odd number > 1, kw'=3; when tkw is an even number, kw'=2.
[0018] Optionally, the step S3 includes: tkh=tkh-(kh'-1), tkw=tkw-(kw'-1); if tkh=1, then kh'=1, otherwise kh'=3; if tkw=1, then kw'=1, otherwise kw'=3.
[0019] In a second aspect, the present invention provides a processing device for performing maximum pooling in a neural network, comprising a controller, a pooling unit, and a sampling unit;
[0020] The controller controls the pooling unit and the sampling unit based on any of the processing methods described above;
[0021] The pooling unit is a maximum pooling hardware unit that implements a step size of 1*1 and a pooling window of less than 3*3;
[0022] The sampling unit performs a sampling extraction operation on the maximum pooling result with a step size of 1 with an interval of sh in height and an interval of sw in width, thereby realizing a maximum pooling operation with a step size of sh*sw.
[0023] Optionally, the pooling unit and the sampling unit interact with the external memory through a data buffer.
[0024] Compared with the prior art, the present invention has the following beneficial effects:
[0025] This invention reduces hardware design logic and implementation complexity. The hardware only needs to implement max pooling for simple sizes (<=3*3), eliminating the need for max pooling for sizes greater than 3. The controller can be implemented in software or hardware. It supports max pooling operations for pooling windows of any size. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 A schematic diagram of the composition of a processing device for performing maximum pooling in a neural network provided in an embodiment of the present application;
[0027] Figure 2 This is the flow chart of the controller;
[0028] Figure 3 The maximum pooling implementation flow chart is as follows: the pooling window kh*kw is 4*3 and the step size sh*sw is 2*2;
[0029] Figure 4 Schematic diagram of the distances sh and sw moved in the height direction and width direction respectively;
[0030] Figure 5 Schematic diagram for realizing the simulation of step size sh*sw;
[0031] Figure 6 For the general Figure 5 Step 1 in the code is split into Step 1.1 and Step 1.2. DETAILED DESCRIPTION
[0032] Example:
[0033] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.
[0034] The neural network hardware accelerator needs to implement a hardware unit including a maximum pooling layer, but the size of the pooling window of the maximum pooling layer is different, and the step size is also different. If multiple situations need to be considered, the hardware design logic will be complicated and the cost will be high. This application cascades the maximum pooling with a pooling window size (size unit is pixel) greater than 3 with multiple maximum poolings less than or equal to 3, and adds a sampling unit to complete it. It can realize maximum pooling of any size pooling window and any step size, where the number of cascaded maximum poolings and the pooling window size are dynamically determined according to the parameters of the original pooling.
[0035] For details, see Figure 1 As shown, the processing device for performing maximum pooling in the neural network provided by the embodiment of the present application mainly includes a controller, which is used to control the pooling unit and the sampling unit to realize the splitting logic of the maximum pooling with a pooling window of kh*sw. The controller can be implemented by a computer software program or by a dedicated hardware circuit.
[0036] The pooling unit is a maximum pooling hardware unit that can achieve a step size of 1*1 and a pooling window of less than 3*3 (supporting pooling windows of 1*3, 3*1, 3*3, 1*2, 2*1, and 2*2).
[0037] The sampling unit extracts points from the maximum pooling result with a stride of 1, with a height interval of sh and a width interval of sw, to achieve a maximum pooling operation with a stride of sh*sw. The pooling unit and the sampling unit interact with the external memory through the data buffer.
[0038] The controller splits the largest pooling layer with a pooling window size greater than 3 into multiple pooling layers with a pooling window size of less than 3 and completes them in sequence. If the step size of the pooling layer is greater than 1, a sampling operation is added at the end to achieve it. Set kh*kw as the pooling window size (height and width) to achieve the target, and tkh*tkw as the remaining size after the split pooling window is completed. Initially, tkh=kh, tkw=kw, such as Figure 2 As shown, the controller process includes:
[0039] S1, calculate the pooling window size kh'*kw' for the first split
[0040] If the pooling window height tkh is 1, the height of the split small pooling window kh' is also 1. Otherwise, when tkh is an odd number > 1, kh'=3; when tkh is an even number, kh'=2.
[0041] Similarly, if the pooling window width tkw is 1, the pooling window height kw' after splitting is also 1, otherwise when tkw is an odd number > 1, kw'=3, and when tkw is an even number, kw'=2.
[0042] S2, perform a maximum pooling operation with a pooling window size of kh'*kw'.
[0043] S3, update tkh, tkw and the next pooling window parameters kh'*kw'
[0044] Among them, tkh=tkh-(kh'-1), tkw=tkw-(kw'-1); if tkh=1, then kh'=1, otherwise kh'=3; if tkw=1, then kw'=1, otherwise kw'=3.
[0045] S4: If at least one of tkh and tkw is not 1, perform a maximum pooling operation with a pooling window of kh'*kw' and return to step S3; otherwise, proceed to the next step.
[0046] S5: Finally, the step size is used to determine whether to perform a point extraction operation. If the step size height sh > 1 or the width sw > 1, a point extraction operation of sh * sw is performed. If both sh and sw are 1, this operation is not performed.
[0047] Take the maximum pooling with a pooling window of kh*kw of 4*3 and a step size of sh*sw of 2*2 as an example. Figure 3 As shown:
[0048] Initially, kh=tkh=4, kw=tkw=3. Since kh is an even number and kw is an odd number, the size of the first pooling window after splitting is kh'*kw'=2*3. Update tkh=tkh-(kh'-1)=4-(2-1)=3, tkw=tkw-(kw'-1)=3-2=1. Since tkh is not 1, kh'=3, tkw is 1, so kw'=1. Continue the 3*1 pooling operation for the second split. Update tkh=tkh-(kh'-1)=3-2=1, tkw=tkw-(kw'-1)=1. Since tkh and tkw are both 1, kh' and kw' will also be 1, and the pooling splitting operation is completed. Finally, since the step size sh*sw has a step size greater than 1, a sampling operation of sh*sw=2*2 in the height and width directions is added. The final implementation process is as follows Figure 3 shown.
[0049] It can be seen that the maximum pooling with a pooling window height and width of kh*kw and a step size of sh*sw in the height and width directions is equivalent to finding the maximum value in the kh*kw window in the original feature map each time, and then moving the distance sh and sw in the height direction and width direction respectively before the next operation (such as Figure 4As shown). You can ignore the step size and first perform maximum pooling with a pooling window of kh*kw and a step size of sh*sw=1*1. Finally, perform point extraction operations every sh and sw in the height and width directions to achieve the simulation of the step size sh*sw, as shown in Figure 5 As shown in steps 1 and 2 in . The maximum pooling operation in step 1 can be achieved by performing multiple iterations of maximum pooling within 3*3, as shown in Figure 6 As shown, step 1 can be split into step 1.1 and multiple steps 1.2 to achieve it.
[0050] In summary, this application can reduce hardware design logic and implementation complexity. The hardware only needs to implement maximum pooling for simple sizes (<= 3*3), eliminating the need for maximum pooling for sizes greater than 3. The controller can be implemented in software or hardware circuits. Maximum pooling operations are supported for pooling windows of any size.
[0051] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.
Claims
1. A method for performing maximum pooling in a neural network, wherein kh*kw is set as the height and width of the pooling window to achieve the target, and tkh*tkw is set as the height and width remaining after the split pooling window is completed. Initially, tkh=kh, tkw=kw, characterized in that The method comprises: S1, calculate the pooling window size kh'*kw' for the first split; S2, perform a maximum pooling operation with a pooling window size of kh'*kw'; S3, update tkh, tkw and the next pooling window parameter kh'*kw'; S4, if at least one of tkh and tkw is not 1, perform a maximum pooling operation with a pooling window of kh'*kw' and return to step S3; otherwise, proceed to the next step; S5, determine whether to add a point extraction operation based on the step size: If the step height sh>1 or the width sw>1, the sh*sw sampling operation is added; if both sh and sw are 1, this operation is not added.
2. The method for performing maximum pooling in a neural network according to claim 1, wherein: The step S1 comprises: If the pooling window height tkh is 1, the pooling window height kh' after splitting is also 1, otherwise when tkh is an odd number > 1, kh'=3, and when tkh is an even number, kh'=2; If the pooling window width tkw is 1, the pooling window height kw' after the split is also 1. Otherwise, when tkw is an odd number > 1, kw'=3, and when tkw is an even number, kw'=2.
3. The method for performing maximum pooling in a neural network according to claim 1, wherein: The step S3 comprises: tkh=tkh-(kh'-1), tkw=tkw-(kw'-1); if tkh=1, then kh'=1, otherwise kh'=3; if tkw=1, then kw'=1, otherwise kw'=3.
4. A processing device for performing maximum pooling in a neural network, characterized in that Includes controller, pooling unit and sampling unit; The controller controls the pooling unit and the sampling unit based on the processing method according to any one of claims 1 to 3; The pooling unit is a maximum pooling hardware unit that implements a step size of 1*1 and a pooling window of 3*3 or less. The sampling unit performs a sampling extraction operation on the maximum pooling result with a step size of 1 with an interval of sh in height and an interval of sw in width, thereby realizing a maximum pooling operation with a step size of sh*sw.
5. The processing device for performing maximum pooling in a neural network according to claim 4, wherein The pooling unit and the sampling unit interact with the external memory through the data buffer.