Feature image pooling method and apparatus
By using a pixel-attention-based pooling method in a convolutional neural network model to generate attention feature images and perform pooling based on their values, the problem of traditional pooling methods failing to effectively retain target object pixels is solved, thus improving the model training effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN SEICHITECH TECHN CO LTD
- Filing Date
- 2022-11-24
- Publication Date
- 2026-06-02
AI Technical Summary
Existing max pooling and average pooling methods fail to effectively preserve the pixels of the target object during convolutional neural network model training, resulting in reduced model training performance. This is because these methods only consider the size of pixel values and ignore the importance of pixels in the entire image.
A pixel attention-based pooling method is adopted to determine the importance of each pixel by generating an attention feature image, and then perform filtering pooling based on the attention value to retain pixels in the image that are related to the object to be processed and reduce the interference of irrelevant pixels.
It improves the feature compression accuracy of pooling operations, reduces interference from irrelevant pixels, and enhances the training effect of convolutional neural network models.
Smart Images
Figure CN115861766B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of convolutional neural network models, and more particularly to a pooling method and apparatus for feature images. Background Technology
[0002] In recent years, deep learning has flourished in the field of image processing as an emerging technology. Its ability to autonomously learn image data features greatly avoids the tediousness of manually designing algorithms, and it possesses accurate detection performance, high detection efficiency, and good generalization performance for various types of image tasks. This has led to the widespread application of deep learning technology in image processing, including image detection, image classification, and image reconstruction. Pooling operations, as an effective means of feature compression in deep learning, can suppress noise, reduce information redundancy, improve the model's scale invariance and rotation invariance, reduce model computation, and prevent overfitting. Therefore, pooling operations play a crucial role in deep learning. Currently, there are two main types of pooling operations: max pooling and average pooling.
[0003] Max pooling compares the pixel values within the pooling kernel, retaining only the pixel with the largest value as the pooling result, while discarding the rest, and then integrating it into the generated feature. Average pooling averages the target pixels within the pooling kernel, retaining the average pixel value as the pooling result, and then integrating it into the generated feature.
[0004] However, when training convolutional neural network models, we often need to retain as many pixels of the target object in the input feature image as possible during pooling. The pixel value of the target object is not necessarily the largest, nor should it be the average value. Therefore, both max pooling and average pooling only consider the magnitude of the image pixel value, and the pixel value of the input feature image does not accurately reflect the importance of that pixel in the entire image. Traditional pooling operations only process the magnitude of the target pixel value, without considering the importance of each pixel. The magnitude of a pixel value cannot reflect its importance in the entire image or within a specific image region. In the entire image, the pixel containing the target object is an important pixel and should receive more attention and be retained as much as possible during pooling, while background pixels should receive less attention to avoid interfering with the feature extraction of the target object, thus reducing the training effect of the convolutional neural network model. Summary of the Invention
[0005] This application discloses a pooling method and apparatus for feature images, which can be used to improve the training effect of convolutional neural network models.
[0006] Specifically, this application discloses a pixel attention-based pooling method. It utilizes the attention values of each pixel in the input feature image as the basis for pooling operations, enabling the pooling operation to better preserve the more prominent features in the image. This improves upon the traditional pooling method that relies on pixel value size, allowing more pixels related to the object to be processed in the image to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in pooling operations, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model.
[0007] The first aspect of this application provides a pooling method for feature images, comprising:
[0008] Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0009] Obtain the input feature image, which is an image used in the training of the convolutional neural network model;
[0010] The input feature image is input into the pixel attention generation module to generate an attention feature image, which contains the attention values of the pixels in the input feature image.
[0011] The input feature image and the attention feature image are input into the feature pooling module;
[0012] The feature pooling module is used to filter and pool the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data.
[0013] Optionally, the step of inputting the input feature image into the pixel attention generation module to generate an attention feature image includes:
[0014] The pixel attention generation module compresses the feature channels of the input feature image to generate compressed features.
[0015] The pixel attention generation module restores the compressed features into a feature matrix of the same size as the input feature image.
[0016] The pixel attention generation module calculates the attention value for each pixel in the feature matrix to generate an attention feature image.
[0017] Optionally, the pixel attention generation module includes a BatchNorm-Conv-ReLU layer, a BatchNorm-Conv layer, and a SigMoid function layer;
[0018] The step of compressing the input feature image through the pixel attention generation module to generate compressed features includes:
[0019] The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features.
[0020] Optionally, the step of restoring the compressed features into a feature matrix of the same size as the input feature image through the pixel attention generation module includes:
[0021] The compressed features are restored to a feature matrix of the same size as the input feature image by the BatchNorm-Conv layer in the pixel attention generation module.
[0022] Optionally, the step of calculating an attention value for each pixel in the feature matrix and generating an attention feature image using the pixel attention generation module includes:
[0023] The attention feature image is generated by calculating the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module.
[0024] Optionally, the step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes:
[0025] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0026] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0027] Max attention pooling is performed on the at least one set of pixels to be pooled using the set of attention values to generate pooled data.
[0028] Optionally, the step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes:
[0029] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0030] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0031] Attention-weighted pooling is performed on the at least one set of pixels to be pooled using an attention value set to generate pooled data.
[0032] Optionally, the step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes:
[0033] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0034] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0035] The pooling method of the at least one set of pixels to be pooled is determined based on the pixel values of the set of pixels to be pooled. The pooling method includes maximum attention pooling and attention-weighted pooling.
[0036] Pooling data is generated by performing pooling processing on the at least one set of pixels to be pooled using the corresponding pooling method through the attention value set.
[0037] The first aspect of this application provides a pooling apparatus for a feature image, comprising:
[0038] The first acquisition unit is used to acquire a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0039] The second acquisition unit is used to acquire the input feature image, which is an image used in the training of the input convolutional neural network model.
[0040] The first input unit is used to input the input feature image into the pixel attention generation module to generate an attention feature image, wherein the attention feature image contains the attention values of the pixels in the input feature image;
[0041] The second input unit is used to input the input feature image and the attention feature image into the feature pooling module;
[0042] A pooling unit is used to perform pooling processing on the pixels in the input feature image using the feature pooling module and based on the attention values in the attention feature image to generate pooled data.
[0043] Optionally, the first input unit includes:
[0044] The compression module is used to compress the feature channels of the input feature image through the pixel attention generation module to generate compressed features;
[0045] The restoration module is used to restore the compressed features into a feature matrix of the same size as the input feature image through the pixel attention generation module;
[0046] The generation module is used to calculate the attention value for each pixel in the feature matrix through the pixel attention generation module, and generate an attention feature image.
[0047] Optionally, the pixel attention generation module includes a BatchNorm-Conv-ReLU layer, a BatchNorm-Conv layer, and a SigMoid function layer;
[0048] The compression module includes:
[0049] The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features.
[0050] Optionally, the restoration module includes:
[0051] The compressed features are restored to a feature matrix of the same size as the input feature image by the BatchNorm-Conv layer in the pixel attention generation module.
[0052] Optionally, the generation module includes:
[0053] The attention feature image is generated by calculating the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module.
[0054] Optionally, the pooling unit includes:
[0055] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0056] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0057] Max attention pooling is performed on the at least one set of pixels to be pooled using the set of attention values to generate pooled data.
[0058] Optionally, the pooling unit includes:
[0059] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0060] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0061] Attention-weighted pooling is performed on the at least one set of pixels to be pooled using an attention value set to generate pooled data.
[0062] Optionally, the pooling unit includes:
[0063] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0064] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0065] The pooling method of the at least one set of pixels to be pooled is determined based on the pixel values of the set of pixels to be pooled. The pooling method includes maximum attention pooling and attention-weighted pooling.
[0066] Pooling data is generated by performing pooling processing on the at least one set of pixels to be pooled using the corresponding pooling method through the attention value set.
[0067] A third aspect of this application provides an electronic device, comprising:
[0068] Processor, memory, input / output units, and bus;
[0069] The processor is connected to memory, input / output units, and a bus;
[0070] The memory holds the program, and the processor calls the program to execute, as in the first aspect and any optional pooling method of the first aspect.
[0071] The fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the first aspect and any optional pooling method of the first aspect.
[0072] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0073] In this invention, firstly, a pooling unit is obtained. This pooling unit includes a pixel attention generation module and a feature pooling module. The attention generation module can be located inside the convolutional neural network model and used in conjunction with other working layers, or it can be located outside the convolutional neural network model and used independently. Next, an input feature image is obtained. This input feature image is an image input into the convolutional neural network model during training. This input feature image can be a feature output from a certain stage in the convolutional neural network model, which requires convolution. In this case, the input feature image can be first input into the attention generation module, and then the output data can be fed into subsequent convolution stages. The input feature image is then input into the pixel attention generation module to generate an attention feature image, which contains the attention values of the pixels in the input feature image. The input feature image and the attention feature image are then input into the feature pooling module. The feature pooling module is used to perform pooling processing on the pixels in the input feature image based on the attention values in the attention feature image, generating pooled data. In this invention, an attention feature image is generated using an attention generation module. Then, pixels in the input feature image are filtered based on the attention values in the attention feature image to obtain pixels with high attention. Finally, a unique pooling operation is performed on these high-attention pixels. By utilizing the attention values of each pixel in the input feature image as the basis for the pooling operation, the pooling operation can better preserve the more prominent features in the image. This improves upon traditional pooling methods that rely on pixel value size. It allows more pixels related to the object being processed to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in the pooling operation, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model. Attached Figure Description
[0074] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0075] Figure 1 This is a schematic diagram of an embodiment of the pooling method for feature images in this application;
[0076] Figure 2 This is a schematic diagram of another embodiment of the pooling method for feature images in this application;
[0077] Figure 3 This is a schematic diagram of another embodiment of the pooling method for feature images in this application;
[0078] Figure 4 This is a schematic diagram of another embodiment of the pooling method for feature images in this application;
[0079] Figure 5 This is a schematic diagram of an embodiment of the pooling apparatus for the feature image of this application;
[0080] Figure 6 This is a schematic diagram of another embodiment of the pooling apparatus for the feature image of this application;
[0081] Figure 7 This is a schematic diagram of one embodiment of the electronic device of this application. Detailed Implementation
[0082] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0083] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0084] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0085] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0086] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0087] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0088] In existing technologies, max pooling compares the pixel values of target pixels within the pooling kernel, retaining only the pixel with the largest value as the pooling result, while discarding the rest and incorporating them into the generated features. Average pooling averages the target pixels within the pooling kernel, retaining the average value as the pooling result and incorporating it into the generated features. However, when training convolutional neural network models, we often need to retain as many target object pixels as possible in the input feature image during pooling. The pixel values of the target object are not necessarily the largest, nor should they be the average values. Therefore, both max pooling and average pooling only consider the magnitude of image pixel values, and the pixel values of the input feature image do not accurately reflect the importance of a pixel within the entire image. Traditional pooling operations only process the magnitude of target pixel values, without considering the importance of each pixel, and the magnitude of a pixel value cannot reflect the importance of that pixel in the entire image or a specific image region. In the entire image, the pixels containing the target object are important pixels and should be given more attention and preserved as much as possible during the pooling process, while the background pixels should be given less attention to prevent interference with the feature extraction of the target object, thereby reducing the training effect of the convolutional neural network model.
[0089] Based on this, this application discloses a pooling method and apparatus for feature images to improve the training effect of convolutional neural network models.
[0090] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0091] The method described in this application can be applied to servers, devices, terminals, or other devices with logical processing capabilities; therefore, this application does not limit its application. For ease of description, the following description uses a terminal as the executing entity.
[0092] Please see Figure 1 This application provides an embodiment of a pooling method for feature images, comprising:
[0093] 101. Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0094] The terminal obtains a pooling unit, which includes an attention generation module and a feature pooling module. The attention generation module is used to perform attention judgment on the pixels on the input feature image, determine which pixels are more important, and generate a unique attention value for each pixel.
[0095] The feature pooling module is used in conjunction with the attention value output by the attention generation module to perform pooling operations on the pixels in the input feature image.
[0096] 102. Obtain the input feature image, which is an image used in the training of the convolutional neural network model;
[0097] The terminal acquires an input feature image, which is an image used in the training of the convolutional neural network model. It should be noted that the input feature image can be a complete image or simply feature data generated by processing an image in a specific layer of the convolutional neural network model; in this embodiment, it is uniformly referred to as the input feature image.
[0098] 103. Input the input feature image into the pixel attention generation module to generate an attention feature image, wherein the attention feature image contains the attention values of the pixels in the input feature image;
[0099] The terminal inputs the input feature image into the attention generation module to generate an attention feature image, which contains the attention values of the pixels in the input feature image.
[0100] Deep learning's attention mechanism is a biomimetic of human visual attention, essentially a resource allocation mechanism. The physiological principle is that human visual attention can receive high-resolution signals from a specific area of an image while perceiving its surrounding areas at lower resolution, and the viewpoint can change over time. In other words, the human eye quickly scans the entire image, finds the target area requiring attention, and then allocates more attention to that area to acquire more detailed information and suppress other useless information. This improves the efficiency of convolutional unit representation.
[0101] In convolutional neural network models, the attention mechanism can be considered a resource allocation mechanism. It can be understood as redistributing resources that were originally evenly distributed according to the importance of the attention objects. Important units receive more resources, while unimportant or less important units receive less. In the structural design of deep neural networks, the resources that attention needs to allocate are basically weights.
[0102] 104. Input the input feature image and the attention feature image into the feature pooling module;
[0103] The terminal inputs the input feature image and the attention feature image into the feature pooling module.
[0104] 105. Using the feature pooling module and based on the attention values in the attention feature image, perform filtering and pooling processing on the pixels in the input feature image to generate pooled data.
[0105] The terminal uses the feature pooling module and performs pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data.
[0106] In this embodiment, the pooling process involves traversing the input feature map from top to bottom and from left to right using a pooling kernel sliding window (e.g., 2×2). After each movement, attention values are taken from all pixels (e.g., 2×2) in the pooling kernel window and pooled. These values are then used as elements of the output feature map. After the traversal is complete, a new feature map with compressed dimensions is obtained. The new feature map after pooling is then transmitted to the next layer of the neural network.
[0107] The traditional pooling process involves using a sliding window (e.g., 2×2) of the pooling kernel to traverse the input feature map from top to bottom and from left to right. After each movement, the average (average pooling) or maximum (max pooling) of all pixels (e.g., 2×2) in the pooling kernel window is taken, and this value is used as an element of the output feature map. After the traversal is completed, a new feature map with compressed dimensions is obtained, and then the new feature map after the pooling operation is passed to the next layer of the neural network.
[0108] In this embodiment, firstly, the terminal acquires a pooling unit, which includes a pixel attention generation module and a feature pooling module. The attention generation module can be located inside the convolutional neural network model and used in conjunction with other working layers, or it can be located outside the convolutional neural network model and used independently. The terminal acquires an input feature image, which is an image input into the convolutional neural network model during training. This input feature image can be a feature output from a certain stage in the convolutional neural network model, which requires convolution. In this case, the input feature image can be first input into the attention generation module, and then the output data can be fed into subsequent convolution stages. The input feature image is input into the pixel attention generation module to generate an attention feature image, which contains the attention values of the pixels in the input feature image. The input feature image and the attention feature image are then input into the feature pooling module. The feature pooling module uses the attention values in the attention feature image to perform pooling processing on the pixels in the input feature image, generating pooled data. In this invention, an attention feature image is generated using an attention generation module. Then, pixels in the input feature image are filtered based on the attention values in the attention feature image to obtain pixels with high attention. Finally, a unique pooling operation is performed on these high-attention pixels. By utilizing the attention values of each pixel in the input feature image as the basis for the pooling operation, the pooling operation can better preserve the more prominent features in the image. This improves upon traditional pooling methods that rely on pixel value size. It allows more pixels related to the object being processed to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in the pooling operation, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model.
[0109] Please see Figure 2 This application provides an embodiment of a pooling method for feature images, comprising:
[0110] 201. Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0111] 202. Obtain the input feature image, which is an image used in the training of the convolutional neural network model;
[0112] Steps 201 to 202 in this embodiment are similar to steps 101 to 102 in the previous embodiment, and will not be repeated here.
[0113] 203. The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features;
[0114] 204. The compressed features are restored to a feature matrix of the same size as the input feature image through the BatchNorm-Conv layer in the pixel attention generation module;
[0115] 205. Calculate the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module to generate an attention feature image;
[0116] The terminal compresses the feature channels of the input feature image through the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features, and then restores the compressed features into a feature matrix of the same size as the input feature image through the BatchNorm-Conv layer in the pixel attention generation module. Finally, the terminal calculates the attention value for each pixel in the feature matrix through the SigMoid function in the pixel attention generation module to generate an attention feature image.
[0117] Specifically, in this embodiment, the terminal first needs to use the BatchNorm-3 function in the pixel attention generation module. The 3rd Conv-ReLU layer compresses the feature channels of the input feature image to generate compressed features, and then passes them through the BatchNorm-3 layer in the pixel attention generation module. The 3Conv layer restores the compressed features into a feature matrix of the same size as the input feature image. Finally, the terminal calculates the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module, generating an attention feature image. (BatchNorm-3) A 3-Conv-ReLU layer is the optimal size. Because of the differences in pixels within the input feature image, BatchNorm-1 cannot be used. One Conv-ReLU layer is needed; otherwise, it would increase the workload of convolution.
[0118] 206. Input the input feature image and the attention feature image into the feature pooling module;
[0119] Step 206 in this embodiment is similar to step 104 in the previous embodiment, and will not be repeated here.
[0120] 207. Use the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image;
[0121] 208. Determine the set of attention values for each pixel in the at least one set of pixels to be pooled using the attention feature image;
[0122] 209. Perform maximum attention pooling on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0123] The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled on the input feature image. The terminal uses the attention feature image to determine the attention value set of each pixel in the at least one set of pixels to be pooled. The terminal then performs maximum attention pooling on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0124] For example: the terminal uses the pooling window on the feature pooling module to determine a set of pixels to be pooled in the input feature image, which is 3. Given a set of 3 pixels to be pooled, the attention value of each pixel in this set of pixels is determined based on the attention feature image. The pixel with the highest attention value is selected as the optimal pixel for subsequent pooling operations.
[0125] In this embodiment, firstly, a pooling unit is obtained. This pooling unit includes a pixel attention generation module and a feature pooling module. The attention generation module can be located inside the convolutional neural network model and used in conjunction with other working layers, or it can be located outside the convolutional neural network model and used independently. Next, an input feature image is obtained. This input feature image is an image input into the convolutional neural network model during training. This input feature image can be a feature output from a certain stage in the convolutional neural network model, which requires convolution. In this case, the input feature image can be first input into the attention generation module, and then the output data can be fed into subsequent convolution stages. The input feature image is compressed using the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features. The terminal uses the BatchNorm-Conv layer in the pixel attention generation module to restore the compressed features into a feature matrix of the same size as the input feature image. The terminal uses the SigMoid function in the pixel attention generation module to calculate the attention value for each pixel in the feature matrix, generating an attention feature image. Finally, the input feature image and the attention feature image are input into the feature pooling module. The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image. The terminal uses the attention feature image to determine the attention value set for each pixel in the at least one set of pixels to be pooled. The terminal performs max attention pooling on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0126] In this invention, an attention feature image is generated using an attention generation module. Then, pixels in the input feature image are filtered based on the attention values in the attention feature image to obtain pixels with high attention. Finally, a unique pooling operation is performed on these high-attention pixels. By utilizing the attention values of each pixel in the input feature image as the basis for the pooling operation, the pooling operation can better preserve the more prominent features in the image. This improves upon traditional pooling methods that rely on pixel value size. It allows more pixels related to the object being processed to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in the pooling operation, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model.
[0127] Secondly, this embodiment improves upon the shortcomings of traditional pooling kernels that operate solely based on pixel values. Before pooling, attention is calculated on the input feature map to obtain attention feature images for each pixel. These attention feature images are the same size as the original input feature image, and each element represents the attention value of a pixel at a corresponding position in the input feature image—that is, the importance of that pixel within the entire input feature image. The attention value is a number between 0 and 1; a larger value indicates higher importance. During pooling, the pooling operation is performed based on the attention value of each pixel. This embodiment proposes max attention pooling.
[0128] During the sliding window movement, the attention pooling kernel selects the pixel value with the largest attention value (maximum attention pooling) based on the attention corresponding to the pixels in the pooling window, and uses it as the feature value of the output feature. This allows for better focus on more important pixel values and reduces interference from irrelevant pixels.
[0129] Please see Figure 3 This application provides an embodiment of a pooling method for feature images, comprising:
[0130] 301. Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0131] 302. Obtain the input feature image, wherein the input feature image is an image used in the training of the convolutional neural network model;
[0132] 303. The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features;
[0133] 304. The compressed features are restored to a feature matrix of the same size as the input feature image by the BatchNorm-Conv layer in the pixel attention generation module;
[0134] 305. Calculate the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module to generate an attention feature image;
[0135] 306. Input the input feature image and the attention feature image into the feature pooling module;
[0136] Steps 301 to 306 in this embodiment are similar to steps 201 to 206 in the previous embodiment, and will not be repeated here.
[0137] 307. Use the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image;
[0138] 308. Determine the set of attention values for each pixel in the at least one set of pixels to be pooled using the attention feature image;
[0139] 309. Attention-weighted pooling is performed on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0140] The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled on the input feature image. The terminal uses the attention feature image to determine the attention value set of each pixel in the at least one set of pixels to be pooled. The terminal uses the attention value set to perform attention-weighted pooling processing on the at least one set of pixels to be pooled to generate pooled data.
[0141] For example: the terminal uses the pooling window on the feature pooling module to determine a set of pixels to be pooled in the input feature image, which is 3. Given a set of 3 pixels to be pooled, the attention value of each pixel in this set is determined based on the attention feature image. The N pixel values in the set are then summed using the attention value as a coefficient to obtain the optimal pixel for subsequent pooling operations. Here, N is an integer greater than 1 and less than or equal to 9.
[0142] In this embodiment, firstly, a pooling unit is obtained. This pooling unit includes a pixel attention generation module and a feature pooling module. The attention generation module can be located inside the convolutional neural network model and used in conjunction with other working layers, or it can be located outside the convolutional neural network model and used independently. Next, an input feature image is obtained. This input feature image is an image input into the convolutional neural network model during training. This input feature image can be a feature output from a certain stage in the convolutional neural network model, which requires convolution. In this case, the input feature image can be first input into the attention generation module, and then the output data can be fed into subsequent convolution stages. The input feature image is compressed using the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features. The terminal uses the BatchNorm-Conv layer in the pixel attention generation module to restore the compressed features into a feature matrix of the same size as the input feature image. The terminal uses the SigMoid function in the pixel attention generation module to calculate the attention value for each pixel in the feature matrix, generating an attention feature image. Finally, the input feature image and the attention feature image are input into the feature pooling module. The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image. The terminal uses the attention feature image to determine the attention value set for each pixel in the at least one set of pixels to be pooled. The terminal performs max attention pooling on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0143] In this invention, an attention feature image is generated using an attention generation module. Then, pixels in the input feature image are filtered based on the attention values in the attention feature image to obtain pixels with high attention. Finally, a unique pooling operation is performed on these high-attention pixels. By utilizing the attention values of each pixel in the input feature image as the basis for the pooling operation, the pooling operation can better preserve the more prominent features in the image. This improves upon traditional pooling methods that rely on pixel value size. It allows more pixels related to the object being processed to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in the pooling operation, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model.
[0144] Secondly, this embodiment improves upon the shortcomings of traditional pooling kernels that operate solely based on pixel values. Before pooling, attention is calculated on the input feature map to obtain attention feature images for each pixel. These attention feature images are the same size as the original input feature image, and each element represents the attention value of a pixel at a corresponding position in the input feature image—that is, the importance of that pixel within the entire input feature image. The attention value is a number between 0 and 1; a larger value indicates higher importance. During pooling, the pooling operation is performed based on the attention value of each pixel; this embodiment proposes attention-weighted pooling.
[0145] During the sliding window movement, the attention pooling kernel calculates the weighted average of each pixel based on the attention value corresponding to the pixel in the pooling window (attention-weighted pooling), and uses this average as the feature value of the output feature. This allows for better focus on more important pixel values and reduces interference from irrelevant pixels.
[0146] Please see Figure 4 This application provides an embodiment of a pooling method for feature images, comprising:
[0147] 401. Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0148] 402. Obtain the input feature image, wherein the input feature image is an image used in the training of the convolutional neural network model;
[0149] 403. The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features;
[0150] 404. The compressed features are restored to a feature matrix of the same size as the input feature image by the BatchNorm-Conv layer in the pixel attention generation module;
[0151] 405. Calculate the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module to generate an attention feature image;
[0152] 406. Input the input feature image and the attention feature image into the feature pooling module;
[0153] Steps 401 to 406 in this embodiment are similar to steps 201 to 206 in the previous embodiment, and will not be repeated here.
[0154] 407. Use the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image;
[0155] 408. Determine the attention value set of each pixel in the at least one set of pixels to be pooled using the attention feature image;
[0156] 409. Determine the pooling method of the at least one set of pixels to be pooled based on the pixel values of the set of pixels to be pooled, wherein the pooling method includes maximum attention pooling and attention-weighted pooling.
[0157] 410. Perform pooling processing on the at least one set of pixels to be pooled using the corresponding pooling method through the attention value set to generate pooled data.
[0158] The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image, determines the attention value set of each pixel in the at least one set of pixels to be pooled through the attention feature image, determines the pooling method of the at least one set of pixels to be pooled based on the attention value set, the pooling method includes max attention pooling and attention weighted pooling, and performs pooling processing on the at least one set of pixels to be pooled according to the corresponding pooling method through the attention value set to generate pooled data.
[0159] For example: the terminal uses the pooling window on the feature pooling module to determine a set of pixels to be pooled in the input feature image, which is 3. The set of 3 pixels to be pooled is then used to determine the difference in the current attention. If the difference is 3... If the pixel values of the pixels in the set of pixels to be pooled are almost identical, then attention-weighted pooling is chosen as the pooling method. If the set of pixels to be pooled contains 3 pixels, then attention-weighted pooling is chosen as the pooling method. If the pixel values of the pixels in the set of pixels to be pooled (3) become polarized, then maximum attention pooling is determined as the pooling method.
[0160] In this embodiment, firstly, a pooling unit is obtained. This pooling unit includes a pixel attention generation module and a feature pooling module. The attention generation module can be located inside the convolutional neural network model and used in conjunction with other working layers, or it can be located outside the convolutional neural network model and used independently. Next, an input feature image is obtained. This input feature image is an image input into the convolutional neural network model during training. This input feature image can be a feature output from a certain stage in the convolutional neural network model, which requires convolution. In this case, the input feature image can be first input into the attention generation module, and then the output data can be fed into subsequent convolution stages. The input feature image is compressed using the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features. The terminal uses the BatchNorm-Conv layer in the pixel attention generation module to restore the compressed features into a feature matrix of the same size as the input feature image. The terminal uses the SigMoid function in the pixel attention generation module to calculate the attention value for each pixel in the feature matrix, generating an attention feature image. Finally, the input feature image and the attention feature image are input into the feature pooling module. The terminal uses the pooling window on the feature pooling module to determine at least one set of pixels to be pooled in the input feature image. The terminal uses the attention feature image to determine the attention value set for each pixel in the at least one set of pixels to be pooled. The terminal performs max attention pooling on the at least one set of pixels to be pooled using the attention value set to generate pooled data.
[0161] In this invention, an attention feature image is generated using an attention generation module. Then, pixels in the input feature image are filtered based on the attention values in the attention feature image to obtain pixels with high attention. Finally, a unique pooling operation is performed on these high-attention pixels. By utilizing the attention values of each pixel in the input feature image as the basis for the pooling operation, the pooling operation can better preserve the more prominent features in the image. This improves upon traditional pooling methods that rely on pixel value size. It allows more pixels related to the object being processed to be passed to the next layer of the neural network through the pooling operation, while retaining fewer irrelevant pixels. This improves the accuracy of feature compression in the pooling operation, reduces interference from irrelevant pixels, and ultimately enhances the training effect of the convolutional neural network model.
[0162] Secondly, this embodiment improves upon the shortcomings of traditional pooling kernels that operate solely based on pixel values. Before pooling, attention is calculated on the input feature map to obtain attention feature images for each pixel. These attention feature images are the same size as the original input feature image, and each element represents the attention value of a pixel at a corresponding position in the input feature image—that is, the importance of that pixel within the entire input feature image. Attention values are numbers between 0 and 1; the larger the value, the higher the importance. During pooling, the pooling operation is performed based on the attention values of each pixel. This embodiment proposes first determining the pooling method.
[0163] By analyzing the pixel values of different sets of pixels to be pooled, we can determine which method is suitable for each set and then perform pooling. The pooling method can be different for each set of pixels to be pooled, which can further improve the accuracy of feature compression in pooling operations, reduce interference from irrelevant pixels, and thus improve the training effect of convolutional neural network models.
[0164] Please see Figure 5 This application provides an embodiment of a pooling apparatus for a feature image, comprising:
[0165] The first acquisition unit 501 is used to acquire a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0166] The second acquisition unit 502 is used to acquire an input feature image, wherein the input feature image is an image used in training the convolutional neural network model.
[0167] The first input unit 503 is used to input the input feature image into the pixel attention generation module to generate an attention feature image, wherein the attention feature image has the attention values of the pixels in the input feature image;
[0168] The second input unit 504 is used to input the input feature image and the attention feature image into the feature pooling module;
[0169] Pooling unit 505 is used to perform pooling processing on the pixels in the input feature image using the feature pooling module and based on the attention value in the attention feature image to generate pooled data.
[0170] Please see Figure 6 This application provides an embodiment of a pooling apparatus for a feature image, comprising:
[0171] The first acquisition unit 601 is used to acquire a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module;
[0172] The second acquisition unit 602 is used to acquire an input feature image, wherein the input feature image is an image used in training the convolutional neural network model.
[0173] The first input unit 603 is used to input the input feature image into the pixel attention generation module to generate an attention feature image, wherein the attention feature image has the attention values of the pixels in the input feature image;
[0174] Optionally, the first input unit 603 includes:
[0175] Compression module 6031 is used to compress the feature channels of the input feature image through the pixel attention generation module to generate compressed features;
[0176] The restoration module 6032 is used to restore the compressed features into a feature matrix of the same size as the input feature image through the pixel attention generation module;
[0177] The generation module 6033 is used to calculate the attention value for each pixel in the feature matrix through the pixel attention generation module, and generate an attention feature image.
[0178] Optionally, the pixel attention generation module includes a BatchNorm-Conv-ReLU layer, a BatchNorm-Conv layer, and a SigMoid function layer;
[0179] The compression module 6031 includes:
[0180] The input feature image is compressed by the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features.
[0181] Optionally, the restoration module 6032 includes:
[0182] The compressed features are restored to a feature matrix of the same size as the input feature image by the BatchNorm-Conv layer in the pixel attention generation module.
[0183] Optionally, the generation module 6033 includes:
[0184] The attention feature image is generated by calculating the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module.
[0185] The second input unit 604 is used to input the input feature image and the attention feature image into the feature pooling module;
[0186] Pooling unit 605 is used to perform pooling processing on the pixels in the input feature image using the feature pooling module and based on the attention value in the attention feature image to generate pooled data.
[0187] Optionally, the pooling unit 605 includes:
[0188] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0189] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0190] Max attention pooling is performed on the at least one set of pixels to be pooled using the set of attention values to generate pooled data.
[0191] Optionally, the pooling unit 605 includes:
[0192] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0193] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0194] Attention-weighted pooling is performed on the at least one set of pixels to be pooled using an attention value set to generate pooled data.
[0195] Optionally, the pooling unit 605 includes:
[0196] The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image;
[0197] The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image;
[0198] The pooling method of the at least one set of pixels to be pooled is determined based on the pixel values of the set of pixels to be pooled. The pooling method includes maximum attention pooling and attention-weighted pooling.
[0199] Pooling data is generated by performing pooling processing on the at least one set of pixels to be pooled using the corresponding pooling method through the attention value set.
[0200] Please see Figure 7 This application provides an electronic device, including:
[0201] Processor 701, memory 702, input / output unit 703, and bus 704.
[0202] The processor 701 is connected to the memory 702, the input / output unit 703, and the bus 704.
[0203] The memory 702 stores a program, and the processor 701 calls the program to execute it, such as... Figure 1 , Figure 2 , Figure 3 and Figure 4 Pooling methods in [the context of the game].
[0204] This application provides a computer-readable storage medium on which a program is stored, and when the program is executed on a computer, it performs the following... Figure 1 , Figure 2 , Figure 3 and Figure 4 Pooling methods in [the context of the game].
[0205] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0206] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.
[0207] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0208] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0209] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
Claims
1. A pooling method for feature images, characterized in that, include: Obtain a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module; Obtain the input feature image, which is an image used in the training of the convolutional neural network model; The input feature image is input into the pixel attention generation module to generate an attention feature image, which contains the attention values of the pixels in the input feature image. The step of inputting the input feature image into the pixel attention generation module to generate an attention feature image includes: compressing the feature channels of the input feature image through the pixel attention generation module to generate compressed features; restoring the compressed features into a feature matrix of the same size as the input feature image through the pixel attention generation module; and calculating an attention value for each pixel in the feature matrix through the pixel attention generation module to generate an attention feature image. The pixel attention generation module includes a BatchNorm-Conv-ReLU layer, a BatchNorm-Conv layer, and a SigMoid function layer, wherein the BatchNorm-Conv-ReLU layer is specifically BatchNorm-3. The 3Conv-ReLU layer; the step of compressing the input feature image through the pixel attention generation module to generate compressed features includes: compressing the input feature image through the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features; the step of restoring the compressed features into a feature matrix of the same size as the input feature image through the pixel attention generation module includes: restoring the compressed features into a feature matrix of the same size as the input feature image through the BatchNorm-Conv layer in the pixel attention generation module; the step of calculating the attention value for each pixel in the feature matrix through the pixel attention generation module to generate an attention feature image includes: calculating the attention value for each pixel in the feature matrix through the SigMoid function in the pixel attention generation module to generate an attention feature image; The input feature image and the attention feature image are input into the feature pooling module; The feature pooling module is used to filter and pool the pixels in the input feature image according to the attention value in the attention feature image to generate pooled data. The pooling process is to traverse the input feature image from top to bottom and from left to right through a pooling kernel sliding window. After each movement, the attention value of all pixels in the pooling kernel window is taken and pooled. This value is used as an element of the output feature image. After the traversal is completed, a new feature image with compressed length and width is obtained. Then, the new feature image after the pooling operation is transmitted to the next layer of the neural network.
2. The pooling method according to claim 1, characterized in that, The step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes: The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image; The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image; Max attention pooling is performed on the at least one set of pixels to be pooled using the set of attention values to generate pooled data.
3. The pooling method according to claim 1, characterized in that, The step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes: The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image; The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image; Attention-weighted pooling is performed on the at least one set of pixels to be pooled using an attention value set to generate pooled data.
4. The pooling method according to claim 1, characterized in that, The step of using the feature pooling module and performing filter pooling processing on the pixels in the input feature image based on the attention values in the attention feature image to generate pooled data includes: The pooling window on the feature pooling module is used to determine at least one set of pixels to be pooled in the input feature image; The attention value set of each pixel in the at least one set of pixels to be pooled is determined by the attention feature image; The pooling method of the at least one set of pixels to be pooled is determined based on the pixel values of the set of pixels to be pooled. The pooling method includes maximum attention pooling and attention-weighted pooling. Pooling data is generated by performing pooling processing on the at least one set of pixels to be pooled using the corresponding pooling method through the attention value set.
5. A pooling apparatus for a feature image, characterized in that, include: The first acquisition unit is used to acquire a pooling unit, wherein the pooling unit includes a pixel attention generation module and a feature pooling module; The second acquisition unit is used to acquire the input feature image, which is an image used in the training of the input convolutional neural network model. The first input unit is used to input the input feature image into the pixel attention generation module to generate an attention feature image, wherein the attention feature image contains the attention values of the pixels in the input feature image; The first input unit includes: a compression module, used to compress the input feature image through the pixel attention generation module to generate compressed features; a restoration module, used to restore the compressed features to a feature matrix of the same size as the input feature image through the pixel attention generation module; and a generation module, used to calculate an attention value for each pixel in the feature matrix through the pixel attention generation module to generate an attention feature image. The pixel attention generation module includes a BatchNorm-Conv-ReLU layer, a BatchNorm-Conv layer, and a SigMoid function layer; the compression module includes: compressing the input feature image through the BatchNorm-Conv-ReLU layer in the pixel attention generation module to generate compressed features; the restoration module includes: restoring the compressed features into a feature matrix of the same size as the input feature image through the BatchNorm-Conv layer in the pixel attention generation module; the generation module includes: calculating the attention value for each pixel in the feature matrix using the SigMoid function in the pixel attention generation module to generate an attention feature image; The second input unit is used to input the input feature image and the attention feature image into the feature pooling module; The pooling unit is used to perform pooling processing on the pixels in the input feature image using the feature pooling module and the attention value in the attention feature image to generate pooled data. The pooling process is to traverse the input feature image from top to bottom and from left to right through a pooling kernel sliding window. After each movement, the attention value of all pixels in the pooling kernel window is taken and pooled. This value is used as an element of the output feature image. After the traversal is completed, a new feature image with compressed length and width is obtained. Then, the new feature image after the pooling operation is transmitted to the next layer of the neural network.