Method for detecting occluded and blurred faces in complex scenes

Abstract

The application provides a kind of complex scene under the method for detecting occlusion and fuzzy face, it is related to near view face target identification and detection technical field.The application is repaired to the face image of occlusion and fuzzy by using the GFP-GAN of generating face priori, then the face detection of face image after repair is carried out using improved MTCNN.Improved MTCNN introduces depth separable convolution in model network, so that the amount of calculation is reduced, the model detection time is reduced, and the model detection speed is accelerated;Improved residual attention module combined with Involution convolution is introduced in the R-Net network of the model, the accuracy of face detection is improved through the module, while reducing the size and amount of calculation of the model, the detection accuracy is ensured;Finally, the Dropout layer is introduced at the full connection layer of the network, the problem of overfitting is alleviated, to a certain extent, the effect of regularization is achieved, a kind of weighted post-processing algorithm Confluence is used to screen the face detection frame generated by O-Net network, and the accuracy of face detection frame is improved.

Description

Technical Field

[0001] This invention belongs to the field of close-range face target recognition and detection technology, and particularly relates to a method for detecting occluded and blurred faces in complex scenes. Background Technology

[0002] The deep application of facial detection technology is indispensable in today's smart cities. Facial detection technology is practical in areas such as tracking the movement of people and verifying personal identity, and is therefore being used in an increasing number of scenarios. For example, it plays a crucial role in public security and epidemic prevention, as well as online smart finance, hotel rentals, e-commerce, and WeChat and Alipay facial recognition payments. However, in some complex scenarios, factors such as facial occlusion, differences in appearance and skin color, low lighting intensity, large variations in facial expressions, and low facial resolution make accurate facial detection a significant challenge. For instance, when using WeChat or Alipay for payments, factors such as low lighting intensity, wearing scarves, or masks can make accurate facial detection difficult in situations requiring facial recognition.

[0003] In close-range facial recognition and detection environments, the detection of occluded faces is essential. For example, during the pandemic, people are required to wear masks in various situations, especially in crowded places like train stations. However, in the context of normalized pandemic control, masks pose challenges to facial recognition authentication scenarios, resulting in time and efficiency losses, as well as the risk of infection. In outdoor scenarios, changes in lighting, angle, and facial expressions under natural conditions, as well as the presence of occlusions, can prevent face capture or result in the loss of facial features in the captured images, hindering accurate identity recognition. Therefore, accurate face detection is crucial in various environments. Existing methods suffer from false positives or false negatives in complex scenes involving occluded and blurred faces. How to improve the detection rate and ensure accuracy in various complex scenes with occlusion has become a hot research topic. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method for detecting occluded and blurred faces in complex scenes, comprising the following steps:

[0005] Step 1: Input a face image containing occlusion and blurring, and input the face image into a generative adversarial network (GFP-GAN) to generate face priors;

[0006] Step 2: Perform face restoration on the face image containing occlusions and blurriness, and output the restored face image;

[0007] Step 2.1: Extract latent features from face images using the de-degradation module of the GFP-GAN network. and multi-resolution spatial features ;

[0008] Step 2.2: Extract the latent features from Step 2.1 Latent code mapped to the latent feature space of pre-trained GANs (Generative Adversarial Networks) Up, then the potential code GAN features at multiple resolution scales are generated by pre-training each convolutional layer in the GAN. ;

[0009] Step 2.3: Multi-resolution spatial features extracted in Step 2.1 For some GAN features Spatial modulation is performed, and a restored face image is generated through a channel segmentation-based spatial feature transformation (CS-SFT) layer at multiple resolution scales. ;

[0010] Step 3: Scale the repaired face image from Step 2 using a certain scaling factor to obtain images of different sizes;

[0011] Step 4: Improve the multi-task cascaded convolutional network (MTCNN), which consists of an improved first network P-Net (Proposal Network), an improved second network R-Net (Refine Network), and an improved third network O-Net (Output Network);

[0012] The first improved network, P-Net, uses depthwise separable convolutions instead of ordinary convolutions;

[0013] The improved second network, R-Net, first replaces ordinary convolutional layers with depthwise separable convolutions, then introduces residual blocks into the network and fuses them with involutional convolutions, and finally combines the residual blocks integrated with involutional convolutions with channel attention modules.

[0014] The residual block uses a 1×1 convolution with dimensionality reduction to reduce computational cost, and then restores the original data through a 1×1 convolution operation.

[0015] Assuming the input feature map is set as X is the input vector, H is the height of the input feature map, and W is the width of the input feature map. To determine the number of channels in the input feature map, the Involution convolution first extracts the input feature map. pixels at Its Involution kernel is R represents the domain. The value is the filter size, and 1 represents the number of shared channels, generated by the kernel function. Generate G Involution kernel size ,in, The function is G is the value shared by each group of channels. The number of groups, G Involution kernel size The current pixel in the input feature map is compared across C channels. Multiply the neighboring pixels one by one to get The three-dimensional matrix is ​​then used to sum the width and height dimensions while keeping the channel dimension unchanged, resulting in a new feature map. A vector of size, ultimately outputting a feature map. .

[0016] The residual attention module incorporating involution convolution first transforms a residual structure combined with involution convolution to obtain a feature map u with C feature channels, W width, and H height. This is then compressed (squeeze) and transformed into 1×1×C using global pooling, effectively setting the width and height of the feature map to 1, resulting in a feature map of size 1×1×C. After obtaining the transformed 1×1×C feature map, an excitation operation is performed, first by adding a fully connected layer (Fully Connected). A fully connected (FC) layer with a dimensionality reduction coefficient of hyperparameter r (SERadio) is performed to obtain a 1×1×C×r feature map. Then, a ReLU activation layer is added, followed by a fully connected (FC) layer for restoration, which restores the feature map to 1×1×C. This operation predicts the weights of each channel and applies the different channel weights to the corresponding channels of the previous feature map. Finally, the output weights of the previous operation are multiplied and weighted (scaled) with the previous input features in channels, resulting in a final output feature map of size H×W×C.

[0017] The improved third network, O-Net, replaces ordinary convolutions with depthwise separable convolutions, introduces Dropout layers at the fully connected layers of the network, and uses Confluence instead of NMS to filter face detection boxes.

[0018] Step 5: Input the images of different sizes from Step 3 into the improved MTCNN for face detection;

[0019] Step 5.1: Input images of different sizes into P-Net, the first improved network in MTCNN, and output the detected face image P1;

[0020] Step 5.1.1: Use a 12×12 detection box to perform face detection on images of different sizes, generating a large number of face detection boxes on the image;

[0021] Step 5.1.2: Use non-maximum suppression (NMS) to filter the face detection bounding boxes and output the detected face image P1;

[0022] Step 5.2: Resize the face image P1 output in Step 5.1.2 and the images of different sizes in Step 3, so that the image size becomes 24×24;

[0023] Step 5.3: Input the 24×24 image obtained in step 5.2 into the improved second network R-Net in MTCNN, and output the detected face image P2;

[0024] Step 5.3.1: Perform face detection on the input image to obtain a large number of face detection boxes;

[0025] Step 5.3.2: Use NMS to filter the face detection bounding boxes in the detection area and output the detected face image P2;

[0026] Step 5.4: Perform a resize operation on the face image P2 output in step 5.3.2 and the images of different sizes in step 3, so that the image size becomes 64×64;

[0027] Step 5.5: Input the 64×64 image obtained in step 5.4 into the improved third network O-Net in MTCNN, and output the detected face image P3;

[0028] Step 5.5.1: Perform face detection on the input image to obtain a large number of face detection boxes;

[0029] Step 5.5.2: Use the weighted post-processing algorithm Confluence to replace NMS to filter the face detection boxes in the detection area, and output the detected face image P3.

[0030] The beneficial effects of using the above-mentioned technical methods are as follows:

[0031] This invention provides a method for detecting occlusion and blurred faces in complex scenes, which has the following beneficial effects:

[0032] (1) The occlusion and blurry face detection method in complex scenarios of the present invention provides basic technical support for identity recognition and authentication of occluded and blurry faces in complex scenarios. The improved multi-task convolutional neural network with high efficiency and high detection rate of occluded and blurry faces has good applicability in complex scenarios of smart cities today. The model has practical value of being time-saving and highly accurate.

[0033] (2) Generative adversarial networks with face priors are used to repair occluded and blurred face images in complex scenes, making the face images to be detected clearer. For occluded or blurred faces, the facial features are made clearer. Then, an improved multi-task convolutional neural network is used to detect the repaired occluded and blurred face images, which improves the face detection rate to a certain extent. Compared with directly using the improved multi-task convolutional neural network, this method can further improve the detection rate of occluded and blurred faces in complex scenes containing masks, helmets and randomly occluded faces. Attached Figure Description

[0034] Figure 1 This is a flowchart of the occlusion and blurred face detection method in complex scenes according to a specific embodiment of the present invention;

[0035] Figure 2 This is a structural diagram of a generative adversarial network used for face restoration in a specific embodiment of the present invention to generate facial priors.

[0036] Figure 3 This is an improved structure of the P-Net network in the improved MTCNN according to a specific embodiment of the present invention;

[0037] Figure 4 This is an improved structure of the R-Net network in the improved MTCNN according to a specific embodiment of the present invention;

[0038] Figure 5 This is a diagram of the residual block structure introduced in the R-Net network in a specific embodiment of the present invention;

[0039] Figure 6 This is a structural diagram of the Involution convolution fused with residual blocks in the R-Net network in a specific embodiment of the present invention;

[0040] Figure 7 This is a structural diagram of the improved residual attention module incorporating Involution convolution in a specific embodiment of the present invention.

[0041] Figure 8 This is an improved structure of the O-Net network in the improved MTCNN according to a specific embodiment of the present invention;

[0042] Figure 9 This is a comparison diagram before and after adding a Dropout layer to the network in a specific embodiment of the present invention;

[0043] Figure 10 This is a diagram showing the proximity calculation of two bounding boxes with coordinates in a specific embodiment of the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0045] A method for detecting occlusion and blurred faces in complex scenes, such as Figure 1 As shown, it includes the following steps:

[0046] Step 1: Input a face image containing occlusion and blurring, and input the face image into a generative adversarial network (GFP-GAN) to generate face priors;

[0047] In this embodiment, the network structure of the generative adversarial network used to generate face priors is as follows: Figure 2 As shown, the network consists of a de-degradation module, a pre-trained GAN (StyleGAN2) as a prior, and then connected using a channel segmentation-based spatial feature transformation (CS-SFT). Assume the input image... This is a facial image affected by factors such as occlusion, blurring, and noise. The facial image is first repaired, and then a high-quality image is output. Output face image In terms of realism and fidelity, it is comparable to real images. Make them as similar as possible. Then compare the images. Performing face detection can yield better face detection results.

[0048] Step 2: Perform face restoration on the face image containing occlusions and blurriness, and output the restored face image;

[0049] Step 2.1: Extract latent features from face images using the de-degradation module of the GFP-GAN network. and multi-resolution spatial features ;

[0050] Step 2.2: Extract the latent features from Step 2.1 Latent codes mapped to the latent feature space of a pre-trained GAN (Generative Adversarial Networks) above, and then the latent codes are passed through each convolutional layer in the pre-trained GAN to generate GAN features at multiple resolution scales ;

[0051] Step 2.3: The multi-resolution spatial features extracted in Step 2.1 are used to spatially modulate some of the GAN features and generate a restored face image through a channel-split based spatial feature transformation (CS-SFT) layer at multiple resolution scales ;

[0052] Step 3: Scale the restored face image in Step 2 with a certain scaling factor to obtain images of different sizes;

[0053] Step 4: Improve the multi-task cascaded convolutional networks (MTCNN), which consists of an improved first network P-Net (Proposal Network), an improved second network R-Net (Refine Network), and an improved third network O-Net (Output Network);

[0054] In this embodiment, the training datasets for the three networks of the improved MTCNN are the Widerface dataset, and the dataset is classified by calculating the size of the Intersection over Union (IOU) between the predicted face bounding box and the true face bounding box. When IOU < 0.3 and the number is less than 60, it is classified as a negative sample; when IOU > 0.65, it is classified as a positive sample; when 0.4 < IOU < 0.65, it is classified as a partial sample. The number of model iterations during the training of the three networks of the improved MTCNN is 15, and the thresholds of the three networks are set to 0.7, 0.6, and 0.7 respectively. The threshold of the image scaling factor (factor) is set to 0.7 (to generate images of different sizes).

[0055] The first network P-Net of the improved MTCNN uses depthwise separable convolutions to replace ordinary convolutions, and the network structure is as Figure 3 shown;

[0056] In this embodiment, depthwise separable convolution is divided into channel-wise convolution and pointwise convolution. In channel-wise convolution, the channels and kernels are "one-to-one," meaning one channel corresponds to one kernel and is convolved by that kernel, resulting in a new feature map with the same number of channels as the input. Pointwise convolution adds the feature maps obtained from channel-wise convolution with weights along the channel directions to obtain a new feature map. Depthwise separable convolution can significantly reduce network parameters, thereby reducing computational load.

[0057] The improved MTCNN's second network, R-Net, first replaces ordinary convolutional layers with depthwise separable convolutions. Then, residual blocks are introduced into the network and fused with involutional convolutions. Finally, the residual blocks integrated with involutional convolutions are combined with channel attention modules. The network structure is as follows: Figure 4 As shown;

[0058] The residual block is as follows Figure 5 As shown, two 3×3 convolutional layers are replaced with a three-layer convolution consisting of 1×1 + 3×3 + 1×1. First, a 1×1 convolution with dimensionality reduction is used to reduce computational cost, and then the original structure is restored using another 1×1 convolution operation. This structure has the advantages of reducing the number of parameters while maintaining accuracy.

[0059] The process of generating the Involution convolution kernel is as follows: Figure 6 As shown, assuming the input feature map is set as X is the input vector, H is the height of the input feature map, and W is the width of the input feature map. To determine the number of channels in the input feature map, the Involution convolution first extracts the input feature map. pixels at Its Involution kernel is R represents the domain. The value is the filter size, and 1 represents the number of shared channels, generated by the kernel function. Generate G Involution kernel size , where the function for G is the value shared by each group of channels. The number of groups, G Involution kernel size The current pixel in the input feature map is compared across C channels. Multiply the neighboring pixels one by one to get The three-dimensional matrix is ​​then used to sum the width and height dimensions while keeping the channel dimension unchanged, resulting in a new feature map. A vector of size, ultimately outputting a feature map. Involution convolutions can fuse contextual information in a larger spatial area, automatically assigning appropriate weights to different positions, and then prioritizing the most informative visual elements in the spatial domain. This convolution shares kernels across channels, making it more efficient and lightweight than ordinary convolutions. Residual blocks incorporated into Involution convolutions alleviate the vanishing gradient problem through batch normalization and activation functions, accelerating algorithm convergence, thereby speeding up model training and improving network performance.

[0060] In this embodiment, the kernel generation function Each position The function mapping at is ,in, for The corresponding set of input pixels, i.e. , and The transformation is linear, and together they form a bottleneck structure, where r is a scaling factor. This represents batch normalization and non-linear activation functions. The feature map output after involution is obtained by performing multiply-accumulate operations on the input features:

[0061]

[0062] Where k is the channel number. .

[0063] The residual attention module incorporating Involution convolution is as follows: Figure 7 As shown, firstly, a residual structure combined with Involution convolution is transformed to obtain the feature channel number as follows: A feature map u with width W and height H is compressed (squeeze) and then transformed into 1×1×C using global pooling, i.e., the width and height of the feature map are reduced to 1, resulting in a feature map of size 1×1×C. After obtaining the transformed 1×1×C feature map, an excitation operation is performed. First, a fully connected layer (FC) is added, and a dimensionality reduction operation with a hyperparameter r (SERadio) is performed to obtain a 1×1×C×r feature map. Then, a function activation layer (RELU) is added, and finally, a fully connected layer (FC) with a restoration function is added to restore the obtained feature map to 1×1×C. This operation predicts the weights of each channel and applies the obtained different channel weights to the corresponding channels of the previous feature map. Finally, the output weights of the previous operation are multiplied and scaled with the previous input features in a channel-wise weighted manner, and the final output feature map has a size of H×W×C. The role of the channel attention mechanism is to assign different weights to each channel, allowing the network to focus on important features and suppress unimportant features. Therefore, when the network extracts features, it can focus on important facial features and suppress non-facial features, thereby improving the accuracy of face detection.

[0064] The third network in the improved MTCNN, O-Net, replaces ordinary convolutions with depthwise separable convolutions and introduces Dropout layers at the fully connected layers of the network. It also uses Confluence instead of NMS to filter face detection boxes. The network structure is as follows: Figure 8 As shown;

[0065] The Dropout layer described above can effectively alleviate overfitting and achieve a certain degree of regularization. A comparison of the network before and after adding the Dropout layer is shown below. Figure 9 As shown, the Dropout layer can stop the activation value of a neuron with a certain probability p (Bernoulli distribution) during the forward propagation of the network, thereby enhancing the generalization ability of the model.

[0066] The Confluence algorithm consists of two steps: storing the best detection boxes and removing false positive boxes. First, the intersection between two detection boxes is represented by the Manhattan distance (MH). For example, and The MH between them can be represented as Then, the confidence level c of the regression during detection is weighted to obtain the optimal detection box. Finally, the proximity of other detection boxes to this optimal detection box is calculated to remove other false positive boxes.

[0067] The proximity calculation of two bounding boxes with coordinates is as follows: Figure 10 As shown. Pair the coordinates. , , and Add MH together, where MH are respectively , , , The sum P represents the degree of intersection between any two detection boxes, i.e. The bounding box (bbox) with the lowest p-value represents the best bounding box for the face object.

[0068] Confluence normalizes the coordinates of the detection boxes, reducing them to the range of 0 to 1, while maintaining the relationship between the coordinates. This can solve the problem of different sizes of face targets and their corresponding detection boxes.

[0069]

[0070] in, , , where norm represents the result of normalization.

[0071] The weighted proximity WP is represented by dividing P by its confidence score. .

[0072] Step 5: Input the images of different sizes from Step 3 into the improved MTCNN;

[0073] Step 5.1: Images of different sizes are first fed into the P-Net network in the improved MTCNN;

[0074] Step 5.1.1: Use a 12×12 detection box to perform face detection on images of different sizes, generating a large number of face detection boxes on the image;

[0075] Step 5.1.2: Use non-maximum suppression (NMS) to filter the face detection bounding boxes and output the detected face image P1;

[0076] In this embodiment, all obtained face detection boxes are first divided according to their face classification scores, resulting in multiple sets B. Within each set, the face detection boxes are sorted in descending order based on their classification scores, resulting in a descending list K. Starting with the face detection box with the highest face classification score in list K, the Interchange of Union (IOU) between this face detection box and other face detection boxes in the list is calculated. If the IOU is greater than a set threshold Y, other detection boxes are removed, and this best face detection box is retained and added to the final detection result A from list K. Then, starting with the face detection box with the second highest face classification score in list K, the face detection box removal operation in the previous step is repeated until all face detection boxes in list K have been filtered and removed. Finally, the list of each set is repeatedly traversed until all lists have been traversed.

[0077] Step 5.2: Resize the face image P1 output in Step 5.1.2 and the images of different sizes in Step 3, so that the image size becomes 24×24;

[0078] Step 5.3: Input the 24×24 image obtained in Step 5.2 into the second network R-Net of the improved MTCNN;

[0079] Step 5.3.1: Perform face detection on the input image to obtain a large number of face detection boxes;

[0080] Step 5.3.2: Use NMS to filter the face detection bounding boxes in the detection area and output the detected face image P2;

[0081] Step 5.4: Perform a resize operation on the face image P2 output in step 5.3.2 and the images of different sizes in step 3, so that the image size becomes 64×64;

[0082] Step 5.5: Input the 64×64 image obtained in Step 5.4 into the third network O-Net of the improved MTCNN;

[0083] Step 5.5.1: Perform face detection on the input image to obtain a large number of face detection boxes;

[0084] Step 5.5.2: Use Confluence to replace NMS to filter the face detection bounding boxes in the detection area, and output the detected face image P3.

[0085] In this embodiment, the categories are first divided according to the face classification scores to obtain multiple sets A. Then, the detection boxes (bboxes) in each set A are arranged in descending order according to the face classification scores. The bboxes in each set A are traversed, the Manhattan distance p between a certain bbox bi and other bboxes is calculated and normalized, and then the set where p < 2 is selected as a cluster. The weighted Manhattan distance WP is calculated, and the smaller WP in the cluster is selected as the WP of bi. After the traversal is completed, the bi with the smallest WP is selected as the optimal bbox and taken out from list_k. The operation is repeated.

[0086] In this embodiment, an improved MTCNN is used to perform face detection on two images from complex scenes: an outdoor parade and an indoor stock market. Both images contain small faces, faces with large occlusion areas, densely packed faces, and faces in low lighting conditions. After using the GFP-GAN method to repair occluded and blurred face images, facial features are clearer, and face localization is more accurate. This can further improve the accuracy of face detection for faces wearing masks, helmets, and randomly occluded faces, making it suitable for face detection in various complex scenes with occlusion.

Claims

1. A method for detecting occlusion and blurred faces in complex scenes, characterized in that, include: Step 1: Input a face image containing occlusion and blurring, and input the face image into the generative adversarial network GFP-GAN to generate face priors; Step 2: Perform face restoration on the face image containing occlusions and blurriness, and output the restored face image; Step 3: Scale the repaired face image from Step 2 using a certain scaling factor to obtain images of different sizes; Step 4: Improve the multi-task convolutional neural network MTCNN, which consists of an improved first network P-Net, an improved second network R-Net, and an improved third network O-Net; Step 5: Input the images of different sizes from Step 3 into the improved MTCNN for face detection; Step 2 includes: Step 2.1: Extract latent features from face images using the de-degradation module of the GFP-GAN network. and multi-resolution spatial features ; Step 2.2: Extract the latent features from Step 2.1 Latent code mapped to the latent feature space of a pre-trained GAN Up, then the potential code GAN features at multiple resolution scales are generated by pre-training each convolutional layer in the GAN. ; Step 2.3: Multi-resolution spatial features extracted in Step 2.1 For some GAN features Spatial modulation is performed, and a restored face image is generated through a channel segmentation-based spatial feature transformation layer at multiple resolution scales. ; In step 4, the first improved network, P-Net, uses depthwise separable convolutions instead of ordinary convolutions. The improved second network, R-Net, first replaces ordinary convolutional layers with depthwise separable convolutions, then introduces residual blocks into the network and fuses them with involutional convolutions, and finally combines the residual blocks integrated with involutional convolutions with channel attention modules. The improved third network, O-Net, replaces ordinary convolutions with depthwise separable convolutions, introduces Dropout layers at the fully connected layers of the network, and uses Confluence instead of NMS to filter face detection boxes. Step 5 includes: Step 5.1: Input images of different sizes into P-Net, the first improved network in MTCNN, and output the detected face image P1; Step 5.1.1: Use a 12×12 detection box to perform face detection on images of different sizes, generating a large number of face detection boxes on the image; Step 5.1.2: Use non-maximum suppression to filter the face detection boxes and output the detected face image P1; Step 5.2: Resize the face image P1 output in Step 5.1.2 and the images of different sizes in Step 3, so that the image size becomes 24×24; Step 5.3: Input the 24×24 image obtained in step 5.2 into the improved second network R-Net in MTCNN, and output the detected face image P2; Step 5.3.1: Perform face detection on the input image to obtain a large number of face detection boxes; Step 5.3.2: Use NMS to filter the face detection bounding boxes in the detection area and output the detected face image P2; Step 5.4: Perform a resize operation on the face image P2 output in step 5.3.2 and the images of different sizes in step 3, so that the image size becomes 64×64; Step 5.5: Input the 64×64 image obtained in step 5.4 into the improved third network O-Net in MTCNN, and output the detected face image P3; Step 5.5.1: Perform face detection on the input image to obtain a large number of face detection boxes; Step 5.5.2: Use the weighted post-processing algorithm Confluence to replace NMS to filter the face detection boxes in the detection area, and output the detected face image P3.

2. The method for detecting occlusion and blurred faces in complex scenes according to claim 1, characterized in that, The residual block uses a 1×1 convolution with dimensionality reduction to reduce the amount of computation, and then restores it through a 1×1 convolution operation; The residual attention module incorporating Involution convolution first transforms a residual structure combined with Involution convolution to obtain a feature map u with C feature channels, W width, and H height. This is then compressed by global pooling to transform H×W×C into 1×1×C, effectively reducing the width and height of the feature map to 1, resulting in a feature map of size 1×1×C. After obtaining the transformed 1×1×C feature map, activation is performed. First, a fully connected layer is added, followed by a dimensionality reduction operation with a hyperparameter r, resulting in a 1×1×C×r feature map. Then, a function activation layer is added, followed by a fully connected layer for restoration, ultimately restoring the feature map to 1×1×C. Finally, the output weights of the previous operation are multiplied and weighted across the input features, resulting in a final output feature map of size H×W×C.

Images