A method and device for capturing facial expression in a weak light environment

Abstract

The present invention provides a method and device for capturing facial expressions in a weak light environment, which can accurately frame seats of different scales in a video stream in a complex environment, and detect and extract facial features based on the framed results for expression recognition. It is characterized by comprising the following steps: step S1, obtaining a video stream of a bird's-eye view in the theater to be tested; step S2, detecting the video stream based on a theater seat positioning model, and obtaining a labeling frame of the latest position of all seats in the theater to be tested; step S3, using an expression recognition classification model to perform face positioning and expression recognition on all labeling frames, thereby obtaining the facial expression category of the audience in the theater to be tested. Among them, the theater seat positioning model uses a network combining ResNet50 and FPN as the Backbone, and adds a CBAM module to construct an initial theater seat positioning network and train the initial network, and the expression recognition classification model is based on the construction and training of an improved VGGNet and an improved Focal Loss combined facial expression recognition algorithm.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a method and a device for capturing facial expressions in a weak light environment. Background Art

[0002] Scientific statistical methods for movie viewing data and intelligent methods for maintaining movie viewing order are essential to the modernization of the film industry. Box office data is an important criterion for movie revenue statistics and an intuitive representation of whether a movie is successful. The real attendance rate, the audience status during the screening process, such as facial expressions, the number of times they leave, and other data are important bases for evaluating the audience's viewing experience from more dimensions, and provide important references for producers to control the rhythm of the film and for exhibitors to provide a better viewing environment. Maintaining a good viewing order is the responsibility and obligation of film exhibitors, and intelligent viewing order maintenance is of great significance to film copyright protection and various disaster prevention.

[0003] Auditorium seat positioning is an important part of theater data collection. Traditional theater seat positioning based on expert systems is highly dependent on manual presets. Due to the large number of seats, manual information input is very time-consuming, and preset information is difficult to adapt to changes in camera positions, and the maintenance cost is high. With the continuous development of computer vision technology, especially the outstanding performance of deep learning in the field of target detection, it has been widely used in many fields such as smart medical care and autonomous driving. The seat positioning method based on deep convolutional neural network has multiple advantages such as easy deployment, easy migration and accurate positioning. However, most target detection methods do not have the ability to frame tilted targets, and perform poorly for theater seat targets with highly diverse attributes such as size and scale. The target detection method needs to be improved.

[0004] Collecting and identifying the expressions of moviegoers can effectively help theater managers improve management efficiency and provide audiences with a better viewing experience; it can also provide an important reference for producers to control the rhythm of the film. Also benefiting from the development of deep learning, computer image processing technology has become increasingly sophisticated, and facial expression recognition has gradually become an important part of computer image processing, with wide and necessary uses in human-computer interaction, security, and robot manufacturing. However, facial expression recognition methods do not achieve good results in dim light, so existing methods need to be improved. Summary of the invention

[0005] In order to solve the above problems, a method and device are provided to accurately locate and frame seat targets of different scales and proportions for a cinema camera video stream under various lighting conditions, and to detect and extract facial features within the frame range for expression recognition based on the framed results. The present invention adopts the following technical solutions:

[0006] The invention provides a method for capturing facial expressions in a weak light environment, which is used for locating faces of audiences in a cinema and recognizing facial expressions. The method is characterized in that the method comprises the following steps: step S1, obtaining a video stream of a bird's-eye view angle in a cinema to be tested; step S2, detecting every set frame in the video stream based on a cinema seat positioning model, and obtaining a labeling frame of the latest positions of all seats in the cinema to be tested; step S3, using an expression recognition and classification model to locate faces and recognize expressions in all labeling frames, so as to obtain the facial expression categories of the audiences in the cinema to be tested, wherein the cinema seat positioning model uses a network combining ResNet50 and FPN as a backbone, and adds a CBAM module to construct an initial network for cinema seat positioning and train the initial network, and the expression recognition and classification model is obtained based on a facial expression recognition algorithm combining an improved VGGNet and an improved Focal Loss.

[0007] A method for capturing facial expressions in a weak light environment provided by the present invention may also have such a technical feature, wherein a theater seat positioning model is obtained by the following training steps: step S2-1, collecting a video stream of a bird's-eye view in the theater, and intercepting a plurality of frame images from the video stream; step S2-2, marking all frame images with seat or no seat labels, constructing a first data set using the labeled frame images, and dividing the first data set into a first training set, a first verification set, and a first test set; step S2-3, using the first training set to train an initial network for theater seat positioning for optimization iteration; step S2-4, after each iteration, calculating the loss error according to the model parameters of the last layer of the initial network; step S2-5, using the loss error back propagation, adopting the stochastic gradient descent method and minimizing the loss function to optimize the model parameters; step S2-6, repeating steps S2-3 to S2-5 until a predetermined condition is met, thereby obtaining a theater seat positioning model.

[0008] The method for capturing facial expressions in a weak light environment provided by the present invention may also have such a technical feature, wherein an expression recognition classification model is obtained by the following steps: step S3-1, using a theater seat positioning model to perform seat positioning on all frame images to obtain frame images with seats; step S3-2, marking facial expression labels on areas where people sit in the frame images with seats, using the frame images with facial expression labels as the second data set, and dividing the second data set into a second training set, a second verification set and a second test set; step S3-3, using the second training set to train a facial expression recognition algorithm for optimization iteration; step S3-4, after each iteration, calculating the loss error according to the parameters of the last layer structure of the facial expression recognition algorithm; step S3-5, using the loss error back propagation, adopting the stochastic gradient descent method and minimizing the loss function to perform parameter optimization; step S3-6, repeating steps S3-3 to S3-5 until a predetermined condition is met, thereby obtaining the expression recognition classification model.

[0009] The present invention provides a method for capturing facial expressions in a weak light environment, which may also have the following technical features: ResNet50 includes four residual modules, each of which is at least composed of a convolution operation Conv, a linear rectification function ReLU and a batch normalization BN in sequence, and is responsible for feature extraction; the CBAM module includes a SAM submodule and a CAM submodule, the CAM submodule is composed of a MaxPool pooling layer, an AvgPool pooling layer, a weight-sharing MLP, an addition operation and a sigmoid activation function; the SAM submodule is composed of two channel-based global maximum pooling and a global average pooling, a 7x7 convolution and a sigmoid activation function; the SAM module takes the output of the CAM submodule as input, outputs a spatial attention matrix, and multiplies the attention matrix with the original feature to obtain the final generated feature.

[0010] The method for capturing facial expressions in a weak light environment provided by the present invention may also have such technical features, wherein the improved VGGNet is obtained by improving the VGGNet by adding an output module on the basis of transfer learning, and the improved Focal Loss is obtained by improving the Focal Loss by setting a probability threshold.

[0011] The present invention also provides a facial expression capture device in a low-light environment, which is used to locate the faces of audiences in a theater and recognize facial expressions. The device is characterized in that it includes: a video stream acquisition unit, a theater seat positioning unit, and an expression recognition and classification unit, wherein the video stream acquisition unit is used to obtain a video stream of a bird's-eye view angle in the theater to be tested, the theater seat positioning unit detects the video stream to obtain a position marking frame with seats in the theater to be tested, and the expression recognition and classification unit performs face positioning and expression recognition on the position marking frame to obtain the facial expression category of the audience in the theater to be tested.

[0012] Function and Effect of the Invention

[0013] According to a method for capturing facial expressions in a low-light environment of the present invention, the method can first detect the seat position of the theater to be tested based on the theater seat positioning model under complex camera and lighting environments, and has the advantages of high precision and strong generalization. Then, the expression recognition and classification model locates the face of the seat where the audience sits and recognizes the facial expression, so that the climax and highlight points of the film can be judged according to the facial expression. Among them, the theater seat positioning model adopts a target detection strategy different from the traditional one. Since the FPN and ResNet50 networks are combined as the backbone network Backbone, the scale independence of the detection target is achieved. At the same time, a CBAM module is also added to the positioning model to coordinate the processing of the feature map extracted by the backbone network, and the semantic relationship between the seats closely distributed in the picture is used as a booster during recognition, which is conducive to detecting seats with obvious features. In addition, since a new output module is added to the VGG16 of the facial expression recognition algorithm and a probability threshold is set for Focal Loss, overfitting is effectively avoided and the generalization of the model is enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a flow chart of a method for capturing facial expressions in a low-light environment in an embodiment of the present invention;

[0015] Figure 2 is a training flow chart of a theater seat positioning model in an embodiment of the present invention;

[0016] Figure 3 is a training flow chart of the expression recognition classification model in an embodiment of the present invention;

[0017] Figure 4 It is a structural schematic diagram of a facial expression capture device in a weak light environment according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of a method and device for capturing facial expressions in a weak light environment of the present invention in combination with embodiments and drawings.

[0019] <Example>

[0020] In this embodiment, a method and device for capturing facial expressions in a low-light environment are run by a computer. The computer requires at least one graphics card for GPU acceleration to complete the model training process. The trained theater seat positioning model, expression recognition classification model and image recognition process are stored in the computer in the form of executable code. The computer can call these models through executable code and batch process image data frames in multiple scenes at the same time, obtain and output facial expression analysis in the seats in each scene, and finally output statistics and analysis of all expressions in the picture.

[0021] Figure 1 It is a flow chart of a method for capturing facial expressions in a weak light environment according to an embodiment of the present invention.

[0022] like Figure 1 As shown, a method for capturing facial expressions in a low-light environment for locating faces of audience members in a theater and recognizing facial expressions comprises the following steps:

[0023] Step S1, obtaining a video stream of a bird's-eye view in a cinema to be tested.

[0024] Step S2: Based on the theater seat positioning model, detect every set frame in the video stream to obtain the labeling box of the latest position of all seats in the theater to be tested.

[0025] In this embodiment, a network combining ResNet50 and FPN is used as the backbone network Backbone. The backbone network consists of a residual module, a pooling layer, an activation layer, a batch normalization layer and a jump connection module. A CBAM module is added to the backbone network to collaboratively process the feature maps extracted by the backbone network, and the existing deep learning framework PyTorch is used to complete the initial network construction of theater seat positioning.

[0026] Among them, the residual network ResNet50 includes 4 residual modules, each of which is composed of at least a convolution operation Conv, a linear rectification function ReLU and a batch normalization BN in sequence, and is used to extract features from the input data.

[0027] The CBAM module consists of a CAM (Channel Attention Module) submodule and a SAM (Spatial Attention Module) submodule. The CAM module consists of two different pooling layers, MaxPool and AvgPool, a weighted MLP, a final sum operation, and a sigmoid activation function. The SAM module consists of two channel-based global maximum pooling and global average pooling, a 7x7 convolution, and a final sigmoid activation function. The SAM module takes the output of the CAM module as input and outputs a spatial attention matrix, which is then multiplied by the original feature to obtain the final generated feature.

[0028] Figure 2 It is a training flow chart of the theater seat positioning model in an embodiment of the present invention.

[0029] like Figure 2 As shown, in this embodiment, the theater seat positioning model is obtained by training the theater seat positioning initial network. Specifically:

[0030] Step S2-1, collecting a video stream of a bird's-eye view of the theater, and capturing a number of frames of images from the video stream.

[0031] In this embodiment, the real-time video stream in the unmanned scene of the cinema from a bird's-eye view is preprocessed, and the preprocessing includes size adjustment, center cropping and standardization operations, and image frames are intercepted as training data preparation.

[0032] Step S2-2, labeling all frame images with seats or no seats, constructing a first data set using the labeled frame images, and dividing the first data set into a first training set, a first verification set, and a first test set.

[0033] In this embodiment, the first training set is 2000 images with a resolution of 512x512 cropped from data frames extracted from high-definition videos; the first validation set is 54 images with a resolution of 1920x1080 extracted from high-definition videos, which serve as a generalization control for model training; the first test set is 50 images with a resolution of 1920x1080 extracted from high-definition videos, which serve as a test of the effectiveness of model training.

[0034] Step S2-3, using the images in the first training set to train the initial network for theater seat positioning in batches to optimize the iteration parameters.

[0035] In this embodiment, each layer of the initial network for theater seat positioning contains different parameters, which are randomly set when constructed. The batch size of the training images entering the network model each time is 16, and the training is iterated 150 times in total.

[0036] Step S2-4, after each iteration, the loss error is calculated based on the model parameters of the last layer of the initial network.

[0037] Step S2-5, using the loss error back propagation, adopting the stochastic gradient descent method and minimizing the loss function to optimize the model parameters.

[0038] Step S2-6, repeating steps S2-3 to S2-5 until a predetermined effect is achieved, thereby obtaining a theater seat positioning model.

[0039] The output of the trained theater seat positioning model consists of two parts: classification results and regression results. The classification result represents the probability score of the area being classified as a seat. If the score is greater than 0.5, it is regarded as an area containing seats. The dimension of the regression result is four-dimensional, which respectively represents the horizontal coordinate of the center point of the area, the vertical coordinate of the center point, the width of the area, and the height of the area.

[0040] In this embodiment, after the training is completed, the obtained cinema seat positioning model is verified on the first verification set. The experimental results show that for 54 theater images in overhead perspective scenes, the recognition AP of the cinema seat positioning model reaches more than 94%, and the recall rate reaches 96%, that is, the model can almost completely correctly locate almost all cinema seats.

[0041] Step S3, using the expression recognition classification model to perform face positioning and expression recognition on all the marked frames, so as to obtain the facial expression categories of the audience in the theater to be tested.

[0042] In this embodiment, VGGNet and Focal Loss are improved and combined to obtain a facial expression recognition algorithm, and the facial expression recognition algorithm is trained to obtain an expression recognition classification model.

[0043] For facial expression recognition and analysis algorithms, in the expression recognition task, expression samples between different categories have great similarities and are easily confused, and more convolutional layers are needed to extract deep and abstract features. However, due to the lack of data under dark conditions, deep network models such as ResNet, when faced with expression datasets with a small number of samples, are prone to a sharp increase in the number of parameters due to the excessive depth of the layers, resulting in overfitting and failing to achieve good recognition results. The VGG16 model has two characteristics: small convolution kernels and deep networks. The model is divided into different blocks, each of which contains several convolutional layers and a pooling layer. A large number of convolutional layers enable the model to have better feature extraction capabilities.

[0044] Moreover, in actual classification problems, most samples are simple and easy to classify, and their contribution to the loss is dominant, while complex and difficult samples are easily ignored by the model. This is even more obvious in facial expressions. For some reasons, expressions of disgust are very difficult to understand and capture, which often leads to overfitting of the model.

[0045] In this embodiment, in order to avoid overfitting and enhance the generalization of the model, a new output module is designed for VGGNet based on transfer learning to replace the fully connected module of the pre-trained model. The output module is mainly composed of improved depthwise separable convolution and global average pooling. The depthwise separable convolution layer can further extract features, saving a large number of parameters compared to ordinary convolution, while still having the same feature extraction capability as ordinary convolution.

[0046] At the same time, in order to solve the overfitting caused by uneven sample distribution, a probability threshold is set for Focal Loss to alleviate this phenomenon. Focal Loss controls the modulation coefficient to focus on difficult samples and balances the number of samples of different categories by balancing parameters.

[0047] Figure 3 It is a training flow chart of the expression recognition classification model in an embodiment of the present invention.

[0048] like Figure 3 As shown, in this embodiment, the facial expression recognition classification model is obtained by training the above-mentioned facial expression recognition algorithm. Specifically:

[0049] Step S3-1, using the theater seat positioning model to perform seat positioning on all frame images to obtain frame images with seats.

[0050] Step S3-2, labeling facial expressions in the areas where people sit in the frame images with seats, using the frame images with facial expression labels as the second data set, and dividing the second data set into a second training set, a second verification set and a second test set.

[0051] In this embodiment, all the frames of the seats are recorded, and then the frames with audience members sitting in the video stream are selected, and these parts are deducted according to the coordinates of the seats as the second data set. And they are divided into the second training set, the second verification set and the second test set according to the ratio of training set: verification set: test set = 10:1:1. And the second training set and the second verification set are well labeled.

[0052] Step S3-3, using the second training set to train the facial expression recognition algorithm for optimization iteration.

[0053] In this embodiment, the frame images in the second training set are fed into the facial expression recognition algorithm for training in batches. The batch size of the frame images fed into the training each time is 64, and the training is iterated 100 times in total.

[0054] Step S3-4, after each iteration, the loss error is calculated according to the parameters of the last layer structure of the facial expression recognition algorithm.

[0055] Step S3-5, using the loss error back propagation, adopting the stochastic gradient descent method and minimizing the loss function to perform parameter optimization.

[0056] Step S3-6, repeating steps S3-3 to S3-5 until a predetermined condition is met, thereby obtaining an expression recognition classification model.

[0057] In this embodiment, after completing the training, the obtained expression recognition classification model is verified on the second verification set. The experimental results show that the recognition accuracy of the expression recognition classification model reaches more than 89% (low light state). However, since what is pursued is not the accuracy of each individual sample, but the state of a group of people in a picture, this accuracy rate can already achieve the expected effect.

[0058] Figure 4 It is a structural schematic diagram of a facial expression capture device in a weak light environment according to an embodiment of the present invention.

[0059] This embodiment also provides a facial expression capture device in a weak light environment. Figure 4 As shown, the facial expression capturing device 10 includes a video stream acquiring unit 1 , a theater seat positioning unit 2 and an expression recognition and classification unit 3 .

[0060] The video stream acquisition unit 1 is used to acquire a video stream of a bird's-eye view angle in the theater to be tested.

[0061] The theater seat positioning unit 2 detects the video stream collected by the video stream acquisition unit 1 to obtain the position marking frame of the seats in the theater to be tested.

[0062] The expression recognition and classification unit 3 performs face positioning and expression recognition on the position annotation frame detected by the theater seat positioning unit 2 to obtain the facial expression category of the audience in the theater to be tested.

[0063] Example Function and Effect

[0064] According to a facial expression capture method in a low-light environment provided by this embodiment, the seat position of the theater to be tested can be first detected based on the theater seat positioning model in a complex camera and lighting environment, with the advantages of high precision and strong generalization. Then, the expression recognition and classification model is used to locate the faces of the seats where the audience sits and recognize the facial expressions, so that the climax and highlights of the film can be judged according to the facial expressions.

[0065] In the embodiment, the theater seat positioning model adopts a target detection strategy different from the traditional one. Since the FPN and ResNet50 networks are combined as the backbone network, the scale independence of the detection target is achieved. At the same time, the CBAM module is also added to the positioning model to coordinate the processing of the feature map extracted by the backbone network, and the semantic relationship between the closely distributed seats in the picture is used as a boost for recognition, which is conducive to detecting seats with obvious features.

[0066] In the embodiment, the backbone network Backbone uses a residual module to deepen the number of network layers, which can express high-level features in the image, and therefore has excellent performance in target detection tasks. Combined with the feature pyramid structure, the model makes predictions based on feature maps of multiple scales, and therefore has good compatibility with multi-scale targets.

[0067] In the embodiment, since a new output module is added to the VGG16 of the facial expression recognition algorithm, overfitting is avoided, the generalization of the model is enhanced, and a large number of parameters are saved compared to ordinary convolution, while still having the feature extraction capability equivalent to that of ordinary convolution. At the same time, by setting a probability threshold for Focal Loss, the overfitting problem caused by uneven sample distribution is also solved.

[0068] The above embodiments are only used to illustrate specific implementation modes of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A method for capturing facial expressions in a low-light environment, used for locating the faces of audience members in a theater and recognizing their facial expressions, characterized in that: The following steps are involved: Step S1, obtaining a video stream of a bird's-eye view in the cinema to be tested; Step S2, based on the theater seat positioning model, detect every set frame in the video stream to obtain the labeling box of the latest position of all seats in the theater to be tested, wherein the theater seat positioning model is obtained through the following training steps: Step S2-1, collecting a video stream of a bird's-eye view in the theater, and capturing several frames of images from the video stream, Step S2-2, labeling all the frame images with seats or no seats, constructing a first data set using the labeled frame images, and dividing the first data set into a first training set, a first validation set, and a first test set. Step S2-3, using the first training set to train the initial network for theater seat positioning to perform optimization iterations, Step S2-4, after each iteration, calculating the loss error according to the model parameters of the last layer of the initial network, Step S2-5, using the loss error back propagation, adopting stochastic gradient descent method and minimizing loss function to optimize model parameters, Step S2-6, repeating the steps S2-3 to S2-5 until a predetermined condition is met, thereby obtaining the theater seat positioning model; Step S3, using the expression recognition classification model to perform face positioning and expression recognition on all the marked frames, so as to obtain the facial expression categories of the audience in the theater to be tested, The theater seat positioning model uses a network combining ResNet50 and FPN as the backbone, and adds a CBAM module to construct an initial theater seat positioning network and train the initial network. The expression recognition classification model is obtained based on the facial expression recognition algorithm that builds and trains an improved VGGNet and an improved Focal Loss. The ResNet50 includes 4 residual modules, each of which is composed of at least a convolution operation Conv, a linear rectification function ReLU and a batch normalization BN in sequence, and is responsible for feature extraction. The CBAM module includes a SAM submodule and a CAM submodule. The CAM submodule consists of a MaxPool pooling layer, an AvgPool pooling layer, a weight-sharing MLP, a sum operation, and a sigmoid activation function. The SAM submodule consists of two channel-based global maximum pooling and global average pooling, a 7x7 convolution and a sigmoid activation function. The SAM submodule takes the output of the CAM submodule as input, outputs a spatial attention matrix, and multiplies the attention matrix with the original features to obtain the final generated features. The improved VGGNet is obtained by adding an output module to VGGNet on the basis of transfer learning, wherein the output module is composed of at least a depthwise separable convolution and a global average pooling. The improved Focal Loss is obtained by setting an improved probability threshold for the Focal Loss.

2. The method for capturing facial expressions in a weak light environment according to claim 1, Features: in, The expression recognition classification model is obtained by the following steps: Step S3-1, using the theater seat positioning model to perform seat positioning on all the frame images to obtain frame images with seats; Step S3-2, marking a facial expression label on an area where a person is sitting in the frame image with a seat, using the frame image with the facial expression label as a second data set, and dividing the second data set into a second training set, a second verification set, and a second test set; Step S3-3, using the second training set to train the facial expression recognition algorithm to perform optimization iterations; Step S3-4, after each iteration, calculating the loss error according to the parameters of the last layer structure of the facial expression recognition algorithm; Step S3-5, using the loss error back propagation, adopting the stochastic gradient descent method and minimizing the loss function to perform parameter optimization; Step S3-6, repeating steps S3-3 to S3-5 until a predetermined condition is met, thereby obtaining the expression recognition classification model.

3. A facial expression capture device in a low-light environment, used to locate the face of the audience in a theater and recognize the facial expression, characterized in that: include: Video stream acquisition department, theater seat positioning department and expression recognition and classification department, The video stream acquisition unit is used to acquire the video stream of the top-down angle in the theater to be tested. The theater seat positioning unit detects the video stream to obtain a position marking frame with seats in the theater to be tested, The expression recognition and classification unit uses the expression recognition and classification model to perform face positioning and expression recognition on the position annotation frame to obtain the facial expression category of the audience in the theater to be tested, The theater seat positioning model is obtained through the following training steps: Step S2-1, collecting a video stream of a bird's-eye view in the theater, and capturing several frames of images from the video stream, Step S2-2, labeling all the frame images with seats or no seats, constructing a first data set using the labeled frame images, and dividing the first data set into a first training set, a first validation set, and a first test set. Step S2-3, using the first training set to train the initial network for theater seat positioning to perform optimization iterations, Step S2-4, after each iteration, calculating the loss error according to the model parameters of the last layer of the initial network, Step S2-5, using the loss error back propagation, adopting stochastic gradient descent method and minimizing loss function to optimize model parameters, Step S2-6, repeating the steps S2-3 to S2-5 until a predetermined condition is met, thereby obtaining the theater seat positioning model; The theater seat positioning model uses a network combining ResNet50 and FPN as the backbone, and adds a CBAM module to build an initial theater seat positioning network and train the initial network. The expression recognition classification model is obtained based on the facial expression recognition algorithm that builds and trains an improved VGGNet and an improved Focal Loss. The ResNet50 includes 4 residual modules, each of which is composed of at least a convolution operation Conv, a linear rectification function ReLU and a batch normalization BN in sequence, and is responsible for feature extraction. The CBAM module includes a SAM submodule and a CAM submodule. The CAM submodule consists of a MaxPool pooling layer, an AvgPool pooling layer, a weight-sharing MLP, a sum operation, and a sigmoid activation function. The SAM submodule consists of two channel-based global maximum pooling and global average pooling, a 7x7 convolution and a sigmoid activation function. The SAM submodule takes the output of the CAM submodule as input, outputs a spatial attention matrix, and multiplies the attention matrix with the original features to obtain the final generated features. The improved VGGNet is obtained by adding an output module to VGGNet on the basis of transfer learning, wherein the output module is composed of at least a depthwise separable convolution and a global average pooling. The improved Focal Loss is obtained by setting an improved probability threshold for the Focal Loss.

Images