Prohibited article detection method based on triple attention
By adopting a triple attention-based detection method in the detection of prohibited items, combined with ResNet50 network and multi-module feature extraction technology, the existing detection accuracy and speed are solved, and more efficient and accurate detection of prohibited items is achieved.
Patent Information
- Application Number
- CN202411831557.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-12
- Publication Date
- 2025-05-02
AI Technical Summary
The existing prohibited item detection technology has problems with insufficient detection accuracy and speed, and it is difficult to meet the rapidly growing security inspection tasks.
The triple attention-based prohibited item detection method is adopted to perform feature extraction through the ResNet50 network, and the triple attention fusion module, hollow space pyramid module and multi-scale feature fusion module are used to enhance feature extraction capabilities and detection efficiency.
It improves the detection accuracy of prohibited items, enhances the ability to identify small targets, solves the problems of false detection and missed detection, and improves the detection speed.
Smart Images
Figure CN119919711A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection, and in particular relates to a prohibited article detection method based on triple attention. Background Art
[0002] In the fields of aviation, railways and urban rail transit, transportation safety is of vital importance. With the rapid development of the national economy, the flow of urban and rural populations has become increasingly frequent, and the carrying capacity of the public transportation system has risen sharply, resulting in an explosive growth in the workload of security inspection work.
[0003] Currently, prohibited items are detected through manual inspection and X-ray inspection, but the inspection speed and accuracy are obviously insufficient and cannot meet actual needs. Summary of the invention
[0004] The technical problem of the present invention is to propose a prohibited item detection method based on triple attention, improve the detection accuracy of prohibited items, solve the problems of false detection and missed detection, and increase the inspection speed of prohibited items.
[0005] The technical solution of the present invention is a prohibited article detection method based on triple attention, comprising the following steps: S1: Collect pictures of prohibited items and construct dataset D1; S2: Input the training set of dataset D1 into the ResNet50 network for feature extraction to obtain feature maps at different stages; S3: Input the feature maps at different stages into the triple attention fusion module to obtain feature enhanced feature maps at different stages; S4: Input the feature map of step S2 into the hole space pyramid module to obtain a feature map containing context information; S5: The feature maps of steps S3 and S4 are fused through a multi-scale feature fusion module to obtain the final output result.
[0006] Preferably, the ResNet50 network is used for image recognition and classification, introduces residual blocks, and solves the gradient vanishing and degradation problems in the network through jump connections, and includes an input layer, an initial convolutional layer, a maximum pooling layer, a residual block, a global average pooling layer, and a fully connected layer.
[0007] Furthermore, step S3 includes the following sub-steps: 1) Encode the channel information of the input feature map, and the calculation formula is: ; ; In the formula, h represents the feature map height, wrepresents the width of the feature map, c represents the number of feature map channels, i and j Indicates the counting unit, H and W respectively represent i , j The maximum value of .
[0008] 2) The feature maps obtained by embedding the horizontal and vertical coordinates are concatenated and then fused and transformed through the convolution transformation function. The calculation formula is: ; In the formula, f Represents the intermediate feature map that encodes spatial information in the horizontal and vertical directions, The concat operation represents the spatial dimension. represents a nonlinear activation function; 3) Decompose the intermediate feature map into two tensors and transform them into tensors with the same number of channels using convolution. The calculation formula is: ; ; In the formula, represents the Hardsigmoid activation function, F h and F w Respectively represent 1 Convolution of 1; 4) Calculate the output of the coordinate attention module, the calculation formula is: ; In the formula, Represents the value of the input feature map at position (i, j) under channel c, and Respectively represent the attention weights related to the height and width dimensions under the c channel; 5) Input the output of the coordinate attention module into the channel attention module and the spatial attention module. The calculation formula is: ; ; ; Where, X input represents the output of the coordinate attention module, g AVG and g MAX are the values of average pooling and maximum pooling, respectively, and k AVG and k MAX are the channel average and channel maximum, respectively.
[0009] Preferably, step S4 includes the following sub-steps: 1) The feature enhancement feature map of step S3 is convolved to reduce the dimension; 2) Construct a pooling pyramid and stack corresponding dilated convolutional layers to extract features at different scales; 3) Add a dilated pooling layer and perform convolution operations on the superimposed outputs of each layer to obtain a feature map rich in contextual information.
[0010] Preferably, in step S5, the multi-scale feature fusion module includes replacing the upsampling layer with a Dysample layer, and using a differential sampling method to adjust the upsampling parameters of the input features, reduce the amount of data, reduce the computational complexity, and improve the upsampling speed and efficiency.
[0011] Preferably, the upsampling process of the Dysample submodule includes the following sub-steps: 1) Encode the input feature map; 2) Generate an offset using a linear layer, rearrange pixels, and add it to the offset and the original sampling grid to get a set of sampling points; 3) Use the grid sample function to resample the input feature map according to the sampling set S to obtain the upsampled feature map.
[0012] Preferably, step S5 includes sequentially fusing the feature maps rich in contextual information with the feature enhanced feature maps of different stages to obtain feature maps of the same size.
[0013] Compared with the prior art, the beneficial effects of the present invention include: 1) The prohibited items detection method based on triple attention of the present invention compensates for the feature loss of prohibited items, obtains more feature details, enhances the network feature extraction capability, improves the detection accuracy of prohibited items, and enhances the detection efficiency.
[0014] 2) The prohibited items detection method based on triple attention in this invention introduces a multi-scale feature fusion pyramid to enhance the network's recognition ability for small targets and solve the problem of occlusion between prohibited items due to the large scale variation of prohibited items. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0016] Figure 1 A diagram showing a network structure of prohibited items detection based on triple attention according to an embodiment of the present invention; Figure 2 This is a structural diagram of the triple attention fusion submodule TAFM of an embodiment of the present invention; Figure 3 It is a structural diagram of the hollow space pyramid pooling submodule ASPP of an embodiment of the present invention; Figure 4 This is a structural diagram of the Dysample submodule of an embodiment of the present invention; Figure 5 This is a structural diagram of a multi-scale feature fusion submodule according to an embodiment of the present invention; Figure 6 A schematic diagram showing a comparison of visual detection results of prohibited items according to an embodiment of the present invention; DETAILED DESCRIPTION like Figure 1 As shown, a prohibited item detection method based on triple attention comprises the following steps: S1: Collect pictures of prohibited items, unify the format of the collected pictures, and scale the resolution to 512×512. Construct the dataset D1 and divide it into training set, validation set and test set according to the ratio of 8:1:1.
[0017] S2: Input the training set of dataset D1 into the ResNet50 network for feature extraction to obtain feature maps of four stages: stage1, stage2, stage3 and stage4.
[0018] The ResNet50 network is used for image recognition and classification. It introduces residual blocks and solves the gradient vanishing and degradation problems in the network through jump connections. It includes an input layer, an initial convolutional layer, a maximum pooling layer, a residual block, a global average pooling layer, and a fully connected layer.
[0019] S3: Input the feature maps of stage2, stage3 and stage4 into the triple attention fusion module to obtain the feature enhanced feature maps of three stages, stageF2, stageF3 and stageF4; like Figure 2 As shown, step S3 includes the following sub-steps: 1) Encode the channel information of the input feature map. The coordinate attention mechanism encodes the channel information of the input feature map X into spatial feature representations in two directions through average pooling operations along the horizontal and vertical directions. The calculation formula is: ; ; Where h represents the feature map height, w represents the feature map width, c represents the number of feature map channels, i and j represent counting units, and H and W represent the maximum values of i and j, respectively.
[0020] 2) The feature maps obtained by embedding the horizontal and vertical coordinates are concatenated and then fused and transformed through a 1×1 convolution transformation function. The calculation formula is: ; Where f represents the intermediate feature map encoding spatial information in the horizontal and vertical directions, [ ] represents the concat operation of the spatial dimension, represents a nonlinear activation function; 3) Decompose the intermediate feature map into two tensors and transform them into tensors with the same number of channels using 1×1 convolution. The calculation formula is: ; ; In the formula, represents the Hardsigmoid activation function, F h and F w Respectively represent 1 Convolution of 1; 4) Calculate the output of the coordinate attention module, the calculation formula is: ; In the formula, Represents the value of the input feature map at position (i, j) under channel c, and Respectively represent the attention weights related to the height and width dimensions under the c channel; 5) The output of the coordinate attention module is input into the channel attention module and the spatial attention module, and the average pooling and maximum pooling strategies are used in parallel. The maximum pooling reduces the mean shift caused by the convolution parameter error and captures the texture details. The calculation formula is: ; ; ; Where, X input represents the output of the coordinate attention module, g AVG and g MAX are the average pooling and maximum pooling values, respectively, and k AVG and k MAX are the channel average and channel maximum, respectively.
[0021] S4: The feature map stage4 is passed through the dilated spatial pyramid module with dilated rates of 1, 6, 12, and 18 to obtain the feature map P5 containing context information; like Figure 3 As shown, step S4 includes the following sub-steps: 1) The feature enhancement feature map of step S3 is convolved to reduce the dimension; 2) Construct a pooling pyramid and stack corresponding dilated convolutional layers to extract features at different scales; 3) Add a dilated pooling layer and perform convolution operations on the superimposed outputs of each layer to obtain a feature map rich in contextual information.
[0022] S5: The stage1, stageF2, stageF3, stageF4, and P5 feature maps are fused through a multi-scale feature fusion module to obtain the final output result.
[0023] like Figure 4 As shown, in step S5, the multi-scale feature fusion module includes replacing the upsampling layer with the Dysample layer, and using the differential sampling method to adjust the upsampling parameters of the input features, reduce the amount of data, reduce the computational complexity, and improve the upsampling speed and efficiency.
[0024] Dysample submodule, the upsampling process includes the following substeps: 1) Encode the input feature map; 2) Generate an offset using a linear layer, rearrange pixels, and add it to the offset and the original sampling grid to get a set of sampling points; 3) Use the grid sample function to resample the input feature map according to the sampling set S to obtain the upsampled feature map.
[0025] like Figure 5 As shown, step S5 includes the following sub-steps: 1) P5 and stageF4 are feature-fused to obtain P4; 2) P4 passes through the Dysample module to obtain a feature map of the same size as stageF3, and then performs feature fusion with stageF3 to obtain P3; 3) P3 passes through the Dysample module to obtain a feature map of the same size as stageF2, and then performs feature fusion with stageF2 to obtain P2; 4) P2 passes through the Dysample module to obtain a feature map of the same size as stage1, and then performs feature fusion with stage1 to obtain P1.
[0026] In order to verify the specific effect of the prohibited items detection algorithm designed by the present invention, the trained model was experimented on the HiXray dataset. The entire dataset includes 45,364 X-ray images in real scenes, and 4,360 images are randomly selected as the experimental dataset. The prohibited items are divided into seven categories: power bank without battery PO1, power bank with battery PO2, tablet computer TA, mobile phone MP, computer LP, cosmetics CO and water cup WA.
[0027] The average precision value mAP and recall rate Recall are used as evaluation indicators. The higher the average precision value, the better the model performance, and the higher the recall rate, the lower the missed detection rate of the model. The experimental environment is: Windows 10 operating system, memory is 16GB, CPU is I7-4770, graphics card model is NVIDIA GeForce GTX 1660. The programming language is Python 3.8, and the deep learning framework is Pytorch1.12. The test results are compared with the current mainstream target detection model, ResNet50+FPN is the baseline, and the results on the HiXray dataset are shown in Table 1.
[0028] Table 1
[0029] The average precision and recall rate of the proposed method both reach the maximum values of 92.44% and 88.73% respectively, and show more outstanding performance compared with the classical target detection algorithm and the advanced X-ray prohibited items detection algorithm.
[0030] In order to verify the effect of each module designed in this paper on improving model performance, separate experiments were conducted on each module. Taking ResNet50+FPN as the benchmark model, the experimental data obtained from the HiXray dataset are shown in Table 2.
[0031] Table 2
[0032] Table 2 shows that each module has improved mAP and Recall. Among them, the mAP and Recall of Baseline+TAFM reached 89.04% and 82.51%, respectively, which were 1.72% and 0.85% higher than the baseline model; Baseline+ASPP were 1.31% and 2.46% higher than the baseline model; Baseline+Dysample were 1.04% and 5.9% higher than the baseline model. The experimental results show that TAFM helps detection to achieve more accurate positioning and improve accuracy, while Dysample tends to find more targets and improve recall. TAFNet integrates the three modules, and the detection performance is further improved. The mAP and Recall values reached 93.01% and 88.59%, respectively, which are 5.12% and 7.07% higher than the baseline model.
[0033] like Figure 6 As shown in the figure, the visual comparison results of the original network and the present invention on HiXray data and are not obvious due to being blocked by the background. Thanks to the powerful feature extraction capability of the present invention, the object to be detected is accurately detected, while the original network has false detections and missed detections.
[0034] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limitations of the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.
Claims
1. A method for detecting prohibited items based on triple attention, characterized in that: The following steps are involved: S1: Collect pictures of prohibited items and construct dataset D1; S2: Input the training set of dataset D1 into the ResNet50 network for feature extraction to obtain feature maps at different stages; S3: Input the feature maps at different stages into the triple attention fusion module to obtain feature enhanced feature maps at different stages; S4: Input the feature map of step S2 into the hole space pyramid module to obtain a feature map containing context information; S5: The feature maps of steps S3 and S4 are fused through a multi-scale feature fusion module to obtain the final output result.
2. According to claim 1, a method for detecting prohibited items based on triple attention is characterized in that: In step S2, the ResNet50 network is used for image recognition and classification, introduces residual blocks, and solves the gradient vanishing and degradation problems in the network through jump connections, and includes an input layer, an initial convolutional layer, a maximum pooling layer, a residual block, a global average pooling layer and a fully connected layer.
3. The method for detecting prohibited items based on triple attention according to claim 1, characterized in that: Step S3 includes the following sub-steps: 1) Use the pooling kernel to encode the channels of the input feature map along the horizontal and vertical coordinates respectively. The calculation formula is: ； ； In the formula, h represents the feature map height, w represents the feature map width, and c represents the number of feature map channels. i and j Indicates the counting unit, H and W represent i , j The maximum value of 2) The feature maps obtained by embedding the horizontal and vertical coordinates are concatenated and then fused and transformed through the convolution transformation function. The calculation formula is: ； In the formula, f Represents the intermediate feature map that encodes spatial information in the horizontal and vertical directions, [ ] represents the concat operation of the spatial dimension, represents a nonlinear activation function, F 1 represents the convolution transformation function; 3) Decompose the intermediate feature map into two tensors and transform them into tensors with the same number of channels using convolution. The calculation formula is: ； ； In the formula, represents the Hardsigmoid activation function, F h and F w Respectively represent 1 Convolution of 1; 4) Calculate the output of the coordinate attention module, the calculation formula is: ； In the formula, Indicates that the input feature map is at the position under the c channel ( i,j ), and Respectively represent the attention weights related to the height and width dimensions under the c channel; 5) Input the output of the coordinate attention module into the channel attention module and the spatial attention module. The calculation formula is: ； ； ； In the formula, X input represents the output of the coordinate attention module, g AVG and g MAX are the average pooling and maximum pooling values, respectively, and k AVG and k MAX are the channel average and channel maximum, respectively.
4. According to claim 1, a method for detecting prohibited items based on triple attention is characterized in that: The step S4 includes the following sub-steps: 1) The feature enhancement feature map of step S3 is convolved to reduce the dimension; 2) Construct a pooling pyramid and stack corresponding dilated convolutional layers to extract features at different scales; 3) Add a dilated pooling layer and perform convolution operations on the superimposed outputs of each layer to obtain a feature map rich in contextual information.
5. According to claim 1, a method for detecting prohibited items based on triple attention is characterized in that: In step S5, the multi-scale feature fusion module includes replacing the upsampling layer with the Dysample layer, and using the differential sampling method to adjust the upsampling parameters of the input features, reduce the amount of data, reduce the computational complexity, and improve the upsampling speed and efficiency.
6. According to claim 5, a method for detecting prohibited items based on triple attention is characterized in that: The Dysample submodule, the upsampling process includes the following sub-steps: 1) Encode the input feature map; 2) Generate an offset using a linear layer, rearrange pixels, and add it to the offset and the original sampling grid to get a set of sampling points; 3) Use the grid sample function to resample the input feature map according to the sampling set to obtain the upsampled feature map.
7. The method for detecting prohibited items based on triple attention according to claim 6, characterized in that: In step S5, the feature maps rich in context information are sequentially fused with the feature enhanced feature maps of different stages to obtain feature maps of the same size.