Article detection and identification method based on attention mechanism convolutional neural network
By adopting the item detection and identification method based on the attention mechanism convolutional neural network in the millimeter wave security inspection system, the system's inefficient computing efficiency and insufficient recognition ability of prohibited items when processing huge amounts of data is solved, and a more efficient and accurate security inspection process is achieved.
Patent Information
- Application Number
- CN202411970479.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-02
AI Technical Summary
The existing millimeter-wave security system is inefficient in computing when processing huge amounts of data, and lacks the ability to identify prohibited items in complex contexts.
The object detection and recognition method based on the attention mechanism is adopted, and the data processing efficiency and the accuracy of prohibited items are improved through multi-frame parallel processing and a convolutional neural network containing the attention mechanism.
It significantly improves the data processing speed and the accuracy of identification of prohibited items, improves security inspection efficiency and accuracy, and provides passengers with a safer and more comfortable security inspection experience.
Smart Images
Figure CN119919880A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of security inspection technology, and in particular to an object detection and recognition method based on an attention mechanism convolutional neural network. Background Art
[0002] At the entry and exit site, security inspection is an important measure to ensure national security and maintain social order. Traditional entry and exit security inspection mainly relies on X-ray imaging and metal detectors. Although they can ensure safety to a certain extent, they have many shortcomings. For example, X-rays have certain radiation hazards to the human body, metal detection doors have poor detection effects on non-metallic objects, and manual body searches are not only inefficient, but may also cause discomfort and resistance to passengers. Therefore, with the advancement of science and technology and people's increasing requirements for security inspection efficiency and accuracy, intelligent security inspection equipment based on millimeter wave imaging technology has emerged.
[0003] Millimeter wave body imaging security scanners use millimeter wave's non-contact scanning and high-resolution imaging functions to quickly and accurately detect all kinds of items carried by passengers, including metals, non-metals, liquids, powders and fire sources. The introduction of this technology not only significantly improves the comprehensiveness and accuracy of security inspections, but also greatly shortens passengers' waiting time and improves security inspection efficiency. More importantly, millimeter waves are safe electromagnetic waves that do not penetrate the human body and have no ionizing radiation. They are very safe for special groups such as children, the elderly, and pregnant women, further enhancing the humanization and safety of the security inspection process.
[0004] However, the amount of data generated by millimeter wave imaging technology is huge. Traditional millimeter wave imaging detection systems have low computational efficiency when processing this large amount of data, and their ability to identify prohibited items in complex backgrounds is insufficient. How to efficiently and accurately process this data and extract valuable information from it has become a key technical problem that needs to be solved urgently. In order to improve the intelligence level, detection accuracy and speed of the security inspection system, it is necessary to introduce more advanced intelligent image processing algorithms. Summary of the invention
[0005] The purpose of the present invention is to address the technical difficulties of low efficiency in processing the massive data generated by the millimeter-wave security imaging system at the entry and exit site and poor ability to identify prohibited items under complex backgrounds. In order to process these data more efficiently and accurately and quickly extract information about potentially dangerous foreign objects therefrom, a method for object detection and identification based on an attention mechanism convolutional neural network is proposed. By combining multi-frame parallelism and a convolutional neural network containing an attention mechanism, a more efficient and accurate method for object detection and identification is provided for intelligent security inspection equipment, which will greatly improve the efficiency and accuracy of security inspections and provide passengers with a safer and more comfortable security inspection experience.
[0006] To achieve the purpose of the present invention, the following technical solutions are adopted:
[0007] A method for object detection and recognition based on an attention mechanism convolutional neural network includes the following steps:
[0008] Obtain N frames of millimeter wave security inspection images from the millimeter wave security inspection equipment, where N ≥ 2;
[0009] N frames of millimeter wave security inspection images are converted into N parallel frame input layers through serial-to-parallel conversion, and each frame of millimeter wave security inspection image in the N parallel frame input layers is output as a separate channel;
[0010] The N frames of millimeter-wave security inspection images in the N-way parallel frame input layer are output one by one to N convolutional neural network processing units with attention mechanism for convolution processing to obtain the feature information of the N frames of millimeter-wave security inspection images for subsequent detection and identification of prohibited items.
[0011] A further improvement is that each of the convolutional neural network processing units containing the attention mechanism includes a first convolutional layer, a first maximum pooling layer, an attention mechanism module and a weighted fusion module.
[0012] A further improvement is that the method of performing convolution processing by N convolutional neural network processing units with attention mechanism to obtain feature information of N frames of millimeter wave security inspection images includes:
[0013] Perform preliminary feature extraction on the input millimeter-wave security inspection image through the first convolution layer to generate an original feature map;
[0014] The original feature map is processed through the first maximum pooling layer to reduce the amount of calculation while retaining important features;
[0015] The original feature map processed by the first maximum pooling layer is processed by the attention mechanism module to generate an attention map;
[0016] The attention map is multiplied element by element with the original feature map through the weighted fusion module to achieve effective feature fusion and generate a weighted feature map. N convolutional neural network processing units with attention mechanism perform convolution processing to generate a total of N weighted feature maps.
[0017] A further improvement is that each of the convolutional neural network processing units containing the attention mechanism also includes a second convolutional layer and a second maximum pooling layer for further extracting features from the generated weighted feature map.
[0018] A further improvement is that the object detection and identification method further comprises the following steps:
[0019] N weighted feature maps generated by N convolutional neural network processing units with attention mechanisms are cascaded through cascade layers to form a weighted feature panorama covering N frames of millimeter-wave security inspection images;
[0020] The weighted feature panorama is input into the third convolutional layer and the third maximum pooling layer for deep processing to extract the feature information of prohibited items.
[0021] A further improvement is that the third convolution layer is used to further extract features through convolution operations, and the third maximum pooling layer is used to perform dimensionality reduction and pooling operations on the features to reduce the amount of calculation and retain key features.
[0022] A further improvement is that the object detection and identification method further comprises the following steps:
[0023] The extracted feature information of prohibited items is input into the fully connected layer for feature fusion and classification decision.
[0024] A further improvement is that the specific method of inputting the extracted feature information of prohibited items into the fully connected layer for feature fusion and classification decision-making includes:
[0025] The fully connected layer performs nonlinear combination of the feature information of prohibited items through linear transformation and activation function to form a more discriminative feature representation. Finally, the classifier is used to classify the features to obtain the feature classification results of prohibited items. The feature classification results are output to determine whether prohibited items are contained and the type of prohibited items.
[0026] A further improvement is that the object detection and identification method further includes the following steps: marking, locating and visually displaying classification information of detected prohibited objects.
[0027] A further improvement is that the attention mechanism is a spatial attention mechanism, which is used to enhance the ability to extract key feature information of objects.
[0028] The beneficial effects of the present invention are:
[0029] The object detection and recognition method based on the attention mechanism convolutional neural network proposed in the present invention shows significant technical advantages compared with the existing entry and exit security inspection millimeter wave imaging processing technology.
[0030] First, in terms of processing efficiency, the present invention realizes the rapid processing of millimeter-wave security inspection image data by introducing a multi-frame parallel processing mechanism. The serial processing method in the prior art is inefficient when processing massive amounts of data, while the parallel processing mode of the present invention can process multiple frames of image data at the same time, significantly improving the data processing speed, thereby shortening the security inspection time and improving the traffic efficiency at the entry and exit site.
[0031] Secondly, in terms of the ability to identify prohibited items, the present invention adopts a convolutional neural network with an attention mechanism. This network structure can automatically focus on key areas in the image, that is, those areas that may contain prohibited items, thereby improving the accuracy and robustness of recognition. Especially in complex backgrounds, the attention mechanism can effectively suppress the interference of irrelevant information and ensure that prohibited items can be accurately identified.
[0032] In addition, the present invention can further effectively improve the accuracy and reliability of prohibited items detection by cascading steps such as forming a weighted feature panorama, extracting deep prohibited items features, and fusing fully connected layer features. These steps together constitute a complete millimeter wave intelligent prohibited items detection and identification system suitable for entry-exit security inspection, which can achieve rapid and accurate detection of prohibited items. In general, the present invention has significant technical advantages in the field of millimeter wave imaging processing technology for entry-exit security inspection, which not only improves data processing efficiency, but also significantly enhances the ability to identify prohibited items, providing a more efficient and accurate solution for entry-exit security inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a flowchart of an object detection and recognition method based on an attention mechanism convolutional neural network;
[0034] Figure 2 This is a structural diagram of an object detection and recognition method based on an attention mechanism convolutional neural network;
[0035] Figure 3 A schematic diagram of the N-channel parallel frame input layer formed by serial-to-parallel conversion of millimeter-wave security inspection image frames in the input preprocessing layer;
[0036] Figure 4 This is a block diagram of the convolutional neural network processing unit with an attention mechanism. DETAILED DESCRIPTION
[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0039] The following is an explanation of the technical terms involved in the present invention:
[0040] Millimeter wave security imaging: This is a technology that uses millimeter waves (a type of electromagnetic wave) to scan the human body or objects, which can generate X-ray-like photos to check whether prohibited items are carried.
[0041] Convolutional Neural Network (CNN): A special type of deep learning model that is particularly good at processing image data. It automatically learns features from images to perform classification or recognition tasks by simulating the working method of the human brain.
[0042] Attention mechanism: This is a technique that mimics human vision, allowing the model to "focus" on certain specific areas when processing an image, while ignoring other unimportant parts.
[0043] Please refer to the attached Figure 1 -Attached Figure 4 The embodiment of the present invention proposes an object detection and identification method based on an attention mechanism convolutional neural network, which can be applied to millimeter wave security imaging systems in public places such as entry and exit, airports, and stations to achieve efficient and accurate detection and identification of prohibited items. Figure 1 As shown, the object detection and identification method includes the following steps:
[0044] Step S1: Acquire N frames of millimeter wave security inspection images from a millimeter wave security inspection device, where N≥2.
[0045] It is understandable that N frames of millimeter wave security inspection images are obtained from the millimeter wave security inspection equipment, and these images are the results of scanning people or items entering and leaving the country. In addition, each frame of the image can also be standardized, such as resizing, normalizing pixel values, etc., to ensure that all images have the same format and size.
[0046] Step S2: N frames of millimeter wave security inspection images are converted into N parallel frame input layers through serial-to-parallel conversion, and each frame of millimeter wave security inspection image in the N parallel frame input layers is output as a separate channel.
[0047] It can be understood that by performing serial-to-parallel conversion (i.e., conversion from serial to parallel) on N frames of millimeter-wave security inspection images through the input preprocessing layer to construct an N-way parallel frame input layer, each frame of millimeter-wave security inspection image is regarded as a separate output, thereby realizing the conversion of serial millimeter-wave security inspection image frames into parallel millimeter-wave security inspection image frames. This means that images that originally needed to be processed sequentially can now be received and processed by multiple processing units at the same time.
[0048] Step S3: Output the N frames of millimeter-wave security inspection images in the N-channel parallel frame input layer to N convolutional neural network processing units with attention mechanism for convolution processing, and obtain feature information of the N frames of millimeter-wave security inspection images for subsequent detection and identification of prohibited items.
[0049] It can be understood that by assigning N frames of millimeter-wave security inspection images to N convolutional neural network processing units with attention mechanisms, each processing unit is responsible for processing one frame of image, that is, convolutional neural network processing unit 1 with attention mechanism, convolutional neural network processing unit 2 with attention mechanism, ... convolutional neural network processing unit N with attention mechanism are used to perform convolution processing on N frames of millimeter-wave security inspection image data respectively, so as to extract the spatial feature information of millimeter-wave security inspection images in parallel, so as to process these image data more efficiently and accurately.
[0050] Specifically, in this embodiment, Figure 4 As shown, each of the convolutional neural network processing units containing the attention mechanism includes a first convolutional layer, a first maximum pooling layer, an attention mechanism module and a weighted fusion module.
[0051] In step S3, the method of performing convolution processing by N convolutional neural network processing units with attention mechanisms to obtain feature information of N frames of millimeter wave security inspection images includes:
[0052] Step S31: Perform preliminary feature extraction on the input millimeter wave security inspection image through the first convolution layer to generate an original feature map.
[0053] It can be understood that each convolutional neural network processing unit with an attention mechanism first performs preliminary feature extraction on the input millimeter-wave security inspection image through the first convolution layer. This process generates a set of original feature maps, which contain local patterns and structural information in the input image, such as edges, textures, etc.
[0054] Step S32: Process the original feature map through the first maximum pooling layer to reduce the amount of calculation while retaining important features.
[0055] It can be understood that the image size is reduced by the first maximum pooling layer, which reduces the amount of calculation while retaining important features. This step does not change the content of the original feature map, but only reduces its resolution. The addition of the first maximum pooling layer is used to avoid overfitting and increase the generalization ability of the model.
[0056] Step S33: The original feature map processed by the first maximum pooling layer is processed by the attention mechanism module to generate an attention map. The attention mechanism module in this embodiment is a spatial attention mechanism, which is used to enhance the ability to extract key feature information of objects.
[0057] It can be understood that the spatial attention mechanism is used to analyze which areas in the original feature map are more important and generate a corresponding attention map. The attention map is a map with the same size as the original feature map, in which each pixel value represents the importance weight of the position. This step is used to help the model focus on key areas that may contain prohibited items.
[0058] Step S34: The attention map is multiplied element by element with the original feature map through the weighted fusion module to achieve effective feature fusion and generate a weighted feature map. N convolutional neural network processing units with attention mechanisms perform convolution processing to generate a total of N weighted feature maps.
[0059] It can be understood that by multiplying the attention map with the original feature map element by element to generate a weighted feature map, this process enhances the feature representation of the key area, allowing the model to focus more on the parts that may contain prohibited items.
[0060] In a preferred solution of this embodiment, each of the convolutional neural network processing units containing the attention mechanism also includes a second convolutional layer and a second maximum pooling layer, which are used to further extract features from the generated weighted feature map.
[0061] In the object detection and identification method, if Figure 2 As shown, after step S4, the following steps are also included:
[0062] Step S5: The N weighted feature maps generated by the N convolutional neural network processing units with attention mechanisms are cascaded through a cascade layer to form a weighted feature panoramic map covering N frames of millimeter-wave security inspection images.
[0063] It can be understood that by arranging the N weighted feature maps generated by N convolutional neural network processing units with attention mechanisms in a certain order and merging them into a larger feature map (weighted feature panorama), this is a complete feature image that contains all the important information. The purpose of this is to retain the feature information of all input images and prepare for subsequent deep feature extraction. Cascading can be achieved through simple splicing or through more complex fusion strategies, such as learning a weight matrix to dynamically adjust the contribution of each feature map.
[0064] Step S6: Input the weighted feature panorama image into the third convolutional layer and the third maximum pooling layer for deep processing to extract the feature information of prohibited items.
[0065] Specifically, the third convolution layer is used to further extract features through convolution operations, and the third maximum pooling layer is used to perform dimensionality reduction and pooling operations on the features to reduce the amount of calculation and retain key features.
[0066] It is understandable that the purpose of this step is to extract deeper features of prohibited items from the weighted feature panorama, and input the weighted feature panorama formed after cascading into the third convolutional layer to further extract deep features. The convolution operation at this stage is designed to capture more complex patterns and structures, especially those that may indicate prohibited items. The features are then reduced in dimension through the third maximum pooling layer to reduce the amount of computation while retaining the most critical information, which helps to improve the efficiency and generalization ability of the model.
[0067] In a preferred embodiment of this embodiment, Figure 2 As shown, the object detection and identification method further includes the following steps after step S6:
[0068] Step S7: Input the extracted feature information of prohibited items into the fully connected layer for feature fusion and classification decision, and the fully connected layer includes a first fully connected layer and a second fully connected layer.
[0069] Specifically, the specific method of inputting the extracted feature information of prohibited items into the fully connected layer for feature fusion and classification decision includes:
[0070] The first fully connected layer performs nonlinear combination of the feature information of prohibited items through linear transformation and activation function to form a more discriminative feature representation. This step helps the model better understand the relationship between features, which is further passed to the second fully connected layer. Finally, a classifier (such as the softmax function) is used to classify the features to obtain the feature classification results of prohibited items. The feature classification results are output to determine whether prohibited items are contained and the type of prohibited items.
[0071] In a preferred solution of this embodiment, the object detection and identification method further includes the following steps: marking, locating and visually displaying classification information of detected prohibited items to facilitate security personnel to make quick judgments and responses.
[0072] Specifically, it can be intuitively presented to security personnel through a user interface, assisting them in making quick and accurate judgments.
[0073] In addition, the network structure parameters such as the convolutional layer, the maximum pooling layer, and the fully connected layer in the present invention can be optimized and adjusted according to the specific application scenario to improve the detection accuracy and recognition efficiency.
[0074] The object detection and recognition method based on the attention mechanism convolutional neural network proposed in the present invention shows significant technical advantages compared with the existing entry and exit security inspection millimeter wave imaging processing technology.
[0075] First, in terms of processing efficiency, the present invention realizes the rapid processing of millimeter-wave security inspection image data by introducing a multi-frame parallel processing mechanism. The serial processing method in the prior art is inefficient when processing massive amounts of data, while the parallel processing mode of the present invention can process multiple frames of image data at the same time, significantly improving the data processing speed, thereby shortening the security inspection time and improving the traffic efficiency at the entry and exit site.
[0076] Secondly, in terms of the ability to identify prohibited items, the present invention adopts a convolutional neural network with an attention mechanism. This network structure can automatically focus on key areas in the image, that is, those areas that may contain prohibited items, thereby improving the accuracy and robustness of recognition. Especially in complex backgrounds, the attention mechanism can effectively suppress the interference of irrelevant information and ensure that prohibited items can be accurately identified.
[0077] In addition, the present invention can further effectively improve the accuracy and reliability of prohibited items detection by cascading steps such as forming a weighted feature panorama, extracting deep prohibited items features, and fusing fully connected layer features. These steps together constitute a complete millimeter wave intelligent prohibited items detection and identification system suitable for entry-exit security inspection, which can achieve rapid and accurate detection of prohibited items. In general, the present invention has significant technical advantages in the field of millimeter wave imaging processing technology for entry-exit security inspection, which not only improves data processing efficiency, but also significantly enhances the ability to identify prohibited items, providing a more efficient and accurate solution for entry-exit security inspection.
[0078] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for object detection and recognition based on an attention mechanism convolutional neural network, characterized in that: The following steps are involved: Obtain N frames of millimeter wave security inspection images from the millimeter wave security inspection equipment, where N ≥ 2; N frames of millimeter wave security inspection images are converted into N parallel frame input layers through serial-to-parallel conversion, and each frame of millimeter wave security inspection image in the N parallel frame input layers is output as a separate channel; The N frames of millimeter-wave security inspection images in the N-way parallel frame input layer are output one by one to N convolutional neural network processing units with attention mechanism for convolution processing to obtain the feature information of the N frames of millimeter-wave security inspection images for subsequent detection and identification of prohibited items.
2. According to claim 1, the object detection and recognition method based on the attention mechanism convolutional neural network is characterized in that: Each of the convolutional neural network processing units containing the attention mechanism includes a first convolutional layer, a first maximum pooling layer, an attention mechanism module and a weighted fusion module.
3. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 2 is characterized in that: The method of performing convolution processing on N convolutional neural network processing units with attention mechanisms to obtain feature information of N frames of millimeter wave security inspection images includes: Perform preliminary feature extraction on the input millimeter-wave security inspection image through the first convolution layer to generate an original feature map; The original feature map is processed through the first maximum pooling layer to reduce the amount of calculation while retaining important features; The original feature map processed by the first maximum pooling layer is processed by the attention mechanism module to generate an attention map; The attention map is multiplied element by element with the original feature map through the weighted fusion module to achieve effective feature fusion and generate a weighted feature map. N convolutional neural network processing units with attention mechanism perform convolution processing to generate a total of N weighted feature maps.
4. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 3 is characterized in that: Each of the convolutional neural network processing units containing the attention mechanism also includes a second convolutional layer and a second maximum pooling layer, which are used to further extract features from the generated weighted feature map.
5. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 3 is characterized in that: The object detection and identification method further comprises the following steps: N weighted feature maps generated by N convolutional neural network processing units with attention mechanisms are cascaded through cascade layers to form a weighted feature panorama covering N frames of millimeter-wave security inspection images; The weighted feature panorama is input into the third convolutional layer and the third maximum pooling layer for deep processing to extract the feature information of prohibited items.
6. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 5 is characterized in that: The third convolution layer is used to further extract features through convolution operations, and the third maximum pooling layer is used to perform dimensionality reduction and pooling operations on the features to reduce the amount of calculation and retain key features.
7. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 5 is characterized in that: The object detection and identification method further comprises the following steps: The extracted feature information of prohibited items is input into the fully connected layer for feature fusion and classification decision.
8. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 7 is characterized in that: The specific method of inputting the extracted feature information of prohibited items into the fully connected layer for feature fusion and classification decision-making includes: The fully connected layer performs nonlinear combination of the feature information of prohibited items through linear transformation and activation function to form a more discriminative feature representation. Finally, the classifier is used to classify the features to obtain the feature classification results of prohibited items. The feature classification results are output to determine whether prohibited items are contained and the type of prohibited items.
9. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 8 is characterized in that: The object detection and identification method also includes the following steps: marking, locating and visually displaying the classification information of the detected prohibited objects.
10. The object detection and recognition method based on the attention mechanism convolutional neural network according to claim 1 is characterized in that: The attention mechanism is a spatial attention mechanism, which is used to enhance the ability to extract key feature information of objects.