Deformable convolution-based hazardous article detection method and device, and medium

Abstract

The invention discloses a dangerous goods detection method and device based on deformable convolution and a medium, and belongs to the technical field of dangerous goods detection. According to the method, the detection method based on deformable convolution is adopted, dangerous goods of different shapes and sizes can be more flexibly adapted, so that the detection accuracy is improved, and through introduction of an attention mechanism, especially combination of channel attention and space attention, the recognition capability of shape information of the dangerous goods is further enhanced. Through a multi-feature fusion strategy, the regional enhancement features, the color features and the edge features are subjected to centralized fusion, and each feature is subjected to deep feature extraction through an independent network branch, so that feature information is enriched, and the robustness of the model to different imaging conditions and dangerous goods types is improved. A ternary loss function is adopted to train the model, the model is guided to carry out feature fusion better by setting weights of different losses, and the recognition performance of the model is optimized through a cross entropy loss function.

Description

Technical Field

[0001] The present invention belongs to the technical field of dangerous goods detection, and in particular relates to a dangerous goods detection method, device and medium based on deformable convolution. Background Art

[0002] X-ray security inspection machines are widely used in transportation and various large public venues such as airports, railway stations, and subway stations. They are important facilities for maintaining public safety and transportation safety. Passenger luggage is scanned by an X-ray machine and security inspection images are generated in real time. Security inspectors check the security inspection images to determine whether there are dangerous goods in the luggage, thereby ensuring public safety. However, the manual inspection mechanism has the risk of missed inspections and wrong inspections. Intelligent identification of dangerous goods in X-ray security inspection images has important practical significance.

[0003] However, due to the differences in imaging conditions of different security inspection equipment and the diversity of dangerous goods shapes and sizes, traditional detection methods face many challenges. For example, traditional methods often rely on fixed image processing algorithms, which may not be able to effectively adapt to fluctuations in image quality and the changing characteristics of dangerous goods. In addition, when dealing with objects with irregular shapes, the fixed convolution kernel size of traditional convolutional neural networks may not be able to flexibly adapt to the actual shape of the object, resulting in insufficient feature extraction. Therefore, existing technologies are limited in the accuracy, robustness and efficiency of dangerous goods detection. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention proposes a new dangerous goods detection method based on deformable convolution.

[0005] In order to achieve the above object, the following technical solution is adopted: The present invention provides a dangerous goods detection method based on deformable convolution, comprising:

[0006] Input the X-ray image to be detected into the processing system;

[0007] Preprocess the input X-ray images, including image resizing, normalization and standardization, to eliminate the impact of imaging differences between different security inspection equipment and ensure the consistency of the image processing process;

[0008] The attention mechanism is used to extract regional enhancement features. The pixel-level features learned through training are used to enhance the regional information of dangerous goods while retaining the geometric structure of the image. The attention mechanism includes the concatenation of channel attention and spatial attention to highlight the shape information of dangerous goods.

[0009] A multi-feature fusion strategy is adopted to centrally fuse regional enhancement features with color features and edge features. Each feature is subjected to deep feature extraction through an independent network branch to enrich feature information and improve the robustness of the model.

[0010] In the color feature branch, a convolutional neural network model is used to extract deep features from the RGB three-channel color image, and a one-dimensional feature vector is obtained through a global average pooling operation;

[0011] In the region enhancement feature branch, the CBAM-based attention structure is used to extract the region enhancement features, and the CBRP convolution module is used for deep feature extraction, where the CBRP convolution module includes convolution, normalization, activation and pooling operations;

[0012] In the edge feature branch, the Sobel operator is used to extract image edge information, and the CBRP convolution module is used to extract deep features to enhance the model's ability to recognize the shape and contour of objects.

[0013] For each deep feature extracted, it is mapped into a one-dimensional feature vector through a fully connected layer, and an adjustable weight parameter is set for each feature to achieve adaptive fusion of features. The weight parameter can be adaptively adjusted during the training process so that different features play different roles.

[0014] A multi-layer perceptron is used to further analyze and select the fused feature vectors. The activation function is used to select the feature representations that are meaningful for the task of identifying dangerous goods. Finally, the feature representations are combined to determine whether the image contains dangerous goods.

[0015] Furthermore, the model is trained using a ternary loss function, which includes fusion loss, edge loss, and region enhancement loss. By setting weights for different losses, the model is guided to better perform feature fusion, and the cross entropy loss function is used to calculate the loss between the prediction vector and the label vector to optimize the recognition performance of the model.

[0016] Furthermore, during the training process, the model is evaluated using public datasets, including experimental analysis of overall recognition performance and generalization performance, to ensure that the model has good recognition effects on samples with the same and different color distributions.

[0017] Furthermore, ablation experiments are performed to verify the impact of multi-feature fusion and ternary loss function on model performance, so as to determine the optimal network structure and parameter settings and improve the efficiency and accuracy of the model in practical applications.

[0018] Furthermore, the preprocessing step also includes adjusting the image size to a size required for network input.

[0019] Furthermore, the preprocessing step also includes noise removal processing on the X-ray image, using a non-local mean filtering algorithm to effectively reduce Gaussian noise and salt and pepper noise in the image.

[0020] Furthermore, in the multi-feature fusion strategy, after each feature is deeply extracted through an independent network branch, a feature selection mechanism is also included, which selects the most representative feature subset based on the feature importance score to further optimize the recognition accuracy of the model and reduce the computational complexity.

[0021] Furthermore, the multi-layer perceptron adopts a confidence-based decision-making mechanism when performing feature analysis and selection. This mechanism dynamically adjusts the decision threshold according to the strength of association between the feature vector and the dangerous goods category to achieve adaptive recognition at different confidence levels and improve the robustness of the model in complex security inspection environments.

[0022] The present invention also provides a dangerous goods detection device based on deformable convolution, comprising:

[0023] An image input unit, used for receiving an X-ray image to be detected;

[0024] A preprocessing unit, used for preprocessing the input X-ray image;

[0025] A feature extraction unit, used for extracting region enhancement features, color features and edge features;

[0026] A feature fusion unit is used to fuse different features;

[0027] An identification unit, used for outputting the identification result of dangerous goods according to the fused feature vector;

[0028] The training unit is used to train the device through the ternary loss function to optimize the feature extraction and fusion process.

[0029] The present invention also provides a storage medium, which is used to store instructions, and the instructions are used to execute the steps of the dangerous goods detection method based on deformable convolution.

[0030] The beneficial effects of the present invention are:

[0031] (1) The present invention adopts a detection method based on deformable convolution, which can more flexibly adapt to dangerous goods of different shapes and sizes, thereby improving the accuracy of detection. By introducing the attention mechanism, especially the combination of channel attention and spatial attention, the recognition ability of dangerous goods shape information is further enhanced.

[0032] (2) Through the multi-feature fusion strategy, the present invention centrally fuses regional enhancement features, color features and edge features, and each feature is subjected to deep feature extraction through an independent network branch, which not only enriches the feature information but also improves the robustness of the model to different imaging conditions and types of dangerous goods.

[0033] (3) The present invention uses a ternary loss function to train the model. By setting weights of different losses, the model is guided to better perform feature fusion, and the recognition performance of the model is optimized through the cross-entropy loss function. The confidence-based decision-making mechanism can dynamically adjust the decision threshold according to the correlation strength between the feature vector and the dangerous goods category, thereby achieving adaptive recognition at different confidence levels and further improving the performance of the model in complex security inspection environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of the dangerous goods detection method based on deformable convolution of the present invention.

[0035] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] Unless otherwise defined, all professional and scientific terms used herein have the same meanings as those familiar to those skilled in the art. In addition, any methods and materials similar or equivalent to those described herein may be applied to the present invention. The preferred implementation methods and materials described herein are for demonstration purposes only and are not intended to limit the content of this application.

[0038] The experimental methods in the following examples are conventional methods unless otherwise specified, and the experimental materials used in the following examples are purchased from commercial channels unless otherwise specified.

[0039] Example

[0040] Dangerous goods detection method, equipment and medium based on deformable convolution

[0041] The method comprises:

[0042] Input the X-ray image to be detected into the processing system;

[0043] Preprocess the input X-ray images, including image resizing, normalization and standardization, to eliminate the impact of imaging differences between different security inspection equipment and ensure the consistency of the image processing process;

[0044] The attention mechanism is used to extract regional enhancement features. The pixel-level features learned through training are used to enhance the regional information of dangerous goods while retaining the geometric structure of the image. The attention mechanism includes the concatenation of channel attention and spatial attention to highlight the shape information of dangerous goods.

[0045] A multi-feature fusion strategy is adopted to centrally fuse regional enhancement features with color features and edge features. Each feature is subjected to deep feature extraction through an independent network branch to enrich feature information and improve the robustness of the model.

[0046] In the color feature branch, a convolutional neural network model is used to extract deep features from the RGB three-channel color image, and a one-dimensional feature vector is obtained through a global average pooling operation;

[0047] In the region enhancement feature branch, the CBAM-based attention structure is used to extract the region enhancement features, and the CBRP convolution module is used for deep feature extraction, where the CBRP convolution module includes convolution, normalization, activation and pooling operations;

[0048] In the edge feature branch, the Sobel operator is used to extract image edge information, and the CBRP convolution module is used to extract deep features to enhance the model's ability to recognize the shape and contour of objects.

[0049] For each deep feature extracted, it is mapped into a one-dimensional feature vector through a fully connected layer, and an adjustable weight parameter is set for each feature to achieve adaptive fusion of features. The weight parameter can be adaptively adjusted during the training process so that different features play different roles.

[0050] A multi-layer perceptron is used to further analyze and select the fused feature vectors. The activation function is used to select the feature representations that are meaningful for the task of identifying dangerous goods. Finally, the feature representations are combined to determine whether the image contains dangerous goods.

[0051] The model is trained using a ternary loss function, which includes fusion loss, edge loss, and region enhancement loss. By setting weights for different losses, the model is guided to better perform feature fusion, and the cross entropy loss function is used to calculate the loss between the prediction vector and the label vector to optimize the recognition performance of the model.

[0052] During the training process, the model is evaluated using public datasets, including experimental analysis of overall recognition performance and generalization performance, to ensure that the model has good recognition effects on samples with the same and different color distributions.

[0053] The impact of multi-feature fusion and ternary loss function on model performance is verified through ablation experiments to determine the optimal network structure and parameter settings, thereby improving the efficiency and accuracy of the model in practical applications.

[0054] The preprocessing step also includes adjusting the image size to the size required by the network input.

[0055] The preprocessing step also includes noise removal of the X-ray image, using a non-local mean filtering algorithm to effectively reduce Gaussian noise and salt and pepper noise in the image.

[0056] In the multi-feature fusion strategy, after each feature is deeply extracted through an independent network branch, a feature selection mechanism is also included, which selects the most representative feature subset based on the feature importance score to further optimize the recognition accuracy of the model and reduce the computational complexity.

[0057] The multi-layer perceptron adopts a confidence-based decision-making mechanism when performing feature analysis and selection. The mechanism dynamically adjusts the decision threshold according to the correlation strength between the feature vector and the dangerous goods category to achieve adaptive recognition at different confidence levels and improve the robustness of the model in complex security inspection environments.

[0058] This comparative example also provides a dangerous goods detection device based on deformable convolution, including:

[0059] An image input unit, used for receiving an X-ray image to be detected;

[0060] A preprocessing unit, used for preprocessing the input X-ray image;

[0061] A feature extraction unit, used for extracting region enhancement features, color features and edge features;

[0062] A feature fusion unit is used to fuse different features;

[0063] An identification unit, used for outputting the identification result of dangerous goods according to the fused feature vector;

[0064] The training unit is used to train the device through the ternary loss function to optimize the feature extraction and fusion process.

[0065] This comparative example also provides a storage medium, which is used to store instructions, and the instructions are used to execute the steps of the dangerous goods detection method based on deformable convolution.

[0066] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0067] The present invention and its implementation methods are described above, which is not restrictive. The drawings are only one of the implementation methods of the present invention, and the actual application is not limited thereto. In short, if ordinary technicians in the field are inspired by it and design methods and embodiments similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the protection scope of the present invention.

Claims

1. A dangerous goods detection method based on deformable convolution, characterized by: include: Input the X-ray image to be detected into the processing system; Preprocess the input X-ray images, including image resizing, normalization and standardization, to eliminate the impact of imaging differences between different security inspection equipment and ensure the consistency of the image processing process; The attention mechanism is used to extract regional enhancement features. The pixel-level features learned through training are used to enhance the regional information of dangerous goods while retaining the geometric structure of the image. The attention mechanism includes the concatenation of channel attention and spatial attention to highlight the shape information of dangerous goods. A multi-feature fusion strategy is adopted to centrally fuse regional enhancement features with color features and edge features. Each feature is subjected to deep feature extraction through an independent network branch to enrich feature information and improve the robustness of the model. In the color feature branch, a convolutional neural network model is used to extract deep features from the RGB three-channel color image, and a one-dimensional feature vector is obtained through a global average pooling operation; In the region enhancement feature branch, the CBAM-based attention structure is used to extract the region enhancement features, and the CBRP convolution module is used for deep feature extraction, where the CBRP convolution module includes convolution, normalization, activation and pooling operations; In the edge feature branch, the Sobel operator is used to extract image edge information, and the CBRP convolution module is used to extract deep features to enhance the model's ability to recognize the shape and contour of objects. For each deep feature extracted, it is mapped into a one-dimensional feature vector through a fully connected layer, and an adjustable weight parameter is set for each feature to achieve adaptive fusion of features. The weight parameter can be adaptively adjusted during the training process so that different features play different roles. A multi-layer perceptron is used to further analyze and select the fused feature vectors. The activation function is used to select the feature representations that are meaningful for the task of identifying dangerous goods. Finally, the feature representations are combined to determine whether the image contains dangerous goods.

2. The method for detecting dangerous goods based on deformable convolution according to claim 1, characterized in that: The model is trained using a ternary loss function, which includes fusion loss, edge loss, and region enhancement loss. By setting the weights of different losses, the model is guided to better perform feature fusion, and the loss between the prediction vector and the label vector is calculated through the cross entropy loss function to optimize the recognition performance of the model.

3. The method for detecting dangerous goods based on deformable convolution according to claim 2, characterized in that: During the training process, the model is evaluated using public datasets, including experimental analysis of overall recognition performance and generalization performance, to ensure that the model has good recognition effects on samples with the same and different color distributions.

4. The method for detecting dangerous goods based on deformable convolution according to claim 3, characterized in that: The impact of multi-feature fusion and ternary loss function on model performance is verified through ablation experiments to determine the optimal network structure and parameter settings, thereby improving the efficiency and accuracy of the model in practical applications.

5. The method for detecting dangerous goods based on deformable convolution according to claim 4, characterized in that: The preprocessing step also includes adjusting the image size to the size required by the network input.

6. The method for detecting dangerous goods based on deformable convolution according to claim 5, characterized in that: The preprocessing step also includes noise removal of the X-ray image, using a non-local mean filtering algorithm to effectively reduce Gaussian noise and salt and pepper noise in the image.

7. The method for detecting dangerous goods based on deformable convolution according to claim 6, characterized in that: In the multi-feature fusion strategy, after each feature is deeply extracted through an independent network branch, a feature selection mechanism is also included, which selects the most representative feature subset based on the feature importance score to further optimize the recognition accuracy of the model and reduce the computational complexity.

8. The method for detecting dangerous goods based on deformable convolution according to claim 7, characterized in that: The multi-layer perceptron adopts a confidence-based decision-making mechanism when performing feature analysis and selection. The mechanism dynamically adjusts the decision threshold according to the correlation strength between the feature vector and the dangerous goods category to achieve adaptive recognition at different confidence levels and improve the robustness of the model in complex security inspection environments.

9. A dangerous goods detection device based on deformable convolution according to any one of claims 1 to 8, characterized in that: include: An image input unit, used for receiving an X-ray image to be detected; A preprocessing unit, used for preprocessing the input X-ray image; A feature extraction unit, used for extracting region enhancement features, color features and edge features; A feature fusion unit is used to fuse different features; An identification unit, used for outputting the identification result of dangerous goods according to the fused feature vector; The training unit is used to train the device through the ternary loss function to optimize the feature extraction and fusion process.

10. A storage medium, wherein the storage medium is used to store instructions, wherein the instructions are used to execute the steps of the dangerous goods detection method based on deformable convolution as described in any one of claims 1 to 8.

Images