A dangerous goods identification method and system based on a neural network

Abstract

The application relates to a dangerous goods identification method and identification system based on a neural network, in particular to a dangerous goods identification method and identification system based on a Cascade R-CNN two-stage neural network. In order to solve the problems that an X-ray security scanning image is relatively abstract, the identification accuracy of goods is low, long-term workers are prone to visual fatigue and misjudgment, and a data set cannot be fully utilized, a plurality of scanning images containing dangerous goods are collected, and the scanning images are divided into a labeled image set and an unlabeled image set; a network model is established, the labeled image set is input into the network model for training to obtain an initial network model; the unlabeled image set is input into the initial network model to obtain a pseudo-label file set; the labeled image set and the unlabeled image set are input into the initial network model to obtain a trained network model; an X-ray security scanning image to be identified is collected and input into the trained network model to obtain frame label information of dangerous goods. The application belongs to the field of computer image processing.

Description

Technical Field

[0001] This invention relates to a method and system for identifying dangerous goods, specifically to a method and system for identifying dangerous goods in security inspection images based on a Cascade R-CNN two-stage neural network, belonging to the field of computer image processing. Background Technology

[0002] In recent years, with economic and social progress, people have been traveling more and more frequently. Security checks are required when taking subways, trains, buses, and airplanes. The purpose of security checks is to remove all dangerous items that threaten the personal safety of passengers and ensure their safety. If the adequacy of security measures is not effectively guaranteed, it will threaten the lives of passengers and cause property damage. Therefore, it is very important to effectively identify dangerous items related to violence and terrorism in X-ray security images.

[0003] Current security checks mostly rely on X-ray scanners. However, the images produced by X-ray scanners are relatively abstract, easily causing visual fatigue and missed detections for long-term users. Therefore, improving the accuracy and performance of processing and analyzing these images has become a pressing issue. With the rapid development of computer vision, its application and research in security issues have attracted increasing attention from scholars. However, existing technologies are mostly applied to real-world objects such as photographs and remote sensing images. These algorithms perform poorly when applied to X-ray scanner images, resulting in underutilization of existing datasets. Summary of the Invention

[0004] To address the problems of abstract X-ray security scan images, low accuracy in item identification, visual fatigue and missed detections for long-term staff, and inefficient use of datasets, this invention proposes a dangerous goods identification method and system based on neural networks.

[0005] The technical solution adopted in this invention is:

[0006] A method for identifying hazardous materials based on neural networks includes the following steps:

[0007] S1. Collect multiple X-ray security scan images containing dangerous goods, select a portion of the collected X-ray security scan images and mark them with boxes to form an annotated image set, and the unselected X-ray security scan images form an unannotated image set;

[0008] S2. Establish a network model. Input the labeled image set in S1 into the network model for training until the loss converges, and obtain the trained initial network model.

[0009] S3. Input the unlabeled image set from S1 into the trained initial network model obtained in S2 for recognition, and obtain the pseudo-label file set corresponding to the unlabeled image set;

[0010] S4. Input the labeled image set in S1 and the unlabeled image set with pseudo-label files obtained in S3 into the trained initial network model obtained in S2 for training, and obtain the trained network model.

[0011] S5. Collect the X-ray security scan image to be identified and input it into the trained network model obtained in S4 to obtain the bounding box label information of the dangerous goods.

[0012] Preferably, the dangerous goods in S1 include items related to violence and terrorism, as well as flammable and explosive materials.

[0013] Preferably, the information labeled in the S5 frame includes the location coordinates of the dangerous goods in the X-ray security scan image, the type of dangerous goods, and the confidence level of the type.

[0014] Preferably, in step S2, a network model is established by inputting the labeled image set from step S1 into the network model for training until the loss converges, thus obtaining a trained initial network model. The specific process is as follows:

[0015] S21. Establish network models, including the ResNeXt-50_Cascade R-CNN network model and the Res2Net_Cascade R-CNN network model;

[0016] S22. Input the labeled image set in S1 into the ResNeXt-50_Cascade R-CNN network model for training until the loss converges, and obtain the trained ResNeXt-50_Cascade R-CNN network model.

[0017] Input the labeled image set in S1 into the Res2Net_Cascade R-CNN network model for training until the loss converges, and obtain the trained Res2Net_Cascade R-CNN network model.

[0018] S23. The output of the trained ResNeXt-50_Cascade R-CNN network model obtained in S22 is weighted and fused with the output of the trained Res2Net_Cascade R-CNN network model to obtain the trained initial network model.

[0019] Preferably, the ResNeXt-50_Cascade R-CNN network model in S21 includes ResNeXt-50 and Cascade R-CNN in sequence.

[0020] Preferably, in step S22, the labeled image set from S1 is input into the ResNeXt-50_Cascade R-CNN network model for training until the loss converges, resulting in a trained ResNeXt-50_Cascade R-CNN network model; the labeled image set from S1 is input into the Res2Net_Cascade R-CNN network model for training until the loss converges, resulting in a trained Res2Net_Cascade R-CNN network model. The specific process is as follows:

[0021] S221. Preprocess the labeled image set in S1 to obtain the preprocessed labeled image set;

[0022] S222. Input the preprocessed labeled image set from S221 into the ResNeXt-50_Cascade R-CNN network model for training to obtain the trained ResNeXt-50_Cascade R-CNN network model.

[0023] The preprocessed labeled image set in S221 is input into the Res2Net_Cascade R-CNN network model for training, resulting in the trained Res2Net_Cascade R-CNN network model.

[0024] Preferably, the method for preprocessing the labeled image set in S1 in S221 is an online data augmentation method.

[0025] Preferably, the online data augmentation method is one or more of the Mixup online data augmentation method, the Mosaic online data augmentation method, the AutoAugment online data augmentation method, and the GridMask online data augmentation method.

[0026] Preferably, the method of weighted fusion of the output of the trained ResNeXt-50_Cascade R-CNN network model obtained in S22 and the output of the trained Res2Net_Cascade R-CNN network model in S23 is the WBF model fusion method.

[0027] A neural network-based hazardous materials identification system is used to perform a neural network-based hazardous materials identification method.

[0028] Beneficial effects:

[0029] This invention constructs a fusion network model comprising a ResNeXt-50_Cascade R-CNN network model and a Res2Net_Cascade R-CNN network model. The two networks are fused using the Webof-Flight (WBF) model fusion method to achieve the identification and detection of hazardous materials in X-ray security scan images. The ResNeXt-50_Cascade R-CNN network model consists of ResNeXt-50 and Cascade R-CNN sequentially. The addition of the Res2Net_Cascade R-CNN network model is equivalent to adding an auxiliary layer to the ResNeXt-50_Cascade R-CNN network model, thereby improving the accuracy of hazardous material identification and detection in X-ray security scan images.

[0030] When detecting X-ray security scan images, this invention inputs the X-ray security scan image to be detected into the fusion network model of this invention. The input is then fed into both the ResNeXt-50_Cascade R-CNN and Res2Net_Cascade R-CNN network models within the fusion network model, and the results of the corresponding network models are output. The results from the ResNeXt-50_Cascade R-CNN and Res2Net_Cascade R-CNN network models are then weighted and fused to form the output of the fusion network model. This output includes the location coordinates of the hazardous material in the X-ray security scan image, the type of hazardous material, and the type confidence level. This not only reduces the problem of security personnel missing or misjudging hazardous materials in X-ray security scan images but also avoids visual fatigue and temporary loss of concentration caused by long working hours. Furthermore, it makes full use of datasets that cannot be labeled due to limitations in manpower and resources. Attached Figure Description

[0031] Figure 1 This is a flowchart of the present invention;

[0032] Figure 2 This is a diagram of the Cascade R-CNN architecture; Detailed Implementation

[0033] Specific implementation method one: Combining Figures 1-2 This embodiment describes a method for identifying hazardous materials based on neural networks, which includes the following steps:

[0034] S1. Collect multiple X-ray security scan images containing dangerous goods, select a portion of the collected X-ray security scan images and mark them with boxes to form an annotated image set, and the unselected X-ray security scan images form an unannotated image set;

[0035] A large number of X-ray security scan images were collected, requiring each image to include hazardous materials. Each X-ray security scan image represents the scanned image of an item passing through an X-ray security scanner. Hazardous materials include items related to violence and terrorism (such as controlled knives and firearms) and flammable and explosive materials (such as alcohol, lighters, and fireworks). Regions containing hazardous materials were manually located and extracted, and this data was used as a dataset to train the subsequent model. A portion of the X-ray security scan images collected in S1 were selected and manually labeled with bounding boxes to provide labeled data for model training. The remaining X-ray security scan images were used for semi-supervised training. The bounding box labels included the location coordinates of the hazardous materials in the X-ray security scan images, the type of hazardous material, and the type confidence level. The selected portion of the X-ray security scan images was 1 / 2, 2 / 3, or 3 / 4 of all the X-ray security scan images.

[0036] S2. Establish a network model. Input the labeled image set from S1 into the network model for training until the loss converges, thus obtaining the trained initial network model. The specific process is as follows:

[0037] This step involves using a set of labeled images to train the network model for the first time, giving the model a general shape.

[0038] S21. Establish network models, including the ResNeXt-50_Cascade R-CNN network model and the Res2Net_Cascade R-CNN network model;

[0039] The ResNeXt-50_Cascade R-CNN network model includes ResNeXt-50 and Cascade R-CNN, respectively.

[0040] The Cascade R-CNN network model uses ResNeXt-50 as its backbone. First, it obtains the ImageNet-1k pre-trained model of ResNeXt-50 to construct a convolutional deep learning network. Then, it merges the convolutional deep learning network constructed by ResNeXt-50 with the traditional Cascade R-CNN network model, using the output of ResNeXt-50 as the input of Cascade R-CNN to obtain the initial ResNeXt-50_Cascade R-CNN network model. The addition of the Res2Net_Cascade R-CNN network model is equivalent to adding an auxiliary to the ResNeXt-50_Cascade R-CNN network model, thereby improving the accuracy of dangerous goods identification and detection in X-ray security scan images.

[0041] The structural diagram of ResNeXt-50 is as follows:

[0042]

[0043]

[0044] Where params represents the number of model parameters and FLOPs represents the computational cost of the model.

[0045] S22. Input the labeled image set in S1 into the ResNeXt-50_Cascade R-CNN network model for training until the loss converges, and obtain the trained ResNeXt-50_Cascade R-CNN network model.

[0046] The labeled image set in S1 is input into the Res2Net_Cascade R-CNN network model for training until the loss converges, resulting in a trained Res2Net_Cascade R-CNN network model. The specific process is as follows:

[0047] S221. Preprocess the labeled image set in S1 to obtain the preprocessed labeled image set;

[0048] First, the labeled image set in S1 is input into the ResNeXt-50_Cascade R-CNN network model and the Res2Net_Cascade R-CNN network model, respectively. After input, both models preprocess the labeled image set to obtain the corresponding preprocessed labeled image set. The preprocessing method for the labeled image set is an online data augmentation method, which is one or more of the Mixup online data augmentation method, Mosaic online data augmentation method, AutoAugment online data augmentation method, and GridMask online data augmentation method. During preprocessing, both models automatically select an appropriate data augmentation method, and the selection probability of each of the above online data augmentation methods is 50%. The AutoAugment online data augmentation method includes methods such as horizontal rotation, vertical rotation, scaling, and RGB dithering of the image.

[0049] S222. Input the preprocessed labeled image set from S221 into the ResNeXt-50_Cascade R-CNN network model for training to obtain the trained ResNeXt-50_Cascade R-CNN network model.

[0050] The preprocessed labeled image set in S221 is input into the Res2Net_Cascade R-CNN network model for training, resulting in the trained Res2Net_Cascade R-CNN network model.

[0051] The preprocessed labeled image set was used as input to train the ResNeXt-50_Cascade R-CNN network model and the Res2Net_Cascade R-CNN network model, respectively, to obtain the trained ResNeXt-50_Cascade R-CNN network model and the trained Res2Net_Cascade R-CNN network model.

[0052] S23. The output of the trained ResNeXt-50_Cascade R-CNN network model obtained in S22 is weighted and fused with the output of the trained Res2Net_Cascade R-CNN network model to obtain the trained initial network model.

[0053] The fusion method is the WBF model fusion method.

[0054] The fusion of the ResNeXt-50_Cascade R-CNN and Res2Net_Cascade R-CNN network models effectively adds an auxiliary layer to the ResNeXt-50_Cascade R-CNN model. The Res2Net_Cascade R-CNN model also outputs results when predicting X-ray security scan images. These results are then weighted and fused to form a new, more accurate result for identifying hazardous materials in X-ray security scan images, thus improving the accuracy of hazardous material identification.

[0055] S3. Input the unlabeled image set from S1 into the trained initial network model obtained in S2 for recognition, and obtain the pseudo-label file set corresponding to the unlabeled image set;

[0056] The unlabeled image set in S1 is input into the pre-trained initial network model obtained in S2 for recognition and prediction, resulting in corresponding pseudo-label files. Pseudo-label samples with high confidence are selected, and finally, all selected pseudo-label files are integrated to form a pseudo-label file set. This step involves non-human annotation of the unlabeled image set.

[0057] S4. Input the labeled image set in S1 and the unlabeled image set with pseudo-label files obtained in S3 into the trained initial network model obtained in S2 for training, and obtain the trained network model.

[0058] This step involves a second training of the initial network model obtained in S2. By performing this second training, the accuracy of the model is increased.

[0059] S5. Collect the X-ray security scan image to be identified and input it into the trained network model obtained in S4 to obtain the bounding box label information of the dangerous goods.

[0060] Specific Implementation Method Two: Combining Figures 1-2 This embodiment describes a neural network-based hazardous materials identification system, which is used to execute a neural network-based hazardous materials identification method.

Claims

1. A method for identifying hazardous materials based on neural networks, characterized in that: It includes the following steps: S1. Collect multiple X-ray security scan images containing dangerous goods, select a portion of the collected X-ray security scan images and mark them with boxes to form a marked image set, and the unselected X-ray security scan images form an unmarked image set; the X-ray security scan image represents the scanned image of the item obtained when it passes through the X-ray security inspection machine; S2. Establish a network model. Input the labeled image set in S1 into the network model for training until the loss converges, and obtain the trained initial network model. S3. Input the unlabeled image set from S1 into the trained initial network model obtained in S2 for recognition, and obtain the pseudo-label file set corresponding to the unlabeled image set; S4. Input the labeled image set in S1 and the unlabeled image set with pseudo-label files obtained in S3 into the trained initial network model obtained in S2 for training, and obtain the trained network model. S5. Collect the X-ray security scan image to be identified and input it into the trained network model obtained in S4 to obtain the bounding box labeling information of the dangerous goods. In step S2, a network model is established. The labeled image set from S1 is input into the network model for training until the loss converges, resulting in a trained initial network model. The specific process is as follows: S21. Establish network models, including ResNeXt-50_Cascade R-CNN network model and Res2Net_Cascade R-CNN network model; S22. Input the labeled image set in S1 into the ResNeXt-50_Cascade R-CNN network model for training until the loss converges, and obtain the trained ResNeXt-50_Cascade R-CNN network model. Input the labeled image set in S1 into the Res2Net_Cascade R-CNN network model for training until the loss converges, and obtain the trained Res2Net_Cascade R-CNN network model. S23. The output of the trained ResNeXt-50_Cascade R-CNN network model obtained in S22 is weighted and fused with the output of the trained Res2Net_Cascade R-CNN network model to obtain the trained initial network model. The ResNeXt-50_Cascade R-CNN network model in S21 includes ResNeXt-50 and Cascade R-CNN in sequence. In step S22, the labeled image set from S1 is input into the ResNeXt-50_Cascade R-CNN network model for training until the loss converges, resulting in a trained ResNeXt-50_Cascade R-CNN network model; the labeled image set from S1 is input into the Res2Net_Cascade R-CNN network model for training until the loss converges, resulting in a trained Res2Net_Cascade R-CNN network model. The specific process is as follows: S221. Preprocess the labeled image set in S1 to obtain the preprocessed labeled image set; S222. Input the preprocessed labeled image set from S221 into the ResNeXt-50_Cascade R-CNN network model for training to obtain the trained ResNeXt-50_Cascade R-CNN network model. The preprocessed labeled image set in S221 is input into the Res2Net_Cascade R-CNN network model for training, resulting in the trained Res2Net_Cascade R-CNN network model. The method for preprocessing the labeled image set in S1 in S221 is an online data augmentation method.

2. The method for identifying hazardous materials based on neural networks according to claim 1, characterized in that: The dangerous goods in S1 include items related to violence and terrorism, as well as flammable and explosive materials.

3. The method for identifying hazardous materials based on neural networks according to claim 2, characterized in that: The information labeled in the S5 frame includes the location coordinates of the dangerous goods in the X-ray security scan image, the type of dangerous goods, and the confidence level of the type.

4. The method for identifying hazardous materials based on neural networks according to claim 3, characterized in that: The online data augmentation method is one or more of the following: Mixup online data augmentation method, Mosaic online data augmentation method, AutoAugment online data augmentation method, and GridMask online data augmentation method.

5. The method for identifying hazardous materials based on neural networks according to claim 4, characterized in that: The method in S23 that weights and fuses the output of the trained ResNeXt-50_Cascade R-CNN network model obtained in S22 with the output of the trained Res2Net_Cascade R-CNN network model is the WBF model fusion method.

6. A hazardous materials identification system based on neural networks, characterized in that: The system is used to execute any one of the neural network-based hazardous materials identification methods according to claims 1 to 5.

Images