Robust X-ray image target detection method based on similar target fusion data augmentation

By adopting similar target fusion data augmentation and large loss suppression mechanisms in X-ray security image object detection, the problem of noise data is solved, the robustness and detection accuracy of the model are improved, and it is suitable for X-ray and general object detection tasks. .

CN118968075AActive Publication Date: 2024-11-15XIAMEN UNIV
View PDF 14 Cites 0 Cited by

Patent Information

Application Number
CN202410996894.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-15
Estimated Expiration
2044-07-24

AI Technical Summary

Technical Problem

The prior art is difficult to effectively process noise data in X-ray security image object detection, which affects the model training accuracy. Especially in the case of overlapping objects in X-ray images, it is very difficult to obtain accurate label data sets.

Method used

Using a robust X-ray image object detection method based on the augmented data of similar target fusion, the use of edge smoothing mask algorithm to fuse multiple similar target images, combining large loss suppression mechanism and deep learning object detector, Training the model to reduce the noise impact.

Benefits of technology

Significantly reduces the impact of label noise on model training, simulates overlapping occlusion in X-ray images, enables the model to better learn image features, improves detection accuracy, and performs well on multiple datasets, including X-ray and general Object detection task.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968075A_ABST
    Figure CN118968075A_ABST
Patent Text Reader

Abstract

The invention discloses a robust X-ray image target detection method based on similar target fusion data augmentation, and the method is mainly based on the condition that a working data set contains noise in a label, and can effectively eliminate the influence of the label noise on the training of an X-ray security inspection image target detector. And the overlapping and shielding condition in the X-ray image is well simulated through image fusion, so that the model can better learn the inherent characteristics of the X-ray image. According to the scheme, good performance is achieved on a plurality of public data sets, and compared with a traditional noise label learning method, the scheme is a scheme which is more flexible and meets the actual requirement for X-ray security check image target detection under the noise label condition. In addition, experimental results show that the method is effective in the X-ray data set and also has a good effect in some general target detection data sets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a robust X-ray image target detection method based on similar target fusion data augmentation. Background Art

[0002] Object detection is an important problem in computer vision. It refers to identifying which areas in an image are target objects and giving the location, size and category of these target objects. Object detection is widely used in computer vision applications, such as self-driving cars, drones, security monitoring, etc. In the application scenarios of security monitoring, there is a type of scene whose images are X-ray images, such as the images output by security inspection instruments at airports, train stations, and subway stations. This requires object detection of X-ray images to solve the automatic detection task of dangerous goods during security inspections.

[0003] With the development of deep learning in recent years, more and more computer vision applications have begun to use deep learning-based methods, and X-ray security image detection has also entered the era of deep learning. Early X-ray image target detection used methods based on manual features, which did not work well. Since statistical deep learning requires a large amount of data for training, but there was no large data set specifically for X-ray security at the time, convolutional neural networks could not be well applied to this field at first, but the emergence of transfer learning methods later changed this situation. In 2016, Akcay et al. ( S, Kundegorski ME, Devereux M, et al. Transfer learning using convolutional neural networks for object classification within x-ray baggage security imagery [C] / / 2016 IEEE International Conference on Image Processing (ICIP). IEEE, 2016: 1057-1061.) In order to overcome the problem of lack of data, transfer learning was used for the first time in the field of X-ray security image detection. The parameters of the convolutional layer and the fully connected layer were fine-tuned based on AlexNet. This method achieved better results than previous methods. This is the first study of deep convolutional neural networks in X-ray security image detection. Since then, more and more deep learning-based methods have been applied in this field. Miao et al. (Miao C, Xie L, Wan F, et al. Sixray: A large-scale security inspection x-ray benchmark for prohibited item discovery in overlapping images [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019: 2119-2128.) proposed a large-scale X-ray security inspection dataset SIXray. They also used a class-balanced hierarchical refinement method. Experiments show that this method can effectively solve the problem of imbalance between positive and negative samples. At the same time, they also designed a class-balanced loss function to alleviate the noise introduced by easy negative samples. Wei (Wei Y, Tao R, Wu Z, et al. Occluded Prohibited Items Detection: An X-ray Security Inspection Benchmark and De-occlusion Attention Module [C]. ACM Multimedia 2020.) et al. proposed a high-quality dataset OPIXray designed for target detection tasks in security inspection scenarios. All dangerous goods contained in the images in this dataset are manually labeled by professional security inspectors at an airport.In addition, the paper also proposed a method for detecting occluded prohibited items in security inspection scenarios. The core of this method uses the attention mechanism to enhance the edge information and material information of prohibited items in X-ray images. However, most of the above methods rely on a large-scale dataset for model training. However, due to the serious overlap and occlusion of objects in X-ray images, the annotation of the dataset is more difficult than other tasks. Therefore, it is very difficult to obtain a large-scale X-ray image dataset with completely accurate labels. In fact, most of the current X-ray datasets have more or less noise problems, including inaccurate target bounding boxes and incorrect target categories. When the model is trained on a noisy dataset, the training accuracy may be seriously affected by these noisy data. Therefore, it is very necessary to design a noise-robust X-ray contraband detection method. Summary of the invention

[0004] In view of this, the purpose of the present invention is to propose a robust X-ray image target detection method based on similar target fusion data augmentation, which is reliable, versatile and noise-resistant.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] A robust X-ray image target detection method based on similar target fusion data augmentation includes the following steps:

[0007] A. Prepare a dataset of X-ray security inspection images and divide it into a training set and a test set, where the image data of the training set and the test set both have category labels and target bounding box labels pointing to the target objects;

[0008] B. Add label noise to the image data of the training set to simulate the noise situation. The noise can be added manually, including category label noise and target bounding box label noise;

[0009] C. For each target object T in each image data in the training set, find K-1 images with the same category label from the image data in the dataset, then crop the target object according to the target bounding box label corresponding to the image data, and then convert the cropped image resolution to the same resolution as the bounding box of the target object T;

[0010] D. Use edge smoothing mask algorithm to paste the cropped target object image back to the original position of the original image data;

[0011] E. Send the image data of the training set processed in step D to the target detector based on deep learning for model training;

[0012] F. Use a preset large loss suppression mechanism in the loss function of the target detector corresponding model to suppress the large loss caused by label errors in image data, so as to alleviate the negative impact of label noise on model training;

[0013] G. Use the back propagation algorithm to perform end-to-end model training, and after the model passes the test set test, obtain the trained model;

[0014] H. Use the trained model to perform X-ray image target detection and output the detection results through the model.

[0015] As a possible implementation, further, in step A of this solution, the prepared X-ray security inspection image datasets include OPIXray, PIDray and MS-COCO.

[0016] The OPIXray dataset used in this scheme is an X-ray image dataset that includes folding knife, straight knife, scissor, utility knife and multi-tool knife. It consists of 8885 X-ray images, of which 7109 images are divided into training sets and the remaining 1776 images are divided into test sets.

[0017] The PIDray dataset is a large X-ray dataset containing 47,677 images. The dataset contains 12 different categories, of which 29,457 images are used for training and 18,220 images are used for testing.

[0018] The MS-COCO dataset is a large-scale public general object detection dataset, which contains 135,000 training images and 5,000 test images. The dataset contains 80 different categories. This solution uses two X-ray datasets, OPIXray and PIDray, to test the performance of the proposed method on the X-ray contraband detection task under noisy labels, and uses MS-COCO to verify that the proposed method also has good generalization performance on other tasks.

[0019] Since it is difficult to correctly determine the noise level of a data set, it is also difficult to find a data set with a specific noise rate. In order to test the performance of this solution under different noise rates, as a preferred implementation, preferably, in step B of this solution, label noise is added to the image data of the training set to simulate the noise situation, including: adding label noise to the category label and target bounding box label of the target object respectively;

[0020] The method of adding label noise to the category label is: replacing the original category label of the image data with any other category label in the data set with a preset probability, and the preset probability is the category noise rate of the data set;

[0021] The method of adding label noise to the target bounding box label is as follows: offset and scale the original bounding box of the image data with a preset probability, and the preset probability is the target bounding box noise rate of the data set; set the coordinates (x, y, w, h) as the bounding box of the image data, and perform the following operations with the preset probability:

[0022]

[0023] Among them, Δ x , Δ y , Δ w , Δ h It is randomly sampled from a uniform distribution U(-δ,δ), where δ represents the level of noise disturbance, and x, y, w, h are the bounding box coordinates pointing to the target object in the image data. are the bounding box coordinates after adding noise; as an example, δ=0.3 can be set.

[0024] As a preferred implementation mode, preferably, step C of this solution includes: for each target object T in each image data in the training set, the label corresponding to the target object T contains a target bounding box label pointing to the target object position information and a category label of the target object category information; according to the category information provided by the label, find K-1 images of the target object with the same category information from the original data set, and cut out the target object in the image data according to its corresponding target bounding box label and put it into the result set;

[0025] Among them, each time a picture data is randomly selected from the data set, if the picture data contains a target object with the same category information as the target object T in the corresponding label, the target is cropped out according to its corresponding target bounding box label, and then the image resolution of the cropped target object is changed to the same resolution as the target object T and added to the result set, otherwise, a picture data is randomly selected from the data set again until the result set contains K-1 targets with the same resolution.

[0026] As a preferred embodiment, preferably, step D of this solution includes:

[0027] For all target objects in the result set and their corresponding original target objects T, they are fused into the same image to increase the probability that the target object with the corresponding category label appears in the target bounding box, so as to reduce the probability that the target object with the corresponding category label does not exist in the target bounding box due to noise; in order to make the fusion smoother and obtain a more natural final result, the image data is fused through the edge smoothing mask algorithm, and the corresponding calculation formula is:

[0028]

[0029] Among them, d i,j is the distance between the pixel at position (i, j) in the target bounding box and the nearest edge of the bounding box, and its value is min(i, j, h o -i,w o -j), h o ,w o are the height and width of the cropped target, β is a threshold to control the range of the smooth boundary, and λ is a number randomly sampled from the Beta distribution;

[0030] Use edge smoothing mask to fuse the images in the result set. The fusion formula is:

[0031]

[0032] Among them, α is the edge smoothing mask, B a is the image block corresponding to the original target object T, B n is the image block in the result set, K is the total number of fused images, is element-wise multiplication;

[0033] Finally, the fused image is pasted back to the location of the target object T in the original image data to complete the fusion process.

[0034] Through the above image fusion process, the noise rate of category noise and bounding box noise can be significantly reduced. Assuming that in a data set with a noise rate of P, there is a label of category C, then the probability that this bounding box label actually contains an object of category C is 1-P. When K target image blocks of the same category are fused at this time, the probability that the fused bounding box contains an object becomes 1-P. K , because P is a number between 0 and 1, the probability of containing an object in the bounding box will be greatly improved.

[0035] As a preferred implementation method, preferably, in step E of this scheme, the deep learning-based target detector uses Faster RCNN as the basic target detection network, and its core parts include three parts: basic feature extraction network, region proposal network, and region classification and boundary regression network, which are respectively used for feature extraction, selecting candidate regions from feature maps, and classifying and regressing bounding boxes for each candidate region. Faster RCNN achieves efficient target detection through end-to-end training, and it has achieved significant improvements in speed and accuracy compared to the traditional two-stage method. Although the present invention mainly uses Faster RCNN for experiments, since this method is mainly modified at the image level, it can also be applied to other target detection networks according to task requirements.

[0036] After the method fuses multiple targets in step E, due to the influence of noise, multiple targets of different categories may appear in one position, but they share a label. Even if the model predicts all the targets that appear in the image, other types of targets that appear due to noise will be regarded as wrong predictions and suppressed because they do not have correct labels, even if the prediction is correct, resulting in a decrease in model performance. In order to alleviate the impact of such "false error" predictions on model training, as a preferred implementation method, preferably, step F of this solution includes:

[0037] All predictions of the target detector corresponding model are divided into 4 categories, namely:

[0038] (1) PB neg : The IoU between the model predicted target bounding box and the label bounding box is less than the threshold;

[0039] (2) PB pp : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, but the predicted category is different from the label category and is not background;

[0040] (3) PB fb : The IoU between the model predicted target bounding box and the label bounding box is greater than the threshold, and the predicted category is background;

[0041] (4) PB pos : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, and the predicted category is the same as the label;

[0042] Among them, the "false error" prediction is mainly PB pp The prediction results included in this part are discarded from the loss function calculation, and only the classification losses of the other three parts are calculated. Finally, the total loss function becomes:

[0043] L=L bbox +Lclsneg +L clspos +L clsfb

[0044] Among them, L bbox Represents the regression loss, without changing it, L clsneg , L clspos , L clsfb Respectively represent the above PB neg , P.B. pos ,PB fb The corresponding classification loss.

[0045] As a preferred implementation mode, preferably, step G of this scheme also includes preprocessing the data, which includes: using random flipping data enhancement to normalize the data of all training sets and test sets; the basic feature extraction network is initialized with ResNet pre-trained on ImageNet to start training, and random gradient descent is used as the optimizer, wherein the initial learning rate is 0.0025, and in the 17th and 21st rounds of training, the learning rate is reduced by 10 times, the weight decay parameter is 0.0001, and the momentum parameter is 0.9; the mini-batch of each iteration is set to 2, and the entire network is trained for 24 rounds; during the training process, the preset large loss suppression mechanism is used to modify the loss function, and after calculating the loss, the gradient descent algorithm is used for back propagation to update the model parameters.

[0046] As a preferred implementation method, preferably, step H of this scheme includes: converting the resolution of the input image to be detected to 1333*800, and then inputting it into a trained model for X-ray security inspection image target detection, and inputting the extracted features into the regression network through the model to obtain the detection bounding box coordinate value, and then inputting it into the classification network to predict its corresponding category to generate a detection result.

[0047] By adopting the above-mentioned technical scheme, compared with the prior art, the present invention has the following beneficial effects: the method of this scheme can effectively eliminate the influence of label noise on the training of X-ray security image target detector, and through image fusion, it can well simulate the overlapping occlusion in X-ray images, so that the model can better learn the inherent characteristics of X-ray images. This scheme not only achieves good performance on multiple public data sets, but also is a more flexible and practical solution to X-ray security image target detection under noisy labels compared to traditional noise label learning methods. In addition, experimental results show that the method of this scheme is not only effective in X-ray data sets, but also has good results in some general target detection data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 The figure is a flowchart of the entire implementation of the embodiment of the present invention.

[0050] Figure 2 FIG. 4 is a diagram of the entire network structure of an embodiment of the present invention.

[0051] Figure 3 Some test results of the same model (Faster RCNN) before and after using the solution of the present invention on the test set of the OPIXray dataset. Both models are trained at the same noise rate (60% bounding box noise and category noise). Figure 3 (a) Detection results of the original Faster RCNN model. The blue box is the true category label and bounding box, the red box is the predicted label and bounding box, and the prediction confidence is in brackets; Figure 3 (b) is the detection result of the Faster RCNN model after using this method. It can be seen that the quality of the predicted bounding box and the accuracy of the predicted category have been greatly improved after using this method.

[0052] Figure 4 It is a brief illustration of using the area enclosed by the PR curve as a measurement indicator for average precision. DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1 As shown, the robust X-ray image target detection method based on similar target fusion data augmentation in this embodiment includes the following steps:

[0055] A. Prepare a dataset of X-ray security images and divide it into a training set and a test set. The image data of both the training set and the test set have category labels and target bounding box labels pointing to the target object. The training set used is 2 commonly used datasets for X-ray security image target detection and a general target detection dataset. The X-ray security image target detection datasets are OPIXray and PIDray, and the general target detection dataset is MS-COCO. OPIXray is an X-ray image dataset that includes folding knife, straight knife, scissor, utility knife and multi-tool knife. It consists of 8885 X-ray images, of which 7109 images are divided into training sets and the remaining 1776 images are divided into test sets. PIDray is a larger X-ray dataset containing 47677 images. The dataset contains 12 different categories, of which 29457 images are used for training and 18220 images are used for testing. MS-COCO is a large-scale public general object detection dataset, which contains 135,000 training images and 5,000 test images, and contains 80 different categories. This solution uses two X-ray datasets, OPIXray and PIDray, to test the performance of the method of this solution on the X-ray contraband detection task under noisy labels, and uses MS-COCO as a test set to verify that the invention also has good generalization performance on other tasks.

[0056] B. To simulate the situation where noise occurs, some noise is artificially added to the training data labels in the dataset.

[0057] Since it is difficult to correctly derive the noise level of a certain data set, it is also difficult to find a data set with a specific noise rate. In order to test the performance of the present invention under different noise rates, this scheme directly adds noise of a specific noise rate to the original label of the data set. Since the label of the target detection task contains two parts, namely the target bounding box and the target category, noise needs to be added to these two parts separately. The specific operation is that for category noise, this scheme replaces the original category label in the data set with the label of any other category in the data set with a certain probability, and this probability is the category noise rate of the data set. For bounding box noise, this scheme offsets and scales the original bounding box in the data set with a certain probability, and this probability is the target bounding box noise rate of the data set. Specifically, for a bounding box with coordinates (x, y, w, h), this scheme performs the following operations with a certain probability:

[0058]

[0059] Among them, Δ x , Δy , Δ w , Δ h It is randomly sampled from a uniform distribution U(-δ,δ), where δ represents the disturbance level of the noise. As an example, this scheme sets δ=0.3 in all experiments.

[0060] C. For each target T in each image in the training dataset, find K-1 targets with the same category label from the dataset, crop the target according to the target bounding box provided by the label, and convert the cropped image to the same resolution as the bounding box of target T.

[0061] For each target T in the training image, the label corresponding to the target will contain the location information and category information of the target. This scheme finds K-1 images with the same category of targets from the original dataset based on the category information provided by the label, and crops the targets according to their corresponding target bounding box labels and puts them into the result set. Specifically, this scheme randomly selects a picture from the dataset each time. If the picture contains a target of the same category as in label A, the target is cropped according to its corresponding target bounding box label, and the resolution of the cropped target picture is changed to the same as that of target T and added to the result set. Otherwise, a picture is randomly selected from the dataset again until the result set contains K-1 targets of the same resolution.

[0062] D. Combine Figure 2 As shown, an edge smoothing mask algorithm is used to fuse the above images and paste them back to their original positions in the original images.

[0063] For all the targets in the result set and the original target T, this solution fuses them into the same image to increase the probability of the corresponding category targets appearing in the target bounding box and reduce the probability that the corresponding category targets do not exist in the target bounding box due to noise. In order to make the fusion smoother and the final result looks more natural, this method invents an edge smoothing mask algorithm to fuse the images. The edge smoothing mask calculation formula is:

[0064]

[0065] where d i,j is the distance between the pixel at position (i, j) in the target bounding box and the nearest edge of the bounding box, and its value is min(i, j, h o -i,w o -j), h o ,w o are the height and width of the cropped target, β is a threshold to control the range of the smooth boundary, and λ is a number randomly sampled from the Beta distribution.

[0066] Based on the above, this scheme uses edge smoothing mask to fuse the images in the result set. The fusion formula is:

[0067]

[0068] Among them, α is the edge smoothing mask, B a is the image block corresponding to the original target T, B n is the image block in the result set, K is the total number of fused images, is the element-by-element multiplication. Finally, the fused image is pasted back to the location of the target T in the original image, and the fusion process is completed.

[0069] Through the above image fusion process, the noise rate of category noise and bounding box noise can be significantly reduced. Assuming that in a data set with a noise rate of P, there is a label of category C, then the probability that this bounding box label actually contains an object of category C is 1-P. When K target image blocks of the same category are fused at this time, the probability that the fused bounding box contains an object becomes 1-P. K , because P is a number between 0 and 1, the probability of containing an object in the bounding box will be greatly improved.

[0070] E. Send the processed image to the deep learning based object detector.

[0071] This method uses Faster RCNN as the basic target detection network. Faster RCNN is a classic two-stage target detection method. Its core parts include three parts: basic feature extraction network, region proposal network and region classification and boundary regression network, which are used for feature extraction, selecting candidate regions from feature maps and classifying and regressing bounding boxes for each candidate region. Faster RCNN achieves efficient target detection through end-to-end training. Compared with the traditional two-stage method, it has achieved significant improvements in speed and accuracy. Although the present invention mainly uses Faster RCNN for experiments, since this method is mainly modified at the image level, it can also be applied to other target detection networks according to task requirements.

[0072] F. Combination Figure 2 As shown in Figure 1, a large loss suppression mechanism is used in the loss function to suppress some large losses that may be caused by label errors, so as to alleviate the negative impact of label noise on model training.

[0073] After this method fuses multiple targets in the result step E, due to the influence of noise, multiple targets of different categories may appear in one position, but they share the same label. Even if the model predicts all the targets that appear in the image, other types of targets that appear due to noise will be regarded as wrong predictions and suppressed even if the prediction is correct because they do not have the correct labels, resulting in a decrease in model performance. In order to alleviate the impact of this "false error" prediction on model training, this method designs a large loss suppression mechanism to modify the loss function. Specifically, this method divides all the predictions of the model into 4 categories, namely, PB neg : The IoU between the model-predicted target bounding box and the label bounding box is less than the threshold; PB pp : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, but the predicted category is different from the label category and is not background; PB fb : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, and the predicted category is background; PB pos :The model predicts that the IoU between the target bounding box and the label bounding box is greater than the threshold, and the predicted category is the same as the label; the "false error" prediction mentioned above is mainly PB pp Therefore, this method discards this part of the prediction results from the loss function calculation and only calculates the classification loss of the other three parts. Finally, the total loss function becomes:

[0074] L=L bbox +L clsneg +L clspos +L clsfb

[0075] Among them, L bbox Represents the regression loss, which is not changed in this scheme. L clsneg , L clspos , L clsfb Respectively represent the above PB neg , P.B. pos ,PB fb The corresponding classification loss.

[0076] G. Use the back propagation algorithm to perform end-to-end training to obtain a trained model.

[0077] During the training process, for data preprocessing, in addition to the methods mentioned above, this scheme uses random flipping data augmentation and normalizes all training and test data. The feature extraction network of this scheme uses the ResNet pre-trained on ImageNet to initialize the training, and uses stochastic gradient descent as the optimizer. The initial learning rate is 0.0025. In the 17th and 21st rounds of training, this scheme reduces the learning rate by 10 times, the weight decay parameter is 0.0001, and the momentum parameter is 0.9. The mini-batch of each iteration is set to 2, and the entire network is trained for 24 rounds. During the training process, this scheme uses the above-mentioned large loss suppression to modify the loss function. After calculating the loss, the gradient descent algorithm is used for back propagation to update the model parameters.

[0078] In order to test the detection method of this scheme, this embodiment compares the detection method of this scheme with the traditional Faster RCNN detection model, and uses the OPIXray dataset as the test set for test comparison. Both models are trained under the same noise rate (60% bounding box noise and category noise). The results are as follows: Figure 3 As shown, it shows the detection results of the proposed model and the traditional Faster RCNN model for X-ray images. It can be seen that Figure 3 In the figure, (a) shows the detection result of the original Faster RCNN model, the blue box is the true category label and bounding box, the red box is the predicted label and bounding box, and the one in brackets is the prediction confidence. Figure 3 In (b), the detection result of the Faster RCNN model after using this method is shown. It can be seen that the quality of the predicted bounding box and the accuracy of the predicted category are greatly improved after using this method.

[0079] In addition, this scheme also compares the performance of this implementation method with several X-ray image target detection methods recorded in existing literature on the OP IXray dataset with 60% class label noise and 60% target bounding box offset noise added. The results are shown in Table 1;

[0080] method Accuracy (mAP@.5) Training time (24 rounds) Speed ​​(fps) Faster RCNN 56.7 4h 27min 18.7 LIM 72.7 19h 18min 7.3 SDANet 52.4 6h 20min 16.2 SCE 48.6 4h 28min 19.3 LNCIS 65.9 4h 30min 19.3 OA-MIL 56.4 4h 33min 19.4 This embodiment scheme 81.8 4h 30min 19.0

[0081] Remark:

[0082] Faster RCNN corresponds to the method proposed by S. Ren et al. (S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: Towards real time object detection with region proposal networks," in Adv. Neural Inform. Process. Syst., 2015.)

[0083] LIM corresponds to the method proposed by R.Tao et al. (R.Tao, Y.Wei,

[0084] SDANet corresponds to the method proposed by L. Zhang et al. (L. Zhang, L. Jiang, R. Ji, and H. Fan, "PIDray: A large-scale X-ray benchmark for real-world prohibited item detection," in Proc. Int. J. Comput. Vis., 2023.)

[0085] SCE corresponds to the method proposed by Y. Wang et al. (Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey, "Symmetric cross entropy for robust learning with noisy labels," in Proc. Int. Conf. Comput. Vis., 2019.)

[0086] LNCIS corresponds to the method proposed by L. Yang et al. (L. Yang, F. Meng, H. Li, Q. Wu, and Q. Cheng, "Learning with noisy class labels for instance segmentation," in Proc. Eur. Conf. Comput. Vis., 2020.)

[0087] OA-MIL corresponds to the method proposed by C. Liu et al. (C. Liu, K. Wang, H. Lu, Z. Cao, and Z. Zhang, "Robust object detection with inaccurate bounding boxes," in Proc. Eur. Conf. Comput. Vis., 2022.)

[0088] The training time is the time required for 24 rounds of training on the training set of the OP IXray dataset. The model speed is in fps (frames per second), which is the number of pictures that the model can process per second. The larger the value, the faster the speed. The precision unit is mAP@0.5 (mean average precision), which means the average precision when the IoU threshold is 0.5. This indicator combines the two indicators of precision and recall, IoU (Intersection over Union) calculation.

[0089] The IoU value of two rectangular boxes is the area of ​​the intersection of the two boxes divided by the area of ​​the union, which is used to measure the quality of the predicted box. The calculation formula is:

[0090]

[0091] The calculation method of precision and recall is:

[0092] First, all prediction boxes are divided into True Positive (TP): the number of prediction boxes whose IoU with the ground truth box is greater than the threshold (0.5) and the prediction score is greater than the threshold, and each Ground Truth label is only calculated once. When the IoU of multiple prediction boxes with the Ground Truth is greater than the threshold, the value with the largest prediction score is taken.

[0093] False Positive (FP): The number of predicted boxes with an IoU less than the threshold (0.5) and the number of redundant detection boxes that detect the same Ground Truth.

[0094] False Negative (FN): The number of ground truths that were not detected.

[0095] True Negative (TN, not included in the calculation): The number of prediction boxes whose IoU is less than the threshold and whose predicted label is background. Since negative classes will not be labeled in the experiment, this item is not of concern and can be ignored.

[0096]

[0097] Precision is the accuracy, which is used to judge whether the model's predictions are accurate (precision), and recall is the recall rate, which is used to judge whether the model can find all objects (recall rate). It is difficult to judge the performance of the model using only one of the indicators. For example, as long as the model does not predict anything, the precision rate is 1; similarly, as long as the model predicts enough prediction boxes, even if most of them are wrong predictions, the recall rate can also be 1. Therefore, this solution uses the average precision AP to test the performance of the model. This indicator combines the precision rate and the recall rate.

[0098] The average precision is shown by the PR curve (ref. Figure 4 ) is used as a measurement indicator. By continuously modifying the confidence threshold, multiple recall and precision values ​​are obtained, and then these points are connected by a straight line to obtain the PR curve. The area enclosed by the PR curve and the coordinate axis is the AP value. In the actual calculation, the jagged curve is usually smoothed first, and the precision corresponding to each recall rate is replaced by the maximum precision on its right, so the final smoothed curve will be monotonically decreasing. The average precision AP calculated in this way will be less sensitive to small changes. The figure below is an example of a PR curve. The green curve is the curve after smoothing, and the orange curve is the curve before smoothing. This scheme divides the recall rate value from 0 to 1 into 11 points (0, 0.1, 0.2, ..., 1.0), and then calculates the average value of the precision corresponding to these points (the corresponding values ​​on the smoothed curve) as the AP: (AP r (0) represents the precision value when the recall rate is 0)

[0099]

[0100] mAP is the average value of the AP value of each category. By measuring mAP, the recall and precision of the model are comprehensively considered, which can better reflect the performance of the model.

[0101] This method is mainly based on the situation that the working data set contains noise in the label. In this case, the model training performance is seriously affected by the noise. In order to alleviate the impact of noisy labels on model training, this scheme proposes a novel data enhancement technology, which enables X-ray security image target detection to achieve relatively good performance even in the face of relatively large noise, and greatly improves the robustness of the model without modifying the complexity of the model. Specifically, due to the influence of noisy labels, there may be no correct target at the target position in the image corresponding to the label. In order to alleviate this problem, this scheme greatly increases the probability of the target object appearing at the label position in the image by fusing multiple target image blocks of the same category during the training phase. In addition, the image fusion operation can also simulate the object occlusion characteristics of X-ray security images, so that the model can better learn the occlusion characteristics of X-ray images; at the same time, this scheme also proposes a large loss suppression strategy. In the above fusion process, due to the influence of category noise, the fusion result may contain some other targets, and these targets do not have corresponding correct labels. Therefore, even if the model makes correct predictions for these targets, they will be considered as wrong predictions and suppressed in the loss calculation phase, which will have a negative impact on model training. Therefore, this solution alleviates the negative impact of these predictions on model training by removing the loss calculation for this part of the predictions during loss calculation. The noise-resistant X-ray security inspection image target detection method proposed in this invention has achieved excellent results on the X-ray security inspection image dataset after noise addition. Even the method of this solution has achieved good results on the general target detection dataset, which proves the excellence and versatility of the method of this solution.

[0102] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A robust X-ray image target detection method based on similar target fusion data augmentation, characterized in that: It includes the following steps: A. Prepare a dataset of X-ray security inspection images and divide it into a training set and a test set, where the image data of the training set and the test set both have category labels and target bounding box labels pointing to the target objects; B. Add label noise to the image data of the training set to simulate the noise situation; C. For each target object T in each image data in the training set, find K-1 image data with the same category label from the image data in the dataset, then crop the target object according to the target bounding box label corresponding to the image data, and then convert the cropped image resolution to the same resolution as the bounding box of the target object T; D. Use edge smoothing mask algorithm to paste the cropped target object image back to the original position of the original image data; E. Send the image data of the training set processed in step D to the target detector based on deep learning for model training; F. Use a preset large loss suppression mechanism in the loss function of the target detector corresponding model to suppress the large loss caused by label errors in image data, so as to alleviate the negative impact of label noise on model training; G. Use the back propagation algorithm to perform end-to-end model training, and after the model passes the test set test, obtain the trained model; H. Use the trained model to perform X-ray image target detection and output the detection results through the model.

2. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 1, characterized in that: In step A, the prepared X-ray security inspection image datasets include OPIXray, PIDray, and MS-COCO.

3. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 1 or 2, characterized in that: In step B, label noise is added to the image data of the training set to simulate the noise condition, including: adding label noise to the category label and the target bounding box label of the target object respectively; The method of adding label noise to the category label is: replacing the original category label of the image data with any other category label in the data set with a preset probability, and the preset probability is the category noise rate of the data set; The method of adding label noise to the target bounding box label is as follows: offset and scale the original bounding box of the image data with a preset probability, and the preset probability is the target bounding box noise rate of the data set; set the coordinates (x, y, w, h) as the bounding box of the image data, and perform the following operations with the preset probability: Among them, Δ x , Δ y , Δ w , Δ h It is randomly sampled from a uniform distribution U(-δ,δ), where δ represents the level of noise disturbance, and x, y, w, h are the bounding box coordinates pointing to the target object in the image data. are the bounding box coordinates after adding noise.

4. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 3, characterized in that: Step C includes: for each target object T in each image data in the training set, the label corresponding to the target object T contains a target bounding box label pointing to the target object position information and a category label of the target object category information; according to the category information provided by the label, find K-1 images of the target object with the same category information from the original data set, and cut out the target object in the image data according to its corresponding target bounding box label and put it into the result set; Among them, each time a picture data is randomly selected from the data set, if the picture data contains a target object with the same category information as the target object T in the corresponding label, the target is cropped out according to its corresponding target bounding box label, and then the image resolution of the cropped target object is changed to the same resolution as the target object T and added to the result set, otherwise, a picture data is randomly selected from the data set again until the result set contains K-1 targets with the same resolution.

5. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 4, characterized in that: Step D includes: For all target objects in the result set and their corresponding original target objects T, they are fused into the same image to increase the probability that the target object with the corresponding category label appears in the target bounding box, so as to reduce the probability that the target object with the corresponding category label does not exist in the target bounding box due to noise; in order to make the fusion smoother and obtain a more natural final result, the image data is fused through the edge smoothing mask algorithm, and the corresponding calculation formula is: Among them, d i,j is the distance between the pixel at position (i, j) in the target bounding box and the nearest edge of the bounding box, and its value is min(i, j, h o -i,w o -j), h o ,w o are the height and width of the cropped target, β is a threshold to control the range of the smooth boundary, and λ is a number randomly sampled from the Beta distribution; Use edge smoothing mask to fuse the images in the result set. The fusion formula is: Among them, α is the edge smoothing mask, B a is the image block corresponding to the original target object T, B n is the image block in the result set, K is the total number of fused images, is element-wise multiplication; Finally, the fused image is pasted back to the location of the target object T in the original image data to complete the fusion process.

6. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 5, characterized in that: In step E, the deep learning-based target detector uses Faster RCNN as the basic target detection network, and its core parts include three parts: basic feature extraction network, region proposal network, and region classification and boundary regression network, which are used for feature extraction, selecting candidate regions from feature maps, and classifying and regressing bounding boxes for each candidate region.

7. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 6, characterized in that: Step F comprises: All predictions of the target detector corresponding model are divided into 4 categories, namely: (1) PB neg : The IoU between the model predicted target bounding box and the label bounding box is less than the threshold; (2) PB pp : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, but the predicted category is different from the label category and is not background; (3) PB fb : The IoU between the model predicted target bounding box and the label bounding box is greater than the threshold, and the predicted category is background; (4) PB pos : The IoU between the model-predicted target bounding box and the label bounding box is greater than the threshold, and the predicted category is the same as the label; Among them, the "false error" prediction is mainly PB pp The prediction results included in this part are discarded from the loss function calculation, and only the classification losses of the other three parts are calculated. Finally, the total loss function becomes: L=L bbox +L clsneg +L clspos +L clsfb Among them, L bbox represents the regression loss, without changing it, L clsneg , L clspos , L clsfb Respectively represent the above PB neg , P.B. pos ,PB fb The corresponding classification loss.

8. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 7, characterized in that: Step G also includes preprocessing the data, which includes: using random flipping data enhancement to normalize the data of all training sets and test sets; the basic feature extraction network is initialized with ResNet pre-trained on ImageNet to start training, and random gradient descent is used as the optimizer, wherein the initial learning rate is 0.0025, and in the 17th and 21st rounds of training, the learning rate is reduced by 10 times, the weight decay parameter is 0.0001, and the momentum parameter is 0.9; the mini-batch of each iteration is set to 2, and the entire network is trained for 24 rounds; during the training process, the preset large loss suppression mechanism is used to modify the loss function, and after calculating the loss, the gradient descent algorithm is used for back propagation to update the model parameters.

9. The robust X-ray image target detection method based on similar target fusion data augmentation as claimed in claim 8, characterized in that: Step H includes: transforming the resolution of the input image to be detected into 1333*800, and then inputting it into a trained model for X-ray security inspection image target detection, inputting the extracted features into a regression network through the model to obtain the detection bounding box coordinate value, and then inputting it into a classification network to predict its corresponding category and generate a detection result.

Citation Information

Patent Citations

  • airport X-ray contraband image detection method based on an attention mechanism

    CN109800764A

  • X-ray image sample generation method, system and application

    CN110910467A

  • Intelligent identification device and system for contraband goods

    CN113706497A

  • Cutter image intelligent detection method based on GuidedAnchor optimization

    CN113920432A

  • Target object segmentation method and system for security check CT three-dimensional image

    CN114255240A