Security check article detection method based on data enhancement
By introducing natural image and data enhancement strategies in X-ray security check, we construct training sample pairs with improved diversity and perform step-by-step fusion prediction box processing, which solves the problem of low detection efficiency of prohibited items in X-ray security check, and significantly improves detection Accuracy and performance.
Patent Information
- Application Number
- CN202411133124.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-17
- Publication Date
- 2024-12-06
AI Technical Summary
In the X-ray security inspection, the detection efficiency of prohibited items is low, the possibility of missed and missed detection is high, and the commonly used target detection model is not applicable.
Using detection methods based on natural guidance and data augmentation, data augmentation strategies are used to improve sample diversity by constructing training sample pairs of natural images and X-ray security images, and improving the accuracy of detection of prohibited items by step-by-step fusion of prediction box results. .
By introducing natural image features to supplement information of prohibited items, we will enhance model discrimination capabilities, improve detection accuracy, reduce the impact of data set defects, and improve the detection performance of X-ray prohibited items.
Smart Images

Figure CN119091203A_ABST
Abstract
Description
[0001] This application is a divisional application. The application number of the original application is 202210683874.6, the application date is June 17, 2022, and the name of the invention is "A method and system for detecting prohibited items based on natural guidance and data enhancement." Technical Field
[0002] The present invention relates to the technical field of X-ray security inspection, and more particularly to a method for detecting security inspection items based on data enhancement. Background Art
[0003] X-ray security inspection plays an important role in public transportation places such as trains, airplanes, and subways. X-ray security inspection machines scan luggage in pseudo-color, and security inspectors need to identify the types and locations of prohibited items in luggage based on their memory and experience of imaging various types of prohibited items. However, the efficiency of this security inspection method currently depends entirely on the experience and working status of security inspectors, and is easily affected by the environment, resulting in an increased possibility of missed inspections and false inspections.
[0004] In order to reduce the burden on security personnel and improve the efficiency of security inspections, it has become more urgent and important to use computers to automatically detect prohibited items. With the development of computer vision technology, target detection tasks are often used for object classification and positioning, which is similar to the task of identifying and locating prohibited items in X-ray security images. In existing deep learning methods, transfer learning technology is usually used to directly transfer classic target detection algorithms to the research on prohibited item detection in X-ray security images. However, due to the imaging characteristics of X-ray security inspection machines, X-ray security images are very different from natural light images in appearance. X-ray security images lose a lot of detailed information such as color, texture, etc., and the images overlap and are severely occluded. The currently commonly used target detection models are not applicable. Therefore, how to supplement the prohibited item information in X-ray security images and improve the generalization ability and detectability of the model are technical problems that technicians in this field urgently need to solve. Summary of the invention
[0005] In view of this, the present invention provides a security inspection item detection method based on data enhancement, which solves the problems existing in the background technology.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] A prohibited item detection method based on natural guidance and data enhancement comprises the following steps:
[0008] S1, constructing training sample pairs of natural images and X-ray security inspection images based on the original target detection frame;
[0009] S2. Improving the diversity of the training sample pairs through a data enhancement strategy;
[0010] S3, performing step-by-step fusion on the first prediction frame result obtained by the data enhancement strategy to obtain a second prediction frame result;
[0011] S4. Detect prohibited items according to the second prediction frame result.
[0012] Preferably, the step S1 comprises:
[0013] S11, obtaining the classification accuracy of prohibited items through the original target detection frame;
[0014] S12, judging whether there is confusion in the classification of the prohibited items based on the classification accuracy;
[0015] S13. If confusion exists, obtain the category information of the confusion and determine the introduction category of the natural image;
[0016] S14, searching for prohibited item images containing the introduced category in a natural image sample set for sample pairing.
[0017] Preferably, the step S2 specifically includes: correcting the scale of the X-ray security inspection image through an image pyramid; extracting contraband from the X-ray security inspection image and projecting it into other X-ray security inspection images; and performing contrast enhancement and brightness enhancement processing on the X-ray security inspection image.
[0018] Preferably, the step S3 is specifically:
[0019] S31, obtaining a first prediction box result of the data enhancement strategy, where the first prediction box result is the prediction box position coordinates and confidence information;
[0020] S32, arranging the first prediction frame results in descending order according to confidence, removing the prediction frames whose confidence is lower than a preset threshold, and obtaining a set of prediction frames to be integrated for ensemble learning;
[0021] S33, step by step fusing the prediction frames to be fused obtained in S32 to obtain a preliminary fused prediction frame and an unfused prediction frame.
[0022] Preferably, after step S33, the following steps are further included:
[0023] S34, determining the degree of fusion between the preliminary fusion prediction frame and the unfused prediction frame, if the degree of fusion is greater than a fusion threshold, executing step S35, otherwise executing step S36;
[0024] S35, re-integrating the preliminary fused prediction frame with the unfused prediction frame to obtain a secondary prediction frame result;
[0025] S36, clustering the unfused prediction boxes separately to obtain a secondary prediction box result;
[0026] S37: Obtain the secondary prediction box result collection as the second prediction box result.
[0027] A prohibited item detection system based on natural guidance and data enhancement, comprising: a natural guidance module, a data enhancement module, a strategy integration module, and a target detection module;
[0028] The natural guidance module constructs training sample pairs of natural images and X-ray security inspection images;
[0029] The data enhancement module improves the diversity of the training sample pairs through a data enhancement strategy;
[0030] The strategy integration module performs step-by-step fusion on the first prediction frame result of the data enhancement strategy to obtain a second prediction frame result;
[0031] The object detection module performs prohibited item detection according to the second prediction frame result.
[0032] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method for detecting security items based on data enhancement, which has the following beneficial effects: by introducing natural images to provide appearance features for the X-ray security image data set, supplement the information of prohibited items, enhance the model's ability to distinguish prohibited items, and improve detection accuracy; by using different data enhancement strategies to improve the diversity of the data set and reduce the impact caused by the defects of the data set itself, the detection performance of the X-ray prohibited items detection model can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0034] Figure 1 It is a schematic diagram of the process of the prohibited articles detection method of the present invention;
[0035] Figure 2 It is a block diagram of the prohibited items detection system of the present invention. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0037] Embodiment 1:
[0038] The embodiment of the present invention discloses a method for detecting security items based on data enhancement, such as Figure 1 As shown, the following steps are included:
[0039] S1, constructing training sample pairs of natural images and X-ray security inspection images based on the original target detection frame;
[0040] S2, improve the diversity of training sample pairs through data enhancement strategies;
[0041] S3, performing step-by-step fusion on the first prediction frame result obtained by the data enhancement strategy to obtain a second prediction frame result;
[0042] S4. Detect prohibited items according to the second prediction frame result.
[0043] Embodiment 2:
[0044] The embodiment of the present invention discloses a method for detecting security items based on data enhancement, comprising the following steps:
[0045] S1. Construct training sample pairs of natural images and X-ray security inspection images based on the original target detection frame, specifically including:
[0046] S11, obtaining the classification accuracy of all types of prohibited items through the original target detection frame;
[0047] S12. Determine whether there is confusion in the classification of the prohibited items based on the classification accuracy, specifically:
[0048] mean(|f(O n,c )-f(O n,c' )|)<Δ
[0049] In the formula, O n,c Indicates n types of materials, c types of prohibited items, O n,c' represents n types of materials, c' types of prohibited items, Δ represents item O n,c With O n,c' degree of confusion;
[0050] S13. If there is confusion, obtain the category information of the confusion and determine the introduction category of the natural image, specifically:
[0051] Get the category c with the lowest classification accuracy l
[0052]
[0053] In the formula, c l For the natural image categories introduced;
[0054] S14, searching for prohibited items images containing the introduced category in the natural image sample set for sample pairing, specifically:
[0055]
[0056] In the formula, B represents the X-ray security inspection image, B' represents the training sample pair, Indicates that the natural image sample set contains c l The prohibited items images of the category, countif(B,c l ) is used to calculate the number of cells in B that contain c l The number of prohibited items images, Used to perform random replacement of images when B contains c l When the contraband images of the category are collected, it can be ensured that B' contains at least one natural image sample set.
[0057] S2. Improve the diversity of training sample pairs through data enhancement strategies, specifically:
[0058] The scale of the X-ray security inspection image is corrected through the image pyramid; contraband is extracted from the X-ray security inspection image and projected into other X-ray security inspection images; the contrast and brightness of the X-ray security inspection image are enhanced.
[0059] S3, the first prediction frame result obtained by the data enhancement strategy is integrated step by step to obtain the second prediction frame result, specifically:
[0060] S31, obtaining the first prediction box results of all K data enhancement strategies, the first prediction box results are the prediction box position coordinates and confidence information, specifically:
[0061] Φ=Q1(x)∪Q2(x)∪…∪Q K (x)
[0062] In the formula, Φ represents the first prediction box result set of all K data enhancement strategies, Q K (x) represents the first prediction box result of the Kth data enhancement strategy;
[0063] S32: Arrange the first prediction frame results in descending order according to the confidence S, remove the prediction frames whose confidence is lower than the preset threshold, and obtain the set of prediction frames to be fused that participate in the ensemble learning Φ S :
[0064] Φ S = sort(Φ,S);
[0065] S33, step by step fusion of the prediction frames to be fused obtained in S32, to obtain a preliminary fusion prediction frame and an unfused prediction frame, specifically:
[0066] Determine the area intersection and union ratio of the two prediction frames to be fused. If the area intersection and union ratio is greater than the intersection and union ratio parameter, the prediction frames to be fused will be fused:
[0067] Get the maximum confidence score s in the prediction box to be fused oi :
[0068] s oi =max(s i ,…,s j )
[0069] In the formula, s i ,s j Indicates the confidence of the prediction box to be fused;
[0070] Calculate the coordinate parameters of the prediction box to be fused
[0071]
[0072] Where f oi is the initial fusion prediction box, s k Represents the confidence of the prediction box to be fused, x 1,2k ,y 1,2k Represents the coordinate parameters of the prediction box to be fused;
[0073] S34, determine the fusion degree IOU between the preliminary fusion prediction frame and the unfused prediction frame:
[0074] If the fusion degree is greater than the fusion threshold thr, execute step S35, otherwise execute step S36;
[0075] S35, re-integrate the preliminary fused prediction frame with the unfused prediction frame to obtain a secondary prediction frame result:
[0076] f oi2 =Fus({b i ,…b j}),IOU(f oi1 ,b j )>thr
[0077] In the formula, f oi1 is the initial fusion prediction box, b j is the unfused prediction box, f oi2 is the secondary prediction box result after re-integration, Fus({b i ,…b j}) is the fusion function, IOU(f oi1 ,b j ) is f oi1 and b j The fusion degree, thr is the fusion threshold, the fusion threshold is greater than 0.75;
[0078] S36, cluster the unfused prediction boxes separately to obtain a secondary prediction box result:
[0079] f oi2 =b j ,IOU(f oi1 ,b j ) <thr
[0080] In the formula, f oi2 for b j Secondary prediction box results after separate clustering.
[0081] S37. Obtain a collection of secondary prediction box results as a second prediction box result.
[0082] S4. Detect prohibited items according to the second prediction frame result.
[0083] Embodiment three:
[0084] The embodiment of the present invention discloses a prohibited article detection system based on natural guidance and data enhancement, such as Figure 2 As shown, it includes: natural guidance module, data enhancement module, strategy integration module, target detection module;
[0085] The natural guidance module constructs training sample pairs of natural images and X-ray security images;
[0086] The data enhancement module improves the diversity of training sample pairs through data enhancement strategies;
[0087] The strategy integration module performs step-by-step fusion of the first prediction frame result of the data enhancement strategy to obtain the second prediction frame result;
[0088] The object detection module performs prohibited item detection based on the second prediction frame result.
[0089] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0090] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A security inspection item detection method based on data enhancement, It is characterized in that The following steps are involved: S1, constructing training sample pairs of natural images and X-ray security inspection images based on the original target detection frame; S2. Improving the diversity of the training sample pairs through a data enhancement strategy; S3, performing step-by-step fusion on the first prediction frame result obtained by the data enhancement strategy to obtain a second prediction frame result; Specifically include: S31, obtaining the first prediction box results of all K data enhancement strategies, the first prediction box results are the prediction box position coordinates and confidence information, specifically: Φ=Q 1 (x)∪Q 2 (x)∪…∪Q K (x) In the formula, Φ represents the first prediction box result set of all K data enhancement strategies, Q K (x) represents the first prediction box result of the Kth data enhancement strategy; S32: Arrange the first prediction frame results in descending order according to the confidence S, remove the prediction frames whose confidence is lower than the preset threshold, and obtain the set of prediction frames to be fused that participate in the ensemble learning Φ S : F S =sort(Φ,S) S33, step by step fusion of the prediction frames to be fused obtained in S32, to obtain a preliminary fusion prediction frame and an unfused prediction frame, specifically: Determine the area intersection and union ratio of the two prediction frames to be fused. If the area intersection and union ratio is greater than the intersection and union ratio parameter, the prediction frames to be fused will be fused: Get the maximum confidence score s in the prediction box to be fused oi : s oi =max(s i ,…,s j ) In the formula, s i ,s j Indicates the confidence of the prediction box to be fused; Calculate the coordinate parameters of the prediction box to be fused In the formula, f oi is the initial fusion prediction box, s k Represents the confidence of the prediction box to be fused, x 1,2k ,y 1,2k Represents the coordinate parameters of the prediction box to be fused; S4. Detect prohibited items according to the second prediction frame result.
2. According to the method for detecting security items based on data enhancement according to claim 1, It is characterized in that The step S1 comprises: S11, obtaining the classification accuracy of prohibited items through the original target detection frame; S12. Determine whether there is confusion in the classification of the prohibited items based on the classification accuracy, specifically: mean(|f(O n,c )-f(O n,c' )|)<Δ In the formula, O n,c Indicates n types of materials, c types of prohibited items, O n,c' represents n types of materials, c' types of prohibited items, Δ represents item O n,c With O n,c' degree of confusion; S13. If there is confusion, obtain the category information of the confusion and determine the introduction category of the natural image, specifically: Get the category c with the lowest classification accuracy l : In the formula, c l For the natural image categories introduced; S14, searching for prohibited items images containing the introduced category in the natural image sample set for sample pairing, specifically: In the formula, B represents the X-ray security inspection image, B' represents the training sample pair, Indicates that the natural image sample set contains c l The prohibited items images of the category, counti(B,c l ) is used to calculate the number of cells in B that contain c l The number of prohibited items images, Used to perform random replacement of images when B contains c l When the contraband images of the category are collected, it can be ensured that B' contains at least one natural image sample set.
3. According to the method for detecting security items based on data enhancement according to claim 1, It is characterized in that The step S2 specifically includes: correcting the scale of the X-ray security inspection image through an image pyramid; extracting contraband from the X-ray security inspection image and projecting it into other X-ray security inspection images; and performing contrast enhancement and brightness enhancement processing on the X-ray security inspection image.
4. According to claim 1, a method for detecting security items based on data enhancement, It is characterized in that After step S33, the following steps are also included: S34, determining the fusion degree IOU between the preliminary fusion prediction frame and the unfused prediction frame, if the fusion degree IOU is greater than the fusion threshold thr, executing step S35, otherwise executing step S36; S35, re-integrate the preliminary fused prediction frame with the unfused prediction frame to obtain a secondary prediction frame result: f oi2 =Fus({b i ,…b j }),IOU(f oi1 ,b j )>thr In the formula, f oi1 is the initial fusion prediction box, b j is the unfused prediction box, f oi2 is the secondary prediction box result after re-integration, Fus({b i ,…b j }) is the fusion function, IOU(f oi1 ,b j ) is f oi1 and b j The fusion degree, thr is the fusion threshold, the fusion threshold is greater than 0.75; S36, clustering the unfused prediction boxes separately to obtain a secondary prediction box result: f oi2 =b j ,IOU(f oi1 ,b j )<thr In the formula, f oi2 for b j Secondary prediction box results after separate clustering; S37: Obtain the secondary prediction box result collection as the second prediction box result.
Citation Information
Patent Citations
Forehead area based face detection method and system
CN106250843A
TensorFlow-based security check contraband identification system and method
CN111310635A
X-ray security check image detection method based on improved YOLOv4
CN113362299A
X-ray security check image prohibited article detection method based on style feature channel attention
CN113657493A
