X-ray security image contraband data enhancement method and device and computer equipment
By employing a multi-scale, multi-directional feature data augmentation method, the low accuracy problem caused by noise and occlusion in X-ray contraband detection was solved. This method generates more training samples, improves the accuracy and robustness of the model, and enhances its generalization performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHONGKE HONGTUO (SUZHOU) INTELLIGENT TECH CO LTD
- Filing Date
- 2023-06-25
- Publication Date
- 2026-05-22
AI Technical Summary
Existing technologies for X-ray contraband detection suffer from low accuracy due to factors such as noise, lighting variations, and target occlusion. In particular, it is difficult to effectively collect and process data on contraband targets that are rotated at different times, which affects the performance of deep learning models.
Data augmentation methods using multi-scale and multi-directional features include acquiring and labeling the minimum bounding rectangle of contraband images, randomly adding them to images without contraband for fusion, performing multi-scale training augmentation and seamless fusion, generating more training samples, and improving the diversity and breadth of data.
It improves the accuracy and robustness of the model, reduces its sensitivity to noise and interference, prevents overfitting, and enhances the model's generalization performance.
Smart Images
Figure CN116797478B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing, specifically relating to a method, apparatus, and computer equipment for enhancing contraband data in X-ray security inspection images. Background Technology
[0002] X-ray machines are commonly used in security checks, package inspections, and body scanning. They use the properties of X-rays to create images of objects through transmission. In security checks and package inspections, X-ray machines are frequently used to detect contraband, such as lighters, lithium batteries, alcohol, and controlled knives. Therefore, improving the accuracy of X-ray machine contraband detection is a hot research topic.
[0003] In recent years, with the development of deep learning technology, deep learning-based automatic identification methods have been widely applied in X-ray contraband detection. In deep learning-based X-ray contraband detection, data plays a crucial role. X-ray images contain many interfering factors, such as noise, lighting variations, and target occlusion. These factors can make the detection of contraband targets difficult, especially for contraband targets with varying rotation. The effective acquisition, annotation, and processing of data directly affects the performance of the deep learning model. Summary of the Invention
[0004] This invention aims to provide a method, apparatus, and computer device for data augmentation of prohibited items in X-ray security inspection images. By combining multi-scale and multi-directional features of the data, this invention enhances the data of prohibited items in the luggage frame of X-ray images, helping the model better learn the invariance and generalization performance of the data, thereby improving the model's accuracy and robustness. Simultaneously, it can effectively prevent overfitting and improve the model's generalization performance.
[0005] To address at least one technical problem mentioned in the background section and achieve at least one technical objective, the specific technical solutions provided in this application include the following:
[0006] In a first aspect, the present invention provides a method for enhancing contraband data in X-ray security inspection images, the method comprising:
[0007] Acquire a first image of contraband and a first image of no contraband, both of which are X-ray security images. The first image of contraband is marked with a first mask label of the target object.
[0008] Determine the minimum bounding rectangle of the first mask label for the first image of the contraband item;
[0009] The first contraband image is segmented according to the least outer matrix, and the segmented second contraband image and the corresponding second mask label marked with the target object are obtained.
[0010] Based on the category of the target object, randomly select one or more images of second contraband and their corresponding second mask labels;
[0011] The second contraband image is randomly added to the first image without contraband, and the edge points are fused to generate an enhanced third contraband image. The third contraband image is labeled with a second mask tag of the target object.
[0012] Furthermore, before determining the minimum bounding rectangle of the first mask label for the first contraband image, the process further includes:
[0013] Count the number of bounding boxes (bboxes) for each target category in the image of the first prohibited item;
[0014] Divide the number of bounding boxes (bboxes) for each target category by the sum of the number of bboxes for all categories to obtain the normalization ratio for each target category.
[0015] The number of category samples is calculated based on the preset sampling threshold and normalization ratio.
[0016] For each first contraband image, find the target category with the most bounding boxes (bboxes) and use the number of samples of that target category as the actual number of samples for that image;
[0017] The image is copied repeatedly based on the actual number of samples and added to the first image of contraband.
[0018] Furthermore, before randomly adding the second contraband image to the first image without contraband, the process includes multi-scale training augmentation of the second contraband image, and random scaling of the second contraband image based on area distribution.
[0019] Furthermore, the second contraband image is augmented using multi-scale training, which involves randomly scaling the second contraband image according to an area distribution, including:
[0020] Statistically analyze the area difference distribution of the same type of target objects in the first contraband image, and calculate the area distribution ratio of the largest and smallest areas of the same type of target objects;
[0021] Based on the area distribution ratio, the scaling range of the second contraband image is determined;
[0022] The image of the second contraband is scaled for both the image itself and the target object according to the scaling range.
[0023] The image itself and the target object that exceed the maximum area after scaling are truncated using the maximum area.
[0024] Furthermore, the scaling range of the second contraband image is [αi, βi]; αi = the area distribution ratio of the minimum area of the i-th type of contraband to the maximum area of the i-th type of contraband; βi = the area distribution ratio of the maximum area of the i-th type of contraband to the minimum area of the i-th type of contraband.
[0025] Furthermore, the scaling range of the second contraband image is [αi^, βi^]; αi^ represents the value after truncating αi through a normal distribution; βi^ represents the value after truncating βi through a normal distribution.
[0026] Furthermore, before randomly adding the second contraband image to the first image without contraband, the second contraband image is randomly flipped and randomly rotated.
[0027] Furthermore, randomly adding the second contraband image to the first image without contraband, and fusing the edge points to generate an enhanced third contraband image includes:
[0028] The second contraband image is randomly added to the first contraband image, and edge points are formed in the first contraband image;
[0029] The resulting edge points are fused to generate an enhanced third image free of contraband.
[0030] In a second aspect, the present invention also provides a data enhancement device for contraband in X-ray security inspection images, the device comprising:
[0031] The image acquisition module is used to acquire a first image of contraband and a first image of no contraband. Both the first image of contraband and the first image of no contraband are X-ray security inspection images. The first image of contraband is marked with a first mask label of the target object.
[0032] The image segmentation module is used to determine the minimum bounding rectangle of the first mask label of the first contraband image; segment the first contraband image according to the minimum bounding matrix, and obtain the segmented second contraband image and the corresponding second mask label marked with the target object;
[0033] The image overlay module is used to randomly select one or more second contraband images and corresponding second mask labels according to the category of the target object; randomly add the second contraband images to the first non-contraband images; and fuse the edge points to generate an enhanced third contraband image, wherein the third contraband image is labeled with the second mask label of the target object.
[0034] In a third aspect, the present invention also provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for enhancing contraband data in X-ray security inspection images as described in the first aspect of the present invention.
[0035] The embodiments of this application have the following beneficial effects:
[0036] This application provides a method, apparatus, and electronic device for data augmentation of contraband in X-ray security inspection images. By performing multi-scale training augmentation and seamless multi-sample fusion augmentation on labeled contraband image data, the diversity and breadth of the data can be effectively improved. This helps the model better learn data invariance and generalization performance, reduces sensitivity to image noise and interference, and improves the model's accuracy and robustness. Simultaneously, it can effectively prevent overfitting and improve the model's generalization performance. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This document shows a general flowchart of a method for data enhancement of contraband in X-ray security inspection images provided in an embodiment of this application;
[0039] Figure 2 This document illustrates a flowchart of the basic data balancing method provided in an embodiment of this application.
[0040] Figure 3 This document illustrates a flowchart of the multi-scale training augmentation provided in an embodiment of this application.
[0041] Figure 4 This illustration shows a schematic diagram of the architecture of a contraband data enhancement device for X-ray security inspection images provided in an embodiment of this application;
[0042] Figure 5 Exemplary systems that can be used to implement the various embodiments described in this application are shown. Detailed Implementation
[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0044] It should be understood that, in the description of this application, unless the context explicitly requires it, the words "comprising," "including," and similar terms throughout the specification and claims should be interpreted as encompassing rather than being exclusive or exhaustive; that is, meaning "including but not limited to."
[0045] It should also be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0046] To improve the generalization ability of image classification networks, embodiments of the present invention provide a method, apparatus, and electronic device for data augmentation of contraband in X-ray security inspection images. The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.
[0047] The following describes a method for enhancing contraband data in X-ray security inspection images provided by this invention. Figure 1 This is a flowchart illustrating a method for data enhancement of contraband in X-ray security inspection images according to an embodiment of the present invention. This specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 1 The method for enhancing contraband data in X-ray security inspection images provided in this specification includes the following steps:
[0048] 101. Obtain the first image of the contraband and the first image of the non-contraband, both of which are X-ray security inspection images. The first image of the contraband is marked with a first mask label of the target object.
[0049] In some embodiments, the first contraband image and the second contraband image are security inspection images obtained by an X-ray machine; in the security inspection images, the first contraband image is labeled with the contraband target, and the first image without contraband is also labeled. By fusing the information of the first contraband image into the first image without contraband, the first image without contraband can also have the contraband information of the first contraband image, which enhances the diversity and breadth of the data and helps the model to better learn the invariance and generalization performance of the data; wherein, the target object of the first contraband image may include, but is not limited to, the following contraband items, namely lighters, knives, and batteries.
[0050] In some embodiments of the present invention, a multi-point mask annotation method is used to annotate the target prohibited items in the first prohibited item image with irregular polygons. The first prohibited item image also needs to be annotated and saved for augmentation. The annotation of the first prohibited item image needs to ensure that the first prohibited item image has no mask points, that is, there is no prohibited item image information in the first prohibited item image, and its annotation category is an image of the prohibited item category.
[0051] In some embodiments of the present invention, before annotating the security inspection images, it is necessary to perform data cleaning on the security inspection images, and to perform noise reduction processing on the security inspection images obtained by X-ray machine scanning, such as wavelet denoising, mean filtering, etc., and image segmentation; for example, edge-based segmentation methods such as Canny operator, Sobel operator, Laplacian operator, etc. are used. After segmentation, only clear images of individual luggage frames are retained. These clear images of individual luggage frames will be used for annotation for data enhancement processing of prohibited items.
[0052] In this embodiment of the invention, the first contraband image can be one of multiple images containing contraband targets collected in an experimental environment. Since deep learning training requires a large number of image samples, but the number of images containing contraband targets is small and acquisition costs are high, the acquired multiple image samples, including the first contraband image and the first image without contraband, only contain the first contraband image as a sample. Therefore, sample augmentation is necessary. Based on the coordinate information of the labeled contraband target region, the image in the contraband target region (i.e., the contraband target image) should be extracted. The extracted image in the contraband target region is used as a sample mask. To ensure sample diversity, the sample mask needs to undergo preset augmentation processing. Preset augmentation processing can include, but is not limited to: random cropping, rotation, translation, random flipping, distortion, contrast transformation, brightness adjustment, adding random noise, and superimposing multiple masks. If the size and diversity of the samples are limited, overfitting can easily occur. Preset augmentation processing can reduce overfitting and improve algorithm performance.
[0053] 102. Determine the minimum bounding rectangle of the first mask label of the first contraband image;
[0054] In some embodiments of the present invention, before determining the minimum bounding rectangle of the first mask label of the first contraband image, the data is further processed using an oversampling method as the basis for data equalization, such as... Figure 2 As shown, it specifically includes:
[0055] 201. Count the number of bounding boxes (bboxes) for each target category in the image of the first prohibited item;
[0056] By iterating through the labeled data of the first prohibited item image, the number of bounding boxes (bboxes) for each target category can be counted, thus obtaining the corresponding bounding box number information.
[0057] 202. Divide the number of bounding boxes (bboxes) for each target category by the sum of the number of bboxes for all categories to obtain the normalization ratio for each target category;
[0058] 203. Calculate the number of category samples based on the preset sampling threshold and normalization ratio;
[0059] The number of category samples is obtained from max(1,oversample_thr / n), where oversample_thr represents the preset sampling threshold. When the number of bounding boxes for a certain category is too small, the minimum number of samples is set to 1 to prevent overfitting.
[0060] 204. For each first contraband image, find the target category with the most bounding boxes (bboxes) and use the number of samples of that target category as the actual number of samples for that first contraband image;
[0061] 205. Repeat the image of the first contraband item based on the actual number of samplings.
[0062] For the first contraband image that needs to be resampled, multi-scale training augmentation operations such as left-right and up-down flipping and 360° random rotation are used to process the contraband information in the first contraband image. For each first contraband image that needs to be resampled, the image is repeatedly copied and added to the dataset according to its actual number of samplings, thereby expanding the image dataset and achieving data balance while making the data more diverse.
[0063] In this embodiment of the invention, the first image of the contraband is balanced so that the number of samples in different categories is equal or close, thereby making the prediction accuracy of the model more balanced in different categories and avoiding insufficient learning of categories with few samples, which can easily lead to overfitting.
[0064] In this embodiment of the invention, for the first mask label of the first contraband image, if multiple first mask labels for target objects need to be marked in a first contraband image, then multiple mask image files will be obtained after labeling. Each mask image file is a binary image, where a pixel value of 1 represents a foreground and 0 represents the background, and the corresponding mask coordinates are also reflected in the mask image file. For example, if there is an image with two irregular target objects named A and B respectively, then after we correctly label them, we will get two mask images. In the first mask image, 0 represents the background and 1 represents the irregular image A; similarly, in the second mask image, 0 represents the background and 1 represents the irregular image B.
[0065] To reduce the number of label readings, the first and second masks can be merged and stored as an 8-bit color image, where 0 represents the background, 1 represents irregular image A, and 2 represents irregular image B. This makes it easy to construct the dataset class. When reading the first mask label corresponding to the first contraband image, the pixel value (0, 1, 2, ..., n) of the first mask label can easily determine what category of target object the pixel represents. Therefore, the minimum bounding rectangle of the first mask label of the first contraband image can be determined by the position coordinates of the first mask label, thus defining the minimum bounding rectangle containing the corresponding first mask image.
[0066] 103. Segment the first contraband image according to the minimum outer matrix, and obtain the segmented second contraband image and the corresponding second mask label marked with the target object;
[0067] In this embodiment of the invention, the first contraband image is divided by taking the smallest bounding rectangle of the first mask label. Then the first contraband image will be divided into one or more second contraband images. The labels in the second contraband images correspond to their corresponding first mask labels. That is, each second contraband image corresponds to a second mask label, and the second mask label is marked with the target object.
[0068] 104. Randomly select one or more images of second contraband and their corresponding second mask labels according to the category of the target object;
[0069] In this embodiment of the invention, the type of the target object is the type of contraband. Therefore, a second contraband image of the same category and a corresponding second mask label can be selected according to the category of the target object. These second contraband images of the same category and the corresponding second mask labels can represent the invariant information of the data of that category, thereby helping the model to better learn the invariance and generalization performance of the data, thereby improving the accuracy and robustness of the model.
[0070] 105. Randomly add the second contraband image to the first non-contraband image, and fuse the edge points to generate an enhanced third contraband image, wherein the third contraband image is labeled with a second mask tag of the target object.
[0071] In this embodiment of the invention, before randomly adding the second contraband image to the first image without contraband, the method further includes multi-scale training augmentation of the second contraband image and random scaling of the second contraband image based on area distribution.
[0072] In some embodiments of the present invention, the second contraband image is augmented using multi-scale training, and the second contraband image is scaled randomly according to an area distribution, such as... Figure 3 As shown:
[0073] 501. Statistically analyze the area difference distribution of the same type of target objects in the first contraband image, and calculate the area distribution ratio of the largest and smallest areas of the same type of target objects;
[0074] 502. Based on the area distribution ratio, determine the scaling range of the second contraband image;
[0075] The scaling range of the second contraband image is [αi, βi]. This range is based on the fact that the imaging area of the contraband in the X-ray machine varies, and the scaling standards for different types of contraband are not uniform. Each type of contraband i has a corresponding set of [αi, βi] values. αi = minimum area of type i contraband / area distribution ratio of maximum area of type i contraband; βi = maximum area of type i contraband / area distribution ratio of minimum area of type i contraband.
[0076] In a preferred embodiment of the present invention, considering that different sample numbers have different degrees of influence on the scaling range, this embodiment also dynamically adjusts the scaling range of the second contraband image according to the normal distribution of the sample number, so that the final determined scaling range can reflect the true scaling range value. This embodiment constructs a corresponding normal distribution based on the sample number, and extracts a suitable scaling range from the range [αi, βi] using the 2σ or 3σ principle; the scaling range of the second contraband image is [αi^, βi^]; αi^ represents the value after truncating αi through the normal distribution; βi^ represents the value after truncating βi through the normal distribution.
[0077] In another preferred embodiment of the present invention, considering that the data of the present invention is discrete data, this embodiment also determines the scaling range each time through the inverse transformation sampling method of discrete data; firstly, the cumulative distribution function (CDF) is calculated: F(x) = P(X≤x), where X is a random variable and x is a certain value. A random number r between 0 and 1 is generated. The inverse transformation sampling formula is: x = min{x|F(x)≥r}, that is, the first value x that satisfies F(x)≥r is found, and the value x is used as the scaling value to scale the second contraband image itself and the target object.
[0078] In another preferred embodiment of the present invention, in order to avoid the difference caused by the scaling ratio of the contraband being too small or too large, the preferred embodiment of the present invention further constrains the area distribution ratio so that the scaling range is [0.1, 10].
[0079] 503. Scale the image of the second contraband and the target object according to the scaling range;
[0080] 504. The image itself and the target object that exceed the maximum area after scaling are truncated using the maximum area.
[0081] In this embodiment of the invention, before randomly adding the second contraband image to the first image without contraband, the second contraband image is further subjected to random flipping and random rotation.
[0082] In some embodiments of the present invention, randomly adding the second contraband image to the first image without contraband, and fusing edge points to generate an enhanced third contraband image includes:
[0083] The second contraband image is randomly added to the first contraband image, and edge points are formed in the first contraband image;
[0084] In this embodiment of the invention, since a multi-scale training augmentation method is also used when the second contraband image is randomly added to the first contraband image, the first contraband image can obtain a better data augmentation effect.
[0085] The resulting edge points are fused to generate an enhanced third image free of contraband.
[0086] In this embodiment of the invention, a seamless fusion method is used to fuse edge points, such as the Laplacian pyramid fusion method or image editing methods based on the Poisson equation, so that the source of the generated third contraband-free image cannot be distinguished by humans, that is, it is impossible to distinguish whether the third contraband-free image was obtained by an X-ray machine or by the above-mentioned methods.
[0087] This invention integrates data from different sources, making the model more robust and resistant to noise and anomalies. Integrating different types of data increases the model's adaptability to unknown data, thereby improving generalization ability. Integrating different categories of data increases the model's training data, thus improving model accuracy. For situations where the number of data categories is limited, integrating different categories can alleviate the problem of imbalanced data distribution.
[0088] This invention also provides a data enhancement device for contraband in X-ray security inspection images, such as... Figure 4 As shown, the device 110 may include:
[0089] Image acquisition module 111 is used to acquire a first image of contraband and a first image of no contraband. Both the first image of contraband and the first image of no contraband are X-ray security inspection images. The first image of contraband is marked with a first mask label of the target object.
[0090] Image segmentation module 112 is used to determine the minimum bounding rectangle of the first mask label of the first contraband image; segment the first contraband image according to the minimum bounding matrix, and obtain the segmented second contraband image and the corresponding second mask label marked with the target object;
[0091] The image overlay module 113 is used to randomly select one or more second contraband images and corresponding second mask labels according to the category of the target object; randomly add the second contraband images to the first non-contraband images; and fuse the edge points to generate an enhanced third contraband image, wherein the third contraband image is labeled with the second mask label of the target object.
[0092] This invention provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement an image data enhancement method as provided in the above method embodiments.
[0093] Figure 5 A schematic diagram of a hardware structure for implementing the method provided in the embodiments of this application is shown. This device may participate in or include the apparatus or system provided in the embodiments of this application. Figure 5 As shown, device 120 may include one or more processors 122 (shown as 122a, 122b, ..., 122n in the figure) 122 (processor 122 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 124 for storing data, and a transmission device 126 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 5 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 120 may also include a... Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.
[0094] It should be noted that the aforementioned one or more processors 122 and / or other data processing circuitry are generally referred to herein as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the device 120 (or mobile device). As involved in the embodiments of this application, this data processing circuitry serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).
[0095] The memory 124 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 122 executes various functional applications and data processing by running the software programs and modules stored in the memory 124, thereby realizing the above-described image data enhancement method. The memory 124 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 124 may further include memory remotely located relative to the processor 122, and these remote memories can be connected to the device 120 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0096] The transmission device 126 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 120. In one example, the transmission device 126 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 126 may be a radio frequency (RF) module used for wireless communication with the Internet.
[0097] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 120 (or mobile device).
[0098] This invention also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing an image data enhancement method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the contraband image data enhancement method provided in the above method embodiment.
[0099] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0100] As can be seen from the embodiments of the X-ray security inspection image data augmentation method, apparatus, and electronic device provided by the present invention, the solution provided by the present invention performs multi-sample, multi-scale data augmentation on the basis of the image dataset, generating more training samples, solving the problem of insufficient training data, and reducing overfitting caused by model training. After performing multi-scale training augmentation and multi-sample seamless fusion augmentation operations on the labeled contraband image data, the diversity and breadth of the data can be effectively improved. This helps the model better learn the invariance and generalization performance of the data, reduces the sensitivity to image noise and interference, and improves the accuracy and robustness of the model. At the same time, it can also effectively prevent overfitting and improve the generalization performance of the model. Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for data enhancement of contraband in X-ray security inspection images, characterized in that, The method includes: Acquire a first image of contraband and a first image of no contraband, both of which are X-ray security images. The first image of contraband is marked with a first mask label of the target object. Determine the minimum bounding rectangle of the first mask label for the first image of the contraband item; The first contraband image is segmented according to the minimum bounding rectangle, and the segmented second contraband image and the corresponding second mask label marked with the target object are obtained. Based on the category of the target object, randomly select one or more images of second contraband and their corresponding second mask labels; The second contraband image is randomly added to the first image without contraband, and the edge points are fused to generate an enhanced third contraband image. The third contraband image is labeled with a second mask tag of the target object. Before determining the minimum bounding rectangle of the first mask label for the first contraband image, the process further includes: Count the number of bounding boxes (bboxes) for each target category in the image of the first prohibited item; Divide the number of bounding boxes (bboxes) for each target category by the sum of the number of bboxes for all categories to obtain the normalization ratio for each target category. The number of category samples is calculated based on the preset sampling threshold and normalization ratio. For each first contraband image, find the target category with the most bounding boxes (bboxes) and use the number of samples of that target category as the actual number of samples for that first contraband image; The image of the first contraband item is copied repeatedly based on the actual number of samples taken.
2. The method for enhancing contraband data in X-ray security inspection images according to claim 1, characterized in that, Before randomly adding the second contraband image to the first image without contraband, the process includes multi-scale training augmentation of the second contraband image and random scaling of the second contraband image based on area distribution.
3. The method for enhancing contraband data in X-ray security inspection images according to claim 2, characterized in that, The second contraband image is augmented using multi-scale training, and the second contraband image is randomly scaled according to an area distribution, including: Statistically analyze the area difference distribution of the same type of target objects in the first contraband image, and calculate the area distribution ratio of the largest and smallest areas of the same type of target objects; Based on the area distribution ratio, the scaling range of the second contraband image is determined; The image of the second contraband is scaled for both the image itself and the target object according to the scaling range. The image itself and the target object that exceed the maximum area after scaling are truncated using the maximum area.
4. The method for enhancing contraband data in X-ray security inspection images according to claim 3, characterized in that, The scaling range of the second contraband image is [αi, βi]; αi = the area distribution ratio of the minimum area of the i-th type of contraband to the maximum area of the i-th type of contraband; βi = the area distribution ratio of the largest area of the i-th type of prohibited item to the smallest area of the i-th type of prohibited item.
5. The method for enhancing contraband data in X-ray security inspection images according to claim 4, characterized in that, The scaling range of the second contraband image is [αi^, βi^]; αi^ represents the value after cropping αi through a normal distribution; βi^ represents the value after cropping βi through a normal distribution.
6. The method for enhancing contraband data in X-ray security inspection images according to claim 2, characterized in that, Before randomly adding the second contraband image to the first image without contraband, the process also includes randomly flipping and rotating the second contraband image.
7. The method for enhancing contraband data in X-ray security inspection images according to claim 1, characterized in that, The process involves randomly adding the second contraband image to the first image without contraband, and then fusing edge points to generate an enhanced third contraband image, including: The second contraband image is randomly added to the first contraband image, and edge points are formed in the first contraband image; The resulting edge points are fused to generate an enhanced third image free of contraband.
8. A data enhancement device for contraband in X-ray security inspection images, characterized in that, The device includes: The image acquisition module is used to acquire a first image of contraband and a first image of no contraband. Both the first image of contraband and the first image of no contraband are X-ray security inspection images. The first image of contraband is marked with a first mask label of the target object. The image segmentation module is used to determine the minimum bounding rectangle of the first mask label of the first contraband image; segment the first contraband image according to the minimum bounding rectangle, and obtain the segmented second contraband image and the corresponding second mask label marked with the target object; The image overlay module is used to randomly select one or more second contraband images and corresponding second mask labels according to the category of the target object; randomly add the second contraband image to the first image without contraband, and fuse the edge points to generate an enhanced third contraband image, wherein the third contraband image is labeled with the second mask label of the target object; Before determining the minimum bounding rectangle of the first mask label for the first contraband image, the process further includes: Count the number of bounding boxes (bboxes) for each target category in the image of the first prohibited item; Divide the number of bounding boxes (bboxes) for each target category by the sum of the number of bboxes for all categories to obtain the normalization ratio for each target category. The number of category samples is calculated based on the preset sampling threshold and normalization ratio. For each first contraband image, find the target category with the most bounding boxes (bboxes) and use the number of samples of that target category as the actual number of samples for that first contraband image; The image of the first contraband item is copied repeatedly based on the actual number of samples taken.
9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for enhancing contraband data in X-ray security images as described in any one of claims 1 to 7.