Dual-energy security image dangerous goods identification method based on HSV space enhancement

Abstract

The present application belongs to the technical field of dangerous goods identification, and particularly relates to a dual-energy security image dangerous goods identification method based on HSV space enhancement. The collected dual-energy image is converted from an RGB color space to an HSV color space; image enhancement is performed on the dual-energy image in the HSV color space, the H channel remains unchanged, and the V channel and / or the S channel is subjected to random enhancement operation, so that equivalent atomic number information representing material classification is ensured not to be changed. If a pre-trained model needs to be used, the input module weight of the public pre-trained model can be converted to adapt to the HSV input channel, or the feature representation model of the HSV space can be directly re-pre-trained on a no-label security image data set.

Description

Technical Field

[0001] This invention belongs to the field of hazardous materials identification technology, specifically relating to a method for hazardous materials identification based on HSV spatial enhancement dual-energy security inspection images. Background Technology

[0002] Dual-energy X-ray imaging for security checks utilizes the difference in attenuation rates between high-energy and low-energy X-rays after penetrating an object to identify its equivalent atomic number, thereby distinguishing between organic, mixed, and inorganic substances based on the equivalent atomic number. According to the GB 15208-2018 standard, substances with an equivalent atomic number below 10 are considered organic, appearing as orange of varying saturation in the security image; substances with an equivalent atomic number between 10 and 18 are considered mixed, appearing as green of varying saturation in the security image; and substances with an equivalent atomic number above 18 are considered inorganic, appearing as blue of varying saturation in the security image.

[0003] In recent years, the development of AI technology has led to the widespread application of automatic hazardous materials identification in security inspection images. Currently, AI models for automatic hazardous materials identification are mainly divided into two categories: those based on monoenergetic grayscale images and those based on dual-energy RGB images. Monoenergetic grayscale images lack equivalent atomic number information representing material classification, making it difficult to distinguish objects with similar shapes but different material properties. Dual-energy RGB image-based identification often involves directly inputting the RGB three-channel image into the AI ​​model after image enhancement, which presents two problems: first, the equivalent atomic number representing material classification is coupled with the grayscale values ​​of the three RGB channels, increasing the difficulty for the model to learn material classification information; second, random image enhancement is usually performed before the image is input into the model, and this random enhancement occurs in the RGB space, which leads to random changes in the equivalent atomic number information representing material classification. Summary of the Invention

[0004] To overcome the problems in the prior art, this invention proposes a dual-energy security inspection image dangerous goods identification method based on HSV spatial enhancement.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] This invention provides a method for identifying dangerous goods in dual-energy security inspection images based on HSV spatial enhancement, characterized by comprising the following steps:

[0007] Step 100: Convert the dual-energy RGB image to an HSV image;

[0008] Step 200: Perform image enhancement on the HSV image, keeping the H channel unchanged, and performing random enhancement operations on the V channel and / or S channel;

[0009] Step 300: Based on the enhanced HSV image, train the HSV hazardous materials identification model;

[0010] Step 400: Identify hazardous materials based on the trained HSV hazardous materials identification model.

[0011] Further, in step 200, image enhancement is performed on the HSV image, with the H channel remaining unchanged and the V channel and / or S channel undergoing random enhancement, including:

[0012] The HSV image is decomposed into three independent channels: H, S, and V.

[0013] Perform random enhancement operations on the V channel and / or S channel;

[0014] The enhanced V and / or S channels are merged with the original H channel to obtain the enhanced HSV image.

[0015] Further, in step 300, an HSV hazardous materials identification model is trained based on the enhanced HSV image, including: a hazardous materials identification model based on RGB pre-trained model and enhanced HSV image, and then transferred to train the HSV hazardous materials identification model.

[0016] Furthermore, based on the RGB pre-trained hazardous materials identification model and the enhanced HSV image, transfer training is performed to obtain the HSV hazardous materials identification model, including:

[0017] The input module of the RGB pre-trained hazardous materials identification model is the RGB input module, and the input module of the HSV hazardous materials identification model is the HSV input module.

[0018] Set a loss function to compare the output differences between the HSV input module and the RGB input module;

[0019] Randomly initialize the weights of the HSV input module, load the weights of the RGB input module, and fix the weights of the RGB input module;

[0020] The HSV channel is used as the input to the HSV input module, and the RGB channel is used as the input to the RGB input module. The weights of the HSV input module are updated through training.

[0021] The HSV input module is used to replace the input module of the RGB pre-trained hazardous materials identification model to form the HSV hazardous materials identification model.

[0022] Furthermore, the setup method for the HSV input module includes:

[0023] The input module of the hazardous materials identification model based on RGB pre-training is denoted as the original input module:

[0024] The original input module is copied into two independent modules: the HSV input module and the RGB input module.

[0025] Furthermore, the setup method for the HSV input module includes:

[0026] The input module of the RGB pre-trained hazardous materials identification model is copied and denoted as the RGB input module, and the input module of the HSV hazardous materials identification model is redefined and denoted as the HSV input module.

[0027] Further, in step 300, an HSV hazardous materials identification model is trained based on the enhanced HSV image, including: using the enhanced HSV image for supervised or self-supervised training to obtain the trained HSV hazardous materials identification model.

[0028] Compared with the prior art, the present invention has the following technical effects:

[0029] Before inputting the image into the model, the RGB 3-channel image is converted to an HSV 3-channel image. This ensures that only the H channel values ​​are relevant to material classification, while the S and V channel values ​​are irrelevant. During image enhancement, only the V channel is randomly enhanced, or the V and S channels are randomly enhanced, while the H channel is not randomly enhanced. This ensures that the equivalent atomic number information representing material classification remains unchanged.

[0030] Considering that most pre-trained models are implemented in the RGB space, and that training automatic hazardous materials identification models typically involves training pre-trained models on labeled datasets, two methods can be used to utilize pre-trained models: one is to transform the input layer parameters of the pre-trained model from RGB space weights to HSV space weights; the other is to pre-train directly in the HSV space using an unlabeled security inspection dataset, and then use the pre-trained model to train on a labeled dataset. Attached Figure Description

[0031] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the process of the present invention;

[0033] Figure 2 This is a method to convert the RGB input layer with pre-trained weights into an HSV input layer through training;

[0034] Figure 3 The first image is the recognition result based on the traditional RGB space;

[0035] Figure 4 The first image is identified based on the results of this invention;

[0036] Figure 5 The second type of image is the recognition result based on the traditional RGB space;

[0037] Figure 6 The second type of image is identified based on the results of this invention. Detailed Implementation

[0038] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solutions proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. Specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] In one embodiment of the present invention, reference is made to... Figure 1 This paper provides a method for identifying dangerous goods from dual-energy security inspection images based on HSV spatial enhancement, including the following steps:

[0040] Step 100: Convert the acquired dual-energy RGB image into an HSV image;

[0041] Step 200: Enhance the HSV image, keeping the H channel unchanged, and performing random enhancement operations on the V channel and / or S channel to obtain the enhanced HSV image;

[0042] Step 300: Based on the enhanced HSV image, train the HSV hazardous materials identification model;

[0043] Step 400: Identify hazardous materials based on the trained HSV hazardous materials identification model.

[0044] The following is a detailed explanation of each of the above steps:

[0045] Step 100: Convert the dual-energy RGB image to an HSV image.

[0046] Converting RGB images to HSV images is a common method in the field of image processing, which is introduced in image processing tutorials and has been implemented in mainstream image processing libraries, so it will not be described in detail here.

[0047] Step 200: Enhance the HSV image by keeping the H channel unchanged and performing random enhancement operations on the V and / or S channels to obtain the enhanced HSV image.

[0048] Design an image enhancement algorithm that performs random enhancement operations, such as adjusting brightness and contrast, only on the V and / or S channels. Keep the H channel unchanged to ensure that the equivalent atomic number information is not altered.

[0049] Brightness enhancement of the V channel: Randomly generate brightness adjustment factors ,in, This refers to the range of brightness variation, such as 0.2.

[0050] ;

[0051] ;

[0052] In the above formula, This represents the V channel of a high-energy image after brightness enhancement; This represents the V channel of a low-energy image after brightness enhancement.

[0053] Contrast enhancement of the S channel: Randomly generate contrast adjustment factors ,in This refers to the range of contrast variation, such as 0.15.

[0054] ;

[0055] ;

[0056] In the above formula, This represents the S channel of a high-energy image after contrast enhancement. This represents the S channel of the low-energy image after contrast enhancement; the Clip function ensures that pixel values ​​are in the range [0, 1].

[0057] Step 300: Based on the enhanced HSV image, train the HSV hazardous materials identification model.

[0058] The weights of the HSV hazardous materials identification model can be obtained by converting the pre-trained weights of the RGB pre-trained hazardous materials identification model through the output module; alternatively, the images in the pre-training dataset can be directly converted to HSV format, and then the pre-trained weights can be obtained by using the HSV format images for supervised or self-supervised pre-training.

[0059] Existing hazardous materials identification models (such as ResNet and VGG) are mostly based on training with fused dual-energy RGB images. Given the HSV characteristics of dual-energy security inspection images, it is necessary to address the challenge of adapting RGB pre-trained hazardous materials identification models to the HSV space.

[0060] Method 1: Based on the RGB pre-trained hazardous materials identification model and the enhanced HSV image, a transfer training method is used to obtain the HSV hazardous materials identification model. The weights of the HSV hazardous materials identification model can be obtained by converting the pre-trained weights of the RGB pre-trained hazardous materials identification model through the output module.

[0061] Figure 2 To transform the input layer parameters of a pre-trained RGB hazardous materials identification model from RGB space weights to HSV space weights, the training process includes the following steps:

[0062] Step 310: Set up the input module of the HSV hazardous materials identification model based on the input module of the RGB pre-trained hazardous materials identification model.

[0063] Extract the input layer structure (such as convolutional layers, batch normalization layers, etc.) from the RGB pre-trained hazardous materials identification model, and denote it as the original input module.

[0064] To meet different needs, two module replication strategies are provided.

[0065] The first approach is to copy the original input module into two independent modules: a first input module (HSV input module) specifically for processing HSV image input, which is then adapted to HSV spatial features through targeted training; and a second input module (RGB input module) that directly inherits the weight parameters of the original input module, serving as a baseline module for knowledge transfer, while retaining the pre-trained weights of the RGB pre-trained hazardous materials identification model.

[0066] The second approach is for lightweight design scenarios. The input structure of the RGB pre-trained hazardous materials identification model is extracted and copied into a single copy, which is called the second input module (RGB input module). The input module of the HSV hazardous materials identification model is then redefined and called the first input module (HSV input module).

[0067] Step 320: Set the loss function to compare the output differences between the HSV input module and the RGB input module.

[0068] Mean squared error (MSE) is introduced as a loss function to compare the output differences between the HSV input module and the RGB input module. By forcing the output of the HSV input module to approximate that of the RGB input module, effective knowledge transfer is achieved.

[0069] The loss function is explicitly applied only to the HSV input module. During backpropagation, only the weights of this module are updated, while the weights of the RGB input module remain fixed, ensuring the stability of the pre-trained knowledge.

[0070] Step 330: Randomly initialize the weights of the HSV input module and load the weights of the RGB input module.

[0071] The weights of the HSV input module are randomly initialized, giving the model the ability to learn from scratch; at the same time, the weights of the RGB input module are loaded to make full use of existing pre-trained knowledge and reduce training time and resource consumption.

[0072] Step 340: Solidify the weights of the RGB input modules and update only the weights of the HSV input modules through backpropagation.

[0073] This ensures that the pre-trained knowledge is not altered, allowing the focus to be on adjusting the weights of the HSV input module to match the output of the RGB input module.

[0074] By updating only the weights of the HSV input modules through backpropagation, and configuring the optimizer to include only trainable parameters (such as filter(p.requires_grad) in PyTorch), lossless transfer of RGB knowledge is ensured.

[0075] Step 350: Use the enhanced HSV channels as input to the HSV input module and the RGB channels as input to the RGB input module, and update the weights of the HSV input module through training.

[0076] After calculating the MSE loss, the weights of the HSV input modules are updated using the chain rule.

[0077] The Adam optimizer (learning rate η=0.001) is used to update the weights in each iteration until the output difference converges (e.g., cosine similarity ≥ 0.95).

[0078] Step 360: Replace the input module of the RGB pre-trained hazardous materials identification model with the trained HSV input module to form the HSV hazardous materials identification model, so that it can better adapt to the image features of the HSV space and improve the accuracy of hazardous materials identification.

[0079] Method 2: Convert the images in the pre-training dataset into HSV images, and then use the HSV images for supervised or self-supervised pre-training to obtain the pre-training weights of the HSV hazardous materials identification model.

[0080] The images in the pre-training dataset are converted to HSV format, and pre-training is performed using these HSV images to obtain pre-training weights. Depending on whether the images used are labeled, pre-training can be divided into self-supervised pre-training and supervised pre-training. The training process includes steps such as converting RGB images to HSV images, modifying data augmentation methods, and obtaining pre-training weights through pre-training. After pre-training, the model structure is adjusted to suit the hazardous materials identification task.

[0081] If using unlabeled images for self-supervised pre-training, self-supervised methods such as SimCLR or BYOL can be employed. Different HSV transformations of the same image (e.g., random S / V channel augmentation) are treated as positive sample pairs, and different images as negative sample pairs. A projection head is added on top of the pre-trained model to map HSV features to a low-dimensional space for contrastive learning. NT-Xent loss or mean squared error loss is used to optimize the HSV feature representation. Alternatively, self-supervised methods such as MAE or SimMIM can be used, where a portion of the image is masked, and then an encoder-decoder is trained to reconstruct the image.

[0082] If labeled images are used for supervised pre-training, models with architectures such as CNN and transformer can be trained on the pre-training dataset for tasks such as image classification or object detection.

[0083] Regardless of which method is used, the training process includes the following steps:

[0084] Step 2-310: Convert the RGB images in the pre-training dataset to HSV images;

[0085] Step 2-320: Perform image enhancement on the HSV image, keeping the H channel unchanged, and performing random enhancement operations on the V channel and / or S channel;

[0086] Step 2-330: Obtain the pre-training weights of the HSV hazardous materials identification model through pre-training.

[0087] After pre-training, remove the projection head, retaining the encoder backbone network used for feature extraction. Replace the classification head with an output layer suitable for hazardous materials identification, such as a multi-classifier.

[0088] Step 400: Identify hazardous materials based on the trained HSV hazardous materials identification model.

[0089] Based on the trained HSV hazardous materials identification model, hazardous materials identification is performed. The model is fine-tuned on a labeled dataset to accelerate convergence and reduce memory usage, thereby improving the performance and efficiency of the model in practical applications.

[0090] Fine-tuning on labeled datasets allows the trained HSV hazardous materials identification model to better adapt to specific hazardous materials identification tasks. Since pre-training or previous training is conducted on relatively general data or tasks, while actual hazardous materials identification scenarios have their own specificities, fine-tuning enables the model to learn more accurate features and patterns related to hazardous materials.

[0091] Using a smaller batch size reduces the memory required for each training session, allowing the model to be trained on devices with limited memory. At the same time, a smaller batch size makes gradient estimation more accurate because each batch of data is more likely to come from a similar distribution, which helps the model converge better.

[0092] Mixed-precision training reduces memory usage and computation by using a combination of half-precision floating-point numbers (FP16) and single-precision floating-point numbers (FP32) while maintaining model accuracy. Gradient clipping prevents gradient explosion and ensures the stability of the training process.

[0093] Setting checkpoints allows you to periodically save model parameters. If an unexpected interruption occurs during training, you can resume training from the most recent checkpoint, avoiding the waste of time and resources caused by repeated training. Cosine decay of the learning rate allows the learning rate to gradually decrease as training progresses. Using a larger learning rate in the early stages of training accelerates convergence, while a smaller learning rate in the later stages allows for fine-tuning, helping the model achieve better performance.

[0094] Reference Figures 3-6 , Figures 3-6 According to the standard, orange represents organic matter, green represents mixtures, and blue represents inorganic matter.

[0095] Reference Figure 3 and Figure 4 , Figure 4 The power bank labeled "powerbank 0.68" and several lighters (such as "lighter 0.86", "lighter 0.73", and "lighter 0.79") were identified. Power banks are hybrids with relatively complex shapes and internal structures. In traditional RGB space recognition, due to issues such as information coupling and enhanced interference, identification is difficult and the confidence level may be low. However, this invention utilizes the HSV space, where the H channel accurately reflects material properties. Even with the complex structure of the power bank, the model can still identify it, although the confidence level is 0.68 due to various factors. For lighters, adapting to HSV space features, even with partial occlusion or overlap with other objects, the model can still accurately capture lighter features and provide a high confidence level. Compared to traditional methods, this invention excels in distinguishing objects with different material properties and handling complex situations, significantly improving the accuracy and reliability of dangerous goods identification in security checks.

[0096] Reference Figure 5 and Figure 6 , Figure 5 The glass bottle was labeled and the confidence level was not mentioned in the description (judging from the frame, it is an object to be identified). Figure 6The confidence level is explicitly stated as 0.36. Glass bottles are inorganic materials; this invention utilizes the characteristics of the HSV space to enable the model to identify the material properties of the glass bottle.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for identifying dangerous goods in dual-energy security inspection images based on HSV spatial enhancement, characterized in that, Includes the following steps: Step 100: Convert the dual-energy RGB image to an HSV image; Step 200: Perform image enhancement on the HSV image, keeping the H channel unchanged, and performing random enhancement operations on the V channel and / or S channel; The HSV image is decomposed into three independent channels: H, S, and V. Random enhancement is performed on the V channel and / or the S channel. The enhanced V and / or S channels are then merged with the original H channel to obtain the enhanced HSV image. Step 300: Based on the enhanced HSV image, train the HSV hazardous materials identification model; Based on the RGB pre-trained hazardous materials identification model and the enhanced HSV image, a transfer training process is performed to obtain the HSV hazardous materials identification model, including: The input module of the RGB pre-trained hazardous materials identification model is the RGB input module, and the input module of the HSV hazardous materials identification model is the HSV input module. Set a loss function to compare the output differences between the HSV input module and the RGB input module; Randomly initialize the weights of the HSV input module, load the weights of the RGB input module, and fix the weights of the RGB input module; The HSV channel is used as the input to the HSV input module, and the RGB channel is used as the input to the RGB input module. The weights of the HSV input module are updated through training. The HSV input module is used to replace the input module of the RGB pre-trained hazardous materials identification model to form the HSV hazardous materials identification model. Step 400: Identify hazardous materials based on the trained HSV hazardous materials identification model.

2. The method for identifying dangerous goods in dual-energy security inspection images based on HSV spatial enhancement according to claim 1, characterized in that, The setup methods for the HSV input module include: The input module of the hazardous materials identification model based on RGB pre-training is denoted as the original input module: The original input module is copied into two independent modules: the HSV input module and the RGB input module.

3. The method for identifying dangerous goods in dual-energy security inspection images based on HSV spatial enhancement according to claim 1, characterized in that, The setup methods for the HSV input module include: The input module of the RGB pre-trained hazardous materials identification model is copied and denoted as the RGB input module, and the input module of the HSV hazardous materials identification model is redefined and denoted as the HSV input module.

4. The method for identifying dangerous goods in dual-energy security inspection images based on HSV spatial enhancement according to claim 1, characterized in that, Step 300 involves training an HSV hazardous materials identification model based on the enhanced HSV image, including: performing supervised or self-supervised training using the enhanced HSV image to obtain the trained HSV hazardous materials identification model.

Images