Hazardous materials identification methods, apparatus and storage media

Abstract

This invention discloses a method, apparatus, and storage medium for identifying hazardous materials, relating to the field of artificial intelligence. The method includes a data processing stage, a training stage, and an inference stage. It leverages the advantages of object detection and integrates rich semantic information to correct object detection results using semantic segmentation. Simultaneously, channel attention and spatial attention mechanisms are introduced in feature extraction to optimize inter-domain differences, thereby reducing false positives and false negatives caused by object detection downsampling and X-ray machine inter-domain differences, improving detection performance and increasing recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method, apparatus and storage medium for identifying hazardous materials. Background Technology

[0002] In security inspection and identification systems, most currently, manual methods are used to inspect the X-ray images generated by the X-ray machine. Although this can save the step of opening the box for inspection, it requires a high level of professional competence from the inspectors. At the same time, the inspectors also need to have good patience and vigilance against dangerous objects.

[0003] However, with longer working hours, the decreased perception due to fatigue poses a significant risk of false positives and false negatives. Summary of the Invention

[0004] This invention addresses the problem of high false positives and false negatives in existing security inspection systems for identifying dangerous goods by providing a method, device, and storage medium for identifying dangerous goods.

[0005] The technical solution proposed by this invention to address the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides a method for identifying dangerous goods, applied in a security inspection system, the method comprising:

[0007] Data processing stage:

[0008] The acquired X-ray images are labeled with target detection to obtain target detection labels; the X-ray images are labeled with semantic segmentation to obtain semantic segmentation labels;

[0009] Training phase:

[0010] The backbone network is used to extract features from the X-ray image to obtain a first feature map. The convolutional layers of the backbone network use channel attention modules and spatial attention modules for feature extraction.

[0011] The first feature map is feature-encoded and a second feature map with more scale information is generated using a neck network;

[0012] The second feature map is input into the target detection head and the semantic segmentation head respectively to obtain the target prediction result and the semantic segmentation prediction result;

[0013] The target prediction result and the target detection label are used to calculate a first loss to optimize the target detection head; the semantic segmentation prediction result and the semantic segmentation label are used to calculate a second loss to optimize the semantic segmentation head.

[0014] Reasoning stage:

[0015] The X-ray image to be identified is input into the optimized semantic segmentation head to obtain semantic segmentation results, which include target regions of different categories of targets; the X-ray image to be identified is input into the optimized target detection head to obtain target detection results, which include target classification information and location information.

[0016] The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output.

[0017] Preferably, the convolutional layers of the backbone network utilize channel attention modules for feature extraction, including:

[0018] Average pooling is performed on the features of the first feature map to obtain the average channel descriptor; max pooling is performed on the features of the first feature map to obtain the maximum channel descriptor.

[0019] The average channel descriptor and the maximum channel descriptor are input into a shared network to generate a one-dimensional channel attention map.

[0020] Preferably, the channel attention calculation formula satisfies:

[0021]

[0022] Where W1 and W0 represent two convolutional layers, and σ represents the sigmoid function.

[0023] Preferably, the convolutional layers of the backbone network utilize a spatial attention module for feature extraction, including:

[0024] The features of the first feature map are subjected to average pooling to obtain an average spatial descriptor; the features of the first feature map are subjected to max pooling to obtain a maximum spatial descriptor.

[0025] The average spatial descriptor and the maximum spatial descriptor are input into a shared network to generate a two-dimensional spatial attention map.

[0026] Preferably, the spatial attention calculation formula satisfies:

[0027] ,

[0028] Among them, f 7×7 This indicates that the convolution kernel performs a 7×7 convolution operation.

[0029] Preferably, the step of calculating a first loss between the target prediction result and the target detection label to optimize the target detection head includes:

[0030] Using L location lossThe function performs position loss calculation, where L location loss The function satisfies the following formula:

[0031] L location loss = 1-(I0 / (A p +A g -I0)-((A c -(A p +A g -I0)) / A c ,

[0032] Where I0 represents the intersection of the predicted bounding box and the ground truth label box, and A p +A g -I0 represents the union of the predicted bounding boxes and the ground truth label boxes minus the intersection, A c It represents the smallest bounding rectangle of the predicted bounding box and the actual label bounding box;

[0033] Using L classes loss The function performs class loss calculation, L classes loss The function satisfies the following formula:

[0034] L classes loss = ,

[0035] Where, x n1 This represents the prediction result of the target detection head; x n2 This indicates the detection results of the semantic segmentation head.

[0036] Preferably, the step of calculating a second loss between the semantic segmentation prediction result and the semantic segmentation label includes:

[0037] Using L seg The function performs semantic segmentation loss calculation, satisfying the following formula:

[0038] ,

[0039] Here, α is used to balance the weights of positive and negative samples, y represents the true label value of the pixel; the prediction probability p∈[0,1] of the foreground class, (1-p)^γ and p^γ are used to modulate the weight of each sample, so that hard samples have higher weights.

[0040] Preferably, the intersection-union ratio (IU / U) calculation is performed on the semantic segmentation result and the target detection result, and the region with an overlap greater than a preset value is retained as the final target detection region and output, satisfying the following formula:

[0041] ,

[0042] Among them, I out Indicates the output result; xn1 This represents the prediction result of the target detection head; x n2 This indicates the detection results of the semantic segmentation head.

[0043] In a second aspect, the present invention provides a hazardous materials identification device, the device comprising:

[0044] The data processing module is used to perform target detection annotation on the acquired X-ray images to obtain target detection labels; and to perform semantic segmentation annotation on the X-ray images to obtain semantic segmentation labels.

[0045] The training module is used for:

[0046] The backbone network is used to extract features from the X-ray image to obtain a first feature map. The convolutional layers of the backbone network use channel attention modules and spatial attention modules for feature extraction.

[0047] The first feature map is feature-encoded and a second feature map with more scale information is generated using a neck network;

[0048] The second feature map is input into the target detection head and the semantic segmentation head respectively to obtain the target prediction result and the semantic segmentation prediction result;

[0049] The target prediction result and the target detection label are used to calculate a first loss to optimize the target detection head; the semantic segmentation prediction result and the semantic segmentation label are used to calculate a second loss to optimize the semantic segmentation head.

[0050] The inference module is used for:

[0051] The X-ray image to be identified is input into the optimized semantic segmentation head to obtain semantic segmentation results, which include target regions of different categories of targets; the X-ray image to be identified is input into the optimized target detection head to obtain target detection results, which include target classification information and location information.

[0052] The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output.

[0053] Thirdly, the present invention also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the hazardous materials identification method described above.

[0054] The beneficial effects of the technical solution provided by the embodiments of the present invention are as follows:

[0055] The hazardous materials identification method provided by this invention leverages the advantages of target detection and integrates rich semantic information to correct target detection results using semantic segmentation. Simultaneously, channel attention and spatial attention mechanisms are introduced into feature extraction to optimize inter-domain differences. This reduces false positives and false negatives caused by target detection downsampling and X-ray machine inter-domain differences, thereby improving detection performance and increasing identification accuracy. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart of one embodiment of the hazardous materials identification method provided by the present invention;

[0058] Figure 2 This is a schematic diagram of the functional modules of the hazardous materials identification device provided by the present invention in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0060] See Figure 1 This is a flowchart illustrating one embodiment of the hazardous materials identification method provided by the present invention. The hazardous materials identification method is mainly applied in security inspection systems. By processing and identifying X-ray images, it determines whether the items being inspected are hazardous materials, and identifies the type and location of the hazardous materials.

[0061] like Figure 1 As shown, the hazardous materials identification method may include the following steps:

[0062] S101: Data processing stage. In this stage, the main tasks are to perform target detection annotation on the acquired X-ray images to obtain target detection labels, and to perform semantic segmentation annotation on the X-ray images to obtain semantic segmentation labels. Specifically:

[0063] X-ray images can be acquired first. These images can be acquired and generated in real time by the X-ray machine, or they can be images stored in a specified directory.

[0064] Subsequently, based on the category and location information of the hazardous materials, the specific location of the hazardous materials in the corresponding X-ray images is marked using a MASK, thus obtaining the target detection label y1; at the same time, the X-ray images are calibrated using different pixel values ​​to classify the category of hazardous materials and their specific location in the corresponding X-ray images under different pixel values, thus obtaining the semantic segmentation label y2.

[0065] Subsequently, the X-ray image and the corresponding target detection label y1 and semantic segmentation label y2 can be enhanced. The enhancement process may include cropping, affine transformation and rotation transformation, etc., and after the processing, the X-ray processed image I, target detection label y1` and semantic segmentation label y2` are obtained.

[0066] S102: Training phase. In this phase, the YOLOv5 network is mainly used to extract features from and train the X-ray image I. Specifically:

[0067] (1) The X-ray image I is used to extract features using the backbone network to obtain the first feature map I0. The convolutional layer of the backbone network uses the channel attention module and the spatial attention module to extract features.

[0068] In this invention, since internal factors of X-ray machines, such as imaging principles, hardware parameters, and machine aging, can easily cause inter-domain offset, channel attention modules and spatial attention modules are introduced in the convolutional layers of the feature extraction network to reduce this inter-domain offset and optimize feature extraction.

[0069] In this embodiment, the channel attention module, the convolutional layers of the backbone network may use the channel attention module for feature extraction, including:

[0070] ① Perform average pooling on the features of the first feature map to obtain the average channel descriptor, which can be denoted as F. avg Max pooling is performed on the features of the first feature map to obtain the maximum channel descriptor, which can be denoted as F. max .

[0071] ② Input the average channel descriptor and the maximum channel descriptor into a shared network to generate a one-dimensional channel attention map, which can be denoted as M. c ∈R Cx1x1 .

[0072] Here, the shared network consists of a multilayer perceptron, and the channel attention calculation formula satisfies:

[0073]

[0074] Where W1 and W0 represent two convolutional layers, and σ represents the sigmoid function.

[0075] In the spatial attention module of this embodiment, the feature extraction of the convolutional layers of the backbone network using the spatial attention module may include:

[0076] ① Perform average pooling on the features of the first feature map to obtain the average spatial descriptor; perform max pooling on the features of the first feature map to obtain the maximum spatial descriptor;

[0077] ② Input the average spatial descriptor and the maximum spatial descriptor into a shared network to generate a two-dimensional spatial attention graph.

[0078] Here, the spatial attention calculation formula satisfies:

[0079] ,

[0080] Among them, f 7×7 This indicates that the convolution kernel performs a 7×7 convolution operation.

[0081] In this preferred embodiment, the channel attention module is used to extract features first, and then the spatial attention module is used to extract features.

[0082] (2) The first feature map is encoded and a second feature map with more scale information is generated by the neck network. That is, the first feature map I0 obtained by channel attention and spatial attention feature extraction is encoded by the feature encoder first, and then the encoded first feature map I0 is processed by the neck network to obtain the second feature map I1 with more scale information.

[0083] (3) Input the second feature map into the target detection head (head_objdet) and the semantic segmentation head (head_segment) respectively to obtain the target prediction result and the semantic segmentation prediction result. The target prediction result can be counted as x. n1 The semantic segmentation prediction result can be counted as x. n2 .

[0084] (4) Perform a first loss calculation on the target prediction result and the target detection label to optimize the target detection head; perform a second loss calculation on the semantic segmentation prediction result and the semantic segmentation label to optimize the semantic segmentation head.

[0085] In this step, calculating a first loss between the target prediction result and the target detection label to optimize the target detection head includes:

[0086] Using L location loss The function performs position loss calculation, where L location loss The function satisfies the following formula:

[0087] L location loss = 1-(I0 / (A p +A g -I0)-((A c -(A p +A g -I0)) / A c ,

[0088] Where I0 represents the intersection of the predicted bounding box and the ground truth label box, and A p +A g -I0 represents the union of the predicted bounding boxes and the ground truth label boxes minus the intersection, A c It represents the smallest bounding rectangle of the predicted bounding box and the actual label bounding box;

[0089] Using L classes loss The function performs class loss calculation, L classes loss The function satisfies the following formula:

[0090] L classes loss = .

[0091] In addition, L can be used objectness loss The function calculates the location regression loss and satisfies the following formula:

[0092] L classes loss = L objectness loss ,

[0093] And the loss of all samples obj satisfies:

[0094] L obj =L classes loss +L objectness loss +L location loss .

[0095] In this step, calculating the second loss between the semantic segmentation prediction result and the semantic segmentation label includes:

[0096] Using L seg The function performs semantic segmentation loss calculation, satisfying the following formula:

[0097] ,

[0098] Here, α is used to balance the weights of positive and negative samples, y represents the true label value of the pixel, the prediction probability of the foreground class is p∈[0,1], and (1-p) r times and p r times are used to modulate the weight of each sample so that hard samples have higher weights.

[0099] The total loss calculated above satisfies the following formula:

[0100] L = Llocation loss +L objectness loss +L classes loss +L seg .

[0101] S103: Inference Phase. In this phase, the X-ray image to be identified can be input into the trained network. Specifically:

[0102] (1) Input the X-ray image to be identified into the optimized semantic segmentation head to obtain the semantic segmentation result, the semantic segmentation result including the target regions of different categories of targets; input the X-ray image to be identified into the optimized target detection head to obtain the target detection result, the target detection result including the target classification information and location information.

[0103] (2) Calculate the intersection-union ratio (IUGR) of the semantic segmentation result and the target detection result, and retain the region with an overlap greater than a preset value as the final target detection region and output it. Here, the following formula is satisfied:

[0104] ,

[0105] Among them, I out Indicates the output result; x n1 This represents the prediction result of the target detection head; x n2 This represents the detection result of the semantic segmentation head. Here, the preset value is 0.5, meaning that regions with an overlap greater than 0.5 are preserved.

[0106] Compared to existing manual screening methods, which suffer from time-consuming training, high labor costs, and the risk of missed or false identifications of hazardous materials due to fatigue, current deep learning-based target detection and recognition methods often require multiple downsampling steps during feature extraction. Furthermore, the inter-domain differences caused by internal factors in X-ray imaging can easily lead to missed or false identifications. The hazardous materials identification method provided in this invention leverages the advantages of target detection and integrates rich semantic information to correct target detection results using semantic segmentation. Simultaneously, channel attention and spatial attention mechanisms are introduced into feature extraction to optimize for inter-domain differences. This reduces false and missed identifications caused by target detection downsampling and inter-domain differences in X-ray imaging, improving detection performance and recognition accuracy.

[0107] See Figure 2 This is a functional module diagram of the hazardous materials identification device provided by the present invention in one embodiment. The hazardous materials identification device 100 of this embodiment may include a data processing module 11, a training module 12, and a reasoning module 13, wherein:

[0108] The data processing module 11 is used to perform target detection annotation on the acquired X-ray images to obtain target detection labels; and to perform semantic segmentation annotation on the X-ray images to obtain semantic segmentation labels.

[0109] The training module 12 is used for:

[0110] The backbone network is used to extract features from the X-ray image to obtain a first feature map. The convolutional layers of the backbone network use channel attention modules and spatial attention modules for feature extraction.

[0111] The first feature map is feature-encoded and a second feature map with more scale information is generated using a neck network;

[0112] The second feature map is input into the target detection head and the semantic segmentation head respectively to obtain the target prediction result and the semantic segmentation prediction result;

[0113] The target prediction result and the target detection label are used to calculate a first loss to optimize the target detection head; the semantic segmentation prediction result and the semantic segmentation label are used to calculate a second loss to optimize the semantic segmentation head.

[0114] The inference module 13 is used for:

[0115] The X-ray image to be identified is input into the optimized semantic segmentation head to obtain semantic segmentation results, which include target regions of different categories of targets; the X-ray image to be identified is input into the optimized target detection head to obtain target detection results, which include target classification information and location information.

[0116] The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output.

[0117] It should be understood that after the corresponding module performs its corresponding function, the effect achieved can be the same as the aforementioned dangerous goods identification method, so it will not be repeated here.

[0118] The present invention provides a computer device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps in the above-described hazardous materials identification method.

[0119] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting various parts of the computer device via various interfaces and lines.

[0120] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0121] In addition, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the aforementioned hazardous materials identification method.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for identifying hazardous materials, applied in a security inspection system, characterized in that, The method includes: Data processing stage: The acquired X-ray images are labeled with target detection to obtain target detection labels; the X-ray images are labeled with semantic segmentation to obtain semantic segmentation labels; Training phase: The backbone network is used to extract features from the X-ray image to obtain a first feature map. The convolutional layers of the backbone network use channel attention modules and spatial attention modules for feature extraction. The first feature map is feature-encoded and a second feature map with more scale information is generated using a neck network; The second feature map is input into the target detection head and the semantic segmentation head respectively to obtain the target prediction result and the semantic segmentation prediction result; The target prediction result and the target detection label are used to calculate a first loss to optimize the target detection head; the semantic segmentation prediction result and the semantic segmentation label are used to calculate a second loss to optimize the semantic segmentation head. Reasoning stage: The X-ray image to be identified is input into the optimized semantic segmentation head to obtain semantic segmentation results, which include target regions of different categories of targets; the X-ray image to be identified is input into the optimized target detection head to obtain target detection results, which include target classification information and location information. The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output.

2. The hazardous materials identification method according to claim 1, characterized in that, The convolutional layers of the backbone network utilize channel attention modules for feature extraction, including: Average pooling is performed on the features of the first feature map to obtain the average channel descriptor; max pooling is performed on the features of the first feature map to obtain the maximum channel descriptor. The average channel descriptor and the maximum channel descriptor are input into a shared network to generate a one-dimensional channel attention map.

3. The hazardous materials identification method according to claim 2, characterized in that, The channel attention calculation formula satisfies: Where W1 and W0 represent two convolutional layers, and σ represents the sigmoid function.

4. The hazardous materials identification method according to claim 1, characterized in that, The convolutional layers of the backbone network utilize a spatial attention module for feature extraction, including: The features of the first feature map are subjected to average pooling to obtain an average spatial descriptor; the features of the first feature map are subjected to max pooling to obtain a maximum spatial descriptor. The average spatial descriptor and the maximum spatial descriptor are input into a shared network to generate a two-dimensional spatial attention map.

5. The hazardous materials identification method according to claim 4, characterized in that, The formula for calculating spatial attention satisfies: ， Among them, f 7×7 This indicates that the convolution kernel performs a 7×7 convolution operation.

6. The method for identifying hazardous materials according to any one of claims 1 to 5, characterized in that, The step of calculating a first loss between the target prediction result and the target detection label to optimize the target detection head includes: Using L location loss The function performs position loss calculation, where L location loss The function satisfies the following formula: L location loss = 1-(I0 / (A p +A g -I0)-((A c -(THE p +A g -I0)) / A c , Where I0 represents the intersection of the predicted bounding box and the ground truth label box, and A p +A g -I0 represents the union of the predicted bounding boxes and the ground truth label boxes minus the intersection, A c It represents the smallest bounding rectangle of the predicted bounding box and the actual label bounding box; Using L classes loss The function performs class loss calculation, where L classes loss The function satisfies the following formula: L classes loss = ， Where, x n1 This represents the prediction result of the target detection head; x n2 This indicates the detection results of the semantic segmentation head.

7. The method for identifying hazardous materials according to any one of claims 1 to 5, characterized in that, The step of calculating the second loss between the semantic segmentation prediction result and the semantic segmentation label includes: Using L seg The function performs semantic segmentation loss calculation, satisfying the following formula: ， Here, α is used to balance the weights of positive and negative samples, y represents the true label value of the pixel; the predicted probability p∈[0,1] of the foreground class, (1-p) to the power of γ and p to the power of γ are used to modulate the weight of each sample, so that hard samples have higher weights.

8. The method for identifying hazardous materials according to any one of claims 1 to 5, characterized in that, The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output, satisfying the following formula: ， Among them, I out Indicates the output result; x n1 This represents the prediction result of the target detection head; x n2 This indicates the detection results of the semantic segmentation head.

9. A hazardous materials identification device, characterized in that, The device includes: The data processing module is used to perform target detection annotation on the acquired X-ray images to obtain target detection labels; and to perform semantic segmentation annotation on the X-ray images to obtain semantic segmentation labels. The training module is used for: The backbone network is used to extract features from the X-ray image to obtain a first feature map. The convolutional layers of the backbone network use channel attention modules and spatial attention modules for feature extraction. The first feature map is feature-encoded and a second feature map with more scale information is generated using a neck network; The second feature map is input into the target detection head and the semantic segmentation head respectively to obtain the target prediction result and the semantic segmentation prediction result; The target prediction result and the target detection label are used to calculate a first loss to optimize the target detection head; the semantic segmentation prediction result and the semantic segmentation label are used to calculate a second loss to optimize the semantic segmentation head. The inference module is used for: The X-ray image to be identified is input into the optimized semantic segmentation head to obtain semantic segmentation results, which include target regions of different categories of targets; the X-ray image to be identified is input into the optimized target detection head to obtain target detection results, which include target classification information and location information. The semantic segmentation result and the target detection result are subjected to intersection-union ratio (IU / R) calculation, and the regions with an overlap greater than a preset value are retained as the final target detection regions and output.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the hazardous materials identification method as described in any one of claims 1-8.

Images