Deep learning-based security check X-ray image target product segmentation and identification method
Patent Information
- Application Number
- CN202411771338.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-03-07
- Estimated Expiration
- Not applicable · inactive patent
Smart Images
Figure CN119579903A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision, and in particular to a method for segmenting and identifying target products in security inspection X-ray images based on deep learning. Background Art
[0002] Researchers and scholars at home and abroad have begun to apply computer vision technology to the field of security X-rays in order to solve some practical problems. For decades, research on the identification of contraband in security X-ray images has continued, which can be mainly divided into two categories: traditional image processing methods based on artificial features and detection and segmentation algorithms based on deep learning.
[0003] Compared with traditional image processing methods based on artificial features, the detection and segmentation algorithm based on deep learning can automatically learn and extract multi-layer semantic information in security inspection images and improve recognition accuracy.
[0004] The introduction of fully convolutional neural networks has opened up the research on semantic segmentation based on deep learning. In the field of contraband identification in security X-ray images, semantic segmentation algorithms can classify each pixel of an image and divide the image into regions with clear semantics, which can help security personnel obtain more useful information from X-ray images. Summary of the invention
[0005] The purpose of the present invention is to improve the accuracy and efficiency of segmentation and identification of target products (contraband) in security inspection X-ray images to meet the needs of complex scenarios and reduce the false detection and missed detection rate.
[0006] The technical solution of the present invention is to provide a method for segmenting and identifying target products in security inspection X-ray images based on deep learning, the method comprising:
[0007] Obtain a semantic segmentation dataset of security X-ray images and crop the dataset to a uniform size; divide the dataset into a training set and a test set, and subdivide the test set into subsets of different difficulty levels;
[0008] A neural network structure based on the HRNet backbone network is constructed, a de-occlusion module is embedded in the neural network structure, and features are extracted from the input security inspection X-ray image; the de-occlusion module is constructed based on the attention mechanism, and a material perception map and a boundary enhancement map are generated through multi-branch feature extraction and regional information aggregation;
[0009] The X-ray image input to the de-occlusion module is forwarded in two branches. In the first branch, after double boundary enhancement, an edge image is obtained, and the final boundary enhancement map is obtained after processing by N1 neural network blocks. In the other branch, the output image is spliced and fused with the edge image obtained by the first branch, and regional information aggregation is performed after processing by N2 neural network blocks, where N1 and N2 are both positive integers. Through the regional information aggregation network, the most appropriate feature map is selected from these feature maps as the final material perception map.
[0010] Next, the selected material perception map and the boundary enhancement map extracted by the first branch are spliced into a fused feature map; the fused feature map is used to generate an attention map through the attention map generation module; the attention map and the spliced image before being processed by the N2 neural network blocks are matrix multiplied to obtain the output feature map;
[0011] The output feature map is input into a multi-level feature aggregation module, and data-dependent upsampling, feature vector transformation and spatial rearrangement operations are performed layer by layer on the output feature map;
[0012] The neural network is trained using the training set and the performance is verified using the test set.
[0013] In any of the above technical solutions, further, the double boundary enhancement is implemented by a double boundary enhancement module, and the workflow of the module includes:
[0014] The boundary information of the feature map is enhanced in four directions: up, down, left, and right. Directional boundary extraction is used to ensure that the boundary information of each edge area is not lost, and it is aggregated in space to form a complete boundary map shape. Taking the left boundary information extraction as an example, the representation of the feature map before and after boundary enhancement is and It is calculated using the following formula:
[0015]
[0016] Where c is the number of channels of the feature map, i and j represent the ordinate and abscissa of the pixel points in the feature map, respectively, W represents the width of the feature map, and n is an arbitrary natural number used to control the range of boundary extraction;
[0017] Through the above calculation method, the significant information of the left boundary of the feature map can be extracted. The calculation methods for the upper, lower and right directions are similar.
[0018] After completing the directional boundary enhancement, the Sobel operator is applied to the feature map of each channel to obtain the edge information in the horizontal and vertical directions, generate edge images in two directions, merge the edge images in these two directions, generate a boundary information enhancement map and send it backward.
[0019] In any of the above technical solutions, further, regional information aggregation is implemented through a regional information aggregation network, and the workflow of the network includes:
[0020] The input image is subjected to average pooling of different scales, and then the pooling result is extended and re-expanded to the same size as the original size. The expanded image is spliced with the original image. The splicing integrates information at multiple scales and increases the number of channels of the feature map. The spliced feature map is passed through a convolutional neural network with a convolution kernel of 3×3 and gated to select the most appropriate feature map from the feature maps finally obtained by average pooling of different scales as the output of the regional information aggregation network.
[0021] In any of the above technical solutions, further, the work content of the attention map generation module includes:
[0022] First, the feature maps output by the dual boundary information enhancement module and the local material information perception module are spliced to fuse the feature information of the two modules; the spliced feature maps are compressed in the channel dimension using a 1×1 convolution kernel to generate feature maps for calculating the attention score. The calculation formula of this process is as follows:
[0023] F3=W l (F1‖F2)+b l ;
[0024] In the formula, They represent the feature maps output by the dual boundary information enhancement module and the feature maps output by the local material information perception module; ‖ represents the splicing operation, W l and b l are the parameters of the convolutional layer, Represents a concatenated and compressed image;
[0025] Then input F3 into the Sigmoid function (σ) to generate the attention score map S:
[0026]
[0027] Where S∈R H×W ;
[0028] Finally, the inner product of the attention score map S and the connection image P is calculated to obtain the final attention map
[0029]
[0030] Where P is the image after channel concatenation of the original input image and the boundary feature image.
[0031] In any of the above technical solutions, further, the workflow of the multi-level feature aggregation module includes:
[0032] Construct four branches with different resolutions, first upsample the feature map of the lowest resolution, then perform feature concatenation and fusion with the feature map of the adjacent resolution, then continue to upsample the concatenated feature map, and continue to perform concatenation and fusion with the feature map of the previous layer for the second time until the highest resolution. In this way, a total of three feature fusion operations are performed between feature maps of adjacent resolutions to avoid introducing too much information loss in a one-time upsampling.
[0033] The beneficial effects of the present invention are:
[0034] The technical solution of the present invention significantly improves the segmentation and recognition accuracy of target products (contraband) in security inspection scenarios by improving the existing semantic segmentation algorithm, and has the following beneficial effects:
[0035] Efficient segmentation and recognition capabilities: HRNet is used as the feature extraction backbone network to achieve effective fusion of multi-scale feature information while maintaining high-resolution features, enhancing the ability to process detail information in complex scenes; the de-occlusion module and dual boundary enhancement module are embedded to effectively deal with target occlusion and edge blur problems in the image, ensuring the accuracy of the recognition results.
[0036] Adaptable to a variety of scenarios and complex conditions: The regional information aggregation network enhances the perception of multi-scale targets through average pooling and gated convolution operations at different scales, and can adapt to the recognition tasks of simple scenes, complex occlusions and hidden target products; the data-dependent upsampling module reduces the loss of feature information, improves the model's ability to retain details and edge characteristics, and enhances its adaptability to complex images in real scenes.
[0037] Superior performance: Experiments on the PIDray dataset show that this method outperforms existing mainstream semantic segmentation algorithms in terms of indicators such as mIoU, mPA, and mPrecision, especially in complex scenarios and difficult tasks. Through layer-by-layer feature fusion and optimized decoding process, it effectively reduces misclassification and missed classification, ensuring that the segmentation results are smooth and close to the true label. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The advantages of the above and additional aspects of the present invention will become apparent and easily understood in the description of the embodiments in conjunction with the following drawings, in which:
[0039] Figure 1 is an overall network diagram of a method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to an embodiment of the present invention;
[0040] Figure 2 is a schematic diagram of a HRNet backbone network of a method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to an embodiment of the present invention;
[0041] Figure 3 It is a schematic diagram of the structure of a de-occlusion unit according to a method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to an embodiment of the present invention;
[0042] Figure 4 It is a schematic diagram of a regional information aggregation network structure based on a deep learning security inspection X-ray image target product segmentation and recognition method according to an embodiment of the present invention;
[0043] Figure 5 is a schematic diagram of a multi-level feature aggregation module of a method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to an embodiment of the present invention;
[0044] Figure 6 is a schematic diagram of an upsampling operation of a method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to an embodiment of the present invention;
[0045] Figure 7 This is a schematic diagram of visualization results of various models of a method for target product segmentation and recognition of security inspection X-ray images based on deep learning on a PIDray dataset according to an embodiment of the present invention. DETAILED DESCRIPTION
[0046] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.
[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.
[0048] like Figure 1 As shown, this embodiment provides a method for segmenting and identifying target products in security inspection X-ray images based on deep learning. In this embodiment, the target products are various types of prohibited items for security inspection, including guns, scissors, daggers, lighters, power banks, etc. The method includes:
[0049] The PIDray security X-ray image semantic segmentation dataset is obtained from security inspection devices produced by three different manufacturers in different real scenes such as airports, subway stations and railway stations. The images are uniformly cropped to 480×480 size. This dataset has a wide coverage and diversity, including 12 common prohibited items, with a total of 47,677 images.
[0050] The dataset is divided into a training set and a test set. The training set contains 29,457 images (about 60%) and the test set contains 18,220 images (about 40%). Each image contains the target product (contraband) and is annotated at the pixel level. In order to better evaluate the performance of the algorithm in different scenarios and complexities, the test set is subdivided into three image subsets: Easy, Hard, and Hidden. The Easy image subset means that the images in the test set contain only one contraband, the Hard image subset means that the images in the test set contain multiple contraband, and the Hidden image subset means that the contraband in the images in the test set is severely overlapped and occluded.
[0051] Construct a neural network architecture for image recognition. The neural network uses HRNet (High-Resolution Network) as the feature extraction backbone network and embeds a de-occlusion module. The backbone network structure is as follows: Figure 2 As shown in the figure, HRNet is a deep learning model that is particularly suitable for processing image tasks, such as image segmentation and object detection. Its main feature is that it can extract multi-level feature information while maintaining high resolution. HRNet uses multiple branches to work in parallel, and each branch is responsible for processing features of different scales, so that low-level detail information and high-level abstract features can be obtained at the same time. This design helps the model understand the image more comprehensively.
[0052] On the basis of maintaining the high-resolution features of the HRNet structure, the present invention enhances the feature transfer between different layers through dense connections, effectively processing the detailed information in the image; dense connections mean that each layer in the network is directly connected to all previous layers; in this structure, each layer not only receives input from the previous layer, but also receives output from all previous layers. In this way, the model can utilize rich feature information to improve feature transfer and reuse.
[0053] Specifically, the de-occlusion module embedded in HRNet is built based on the attention mechanism. Before being embedded in the backbone network of HRNet, the de-occlusion module enhances the network's perception of the occluded area and outputs a feature map with richer information for subsequent feature extraction.
[0054] like Figure 3As shown in the figure, the X-ray image input to the de-occlusion module is forwarded in two branches. On the first branch, after double boundary enhancement, an edge image is obtained, and the final boundary enhancement map is obtained after processing by N1 neural network blocks; on the other branch, the output image is spliced and fused with the edge image obtained by the first branch (all "splicing" in this article refers to the splicing of multiple images in the channel dimension, which will not be repeated below), and regional information aggregation is performed after processing by N2 neural network blocks, where N1 and N2 are both positive integers. Through the regional information aggregation network, the most appropriate feature map is selected from these feature maps as the final material perception map. Then, the selected material perception map and the boundary enhancement map extracted by the first branch are spliced into a fused feature map; the fused feature map generates an attention map through the attention map generation module, and the attention map marks which areas are worthy of attention. Finally, matrix multiplication is performed on this attention map and the spliced image before processing by N2 neural network blocks to obtain the output feature map, which emphasizes important features and makes the features more prominent in the X-ray image, which can be used for further feature extraction.
[0055] The double boundary enhancement is realized by constructing a double boundary enhancement module. The workflow of the double boundary enhancement module includes:
[0056] The boundary information of the feature map is enhanced in four directions: up, down, left, and right. Directional boundary extraction is used to ensure that the boundary information of each edge area is not lost, and it is aggregated in space to form a complete boundary map shape. Taking the left boundary information extraction as an example, the representation of the feature map before and after boundary enhancement is and It is calculated using the following formula:
[0057]
[0058] Among them, c is the number of channels of the feature map, i and j represent the ordinate and abscissa of the pixel points in the feature map respectively, W represents the width of the feature map, and n is an arbitrary natural number used to control the range of boundary extraction.
[0059] Through the above calculation method, the significant information of the left boundary in the feature map can be extracted. The calculation methods for the upper, lower and right directions are the same. The boundary enhancement operation in the four directions of left, right, upper and lower directions can effectively enhance the overall boundary information, making the edge contour of the feature map clearer in space.
[0060] After completing the directional boundary enhancement, the Sobel operator is applied to the feature map of each channel to obtain the edge information in the horizontal and vertical directions, generate edge images in two directions, merge the edge images in these two directions, generate a boundary information enhancement map and send it backward.
[0061] The Sobel operator is a classic edge detection operator that calculates the gradient of the image in the horizontal direction (x direction) and the vertical direction (y direction) through convolution operations to generate two edge feature maps. The gradient images in these two directions reflect the significant edge contours in the feature map, thereby further enhancing the boundary information of the image.
[0062] like Figure 4 As shown, regional information aggregation is achieved through a regional information aggregation network. This step can perceive local material information and make full use of material information to determine the type of contraband. This step includes:
[0063] The input image is subjected to average pooling of different scales to reduce the spatial resolution of the image in order to reduce the amount of computation while retaining key information. The pooling result is then extended and re-expanded to the same size as the original size. The expanded image is then spliced with the original image. The splicing integrates information at multiple scales and increases the number of channels in the feature map. The spliced feature map is subjected to a convolution operation to further extract and aggregate useful regional information.
[0064] In order to make the regional information aggregation operation perform well on prohibited items of various scales, a convolutional neural network with a convolution kernel of 3×3 and gated convolution is used to select the most appropriate feature map from the feature maps finally obtained by average pooling at different scales as the output of the regional information aggregation network; gated convolution is a control mechanism that dynamically adjusts the flow of information by learning the weights or importance between different feature maps. Through gating operations, it can more flexibly select which features are more important in different tasks or stages.
[0065] Specifically, the present invention adopts average pooling operations of 8, 16 and 24 times according to the differences in the sizes of pictures in the data set.
[0066] The attention map generation module first splices the feature maps output by the dual boundary information enhancement module and the local material information perception module to fuse the feature information of the two modules; the spliced feature map is compressed in the channel dimension using a 1×1 convolution kernel to generate a feature map for calculating the attention score. The calculation formula of this process is as follows:
[0067] F3=W l (F1‖F2)+b l ;
[0068] In the formula, They represent the feature maps output by the dual boundary information enhancement module and the feature maps output by the local material information perception module. ‖ represents the concatenation operation, W l and b l are the parameters of the convolutional layer, Represents a concatenated and compressed image.
[0069] Then input F3 into the Sigmoid function (σ) to generate the attention score map S:
[0070]
[0071] Where S∈R H×W .
[0072] Finally, the inner product of the attention score map S and the connection image P is calculated to obtain the final attention map
[0073]
[0074] Where P is the image after channel concatenation of the original input image and the boundary feature image.
[0075] The present invention also optimizes the decoding of the HRNet backbone network, designs a multi-level feature aggregation module, replaces the commonly used bilinear interpolation upsampling with data-dependent upsampling to reduce information loss, and adopts a layer-by-layer upsampling approach to better achieve multi-scale feature aggregation.
[0076] like Figure 5 As shown in the figure, four branches with different resolutions are constructed. In these four branches with different resolutions, the feature map with the lowest resolution is first upsampled, and then the feature map with the adjacent resolution is spliced and fused once, and then the spliced feature map is further upsampled, and then the second splicing and fusion with the feature map of the previous layer is performed until the highest resolution. In this way, a total of three feature fusion operations are performed, all of which are performed between feature maps with adjacent resolutions, effectively avoiding the introduction of too much information loss in a one-time upsampling. By gradually increasing the resolution of the feature map, the lost detail information is gradually restored, and the details and semantic information in the original image are better retained and utilized, thereby improving the performance and effect of the model, which helps to improve the model's ability to understand the input image.
[0077] The operation diagram of data-dependent upsampling is as follows Figure 6As shown in the figure. The figure shows how to upsample from a low-resolution input (F on the left) to obtain a high-resolution output (R on the right). On the left is a three-dimensional input with height H, width W, and number of channels C. For each 1×C feature vector, it is transformed by a weight matrix W with dimension C×N, and a 1×N feature vector is output, where N is the number of new feature points. DUpsampling further converts the 1×N feature vector into a 2×2×N / 4 structure through a rearrangement operation. Finally, a feature map R with a dimension of 2H×2W×N / 4 after upsampling is obtained. By observing the change in dimensions in the figure, it can be seen that the rearrangement process increases the spatial resolution (height and width) while reducing the number of channels to maintain the overall size of the feature map.
[0078] The training set is input into the constructed neural network for training. The neural network outputs the segmented image of the area where the prohibited items are located. After the training, the test set is input into the test set to test the recognition ability.
[0079] In another embodiment of the present invention, in order to verify the effectiveness of this algorithm in the segmentation and identification of contraband in security X-ray images, this technology conducted relevant experiments on the PIDray dataset based on the above-mentioned evaluation indicators, conducted comparative experimental analysis with other advanced algorithms, and retained the mainstream high-precision semantic segmentation algorithm for comparative experiments. We selected a public dataset from the real-world security inspection field as the experimental benchmark, trained and tested on the same dataset, and used mIoU, mPA and mPrecision as evaluation indicators to comprehensively evaluate the performance. We conducted experimental tests on the three test subsets (Easy, Hard, Hidden) of the dataset. In addition, this technology calculated the overall average mIoU, mPA and mPrecision by adding and averaging the three test subsets. The experimental results are shown in Tables 1, 2 and 3.
[0080] Table 1 Comparison of our model with other mainstream semantic segmentation models
[0081]
[0082] Table 2 Comparison of our model with other mainstream semantic segmentation models
[0083]
[0084]
[0085] Table 3 Comparison of our model with other mainstream semantic segmentation models
[0086]
[0087]
[0088] According to the experimental results, the improved algorithm of this technology has achieved the best performance in the three evaluation indicators in the semantic segmentation task of security inspection X-ray images, and it performs better than both the classic mainstream semantic segmentation model and the new model in recent years. Specifically, the algorithm achieved an average intersection-over-union ratio of 75.26%, 70.87%, and 59.05%, an average pixel precision of 83.00%, 80.95%, and 66.63%, and an average accuracy of 89.20%, 85.63%, and 79.87% on the three test subsets of the PIDray dataset. And on the overall test set, it achieved an average intersection-over-union ratio of 68.39%, an average pixel precision of 76.86%, and an average accuracy of 84.90%. It shows that the improved algorithm of this technology is more suitable for the segmentation and recognition of contraband in security inspection X-ray images than other high-precision algorithms.
[0089] In order to more intuitively demonstrate the actual segmentation effects of various algorithms on the segmentation and recognition of contraband in security X-ray images, some representative semantic segmentation visualization results are shown as follows: Figure 7 As shown in the figure, this helps us understand the advantages and disadvantages of this improved algorithm compared with other algorithms, so as to better evaluate the performance of various algorithms in the task of segmenting and identifying contraband in security X-ray images.
[0090] From the visualization results, we can see that the improved and optimized model is more accurate than other models in segmentation. It can also accurately segment and identify contraband of different scales, with almost no pixel misclassification. At the same time, the contour details of contraband are processed more smoothly and delicately, which is closer to the real label. Specifically, in the first security inspection X-ray image, HRNet and DeepLabv3+ have better segmentation results in the overlapping part of contraband.
[0091] The segmentation is not clear enough. Some pixels belonging to the hammer class are mistakenly segmented and recognized as pliers class pixels, and DeepLanv3+ does not recognize the occluded part of the hammer handle. In the segmentation and recognition of the hammer head, SegNeXt and U-Net both mistakenly classified some pixels into other categories. And U-Net's segmentation of the hammer handle has segmentation discontinuities. Although algorithms such as SegFormer and DenseASPP have a high classification accuracy for pixels, they have poor segmentation of contours, forming jagged contours and the segmentation is not smooth enough, which may be caused by insufficient processing of details.
[0092] In the second security inspection X-ray image, due to background occlusion, HRNet and SegNeXt both had segmentation discontinuities. Although other algorithms did not have discontinuities, the contour segmentation was obviously slightly rough, with some over-segmentation or under-segmentation, causing deformation of the contraband identified by segmentation. The algorithm improved and optimized by this technology did not have similar problems.
[0093] In the third security X-ray image, the scissors head and the mobile power bank overlap, and the scale difference between the two is large. The mobile power bank is blocked by multiple objects and interfered by the background, making it difficult to segment and identify them finely. In this case, the improved and optimized algorithm can still achieve a segmentation effect close to the true label. In contrast, most algorithms such as PSPNet and FCN only segment and identify some pixels of the mobile power bank. The segmentation effect of DeepLabv3+ is deformed. Although SegFormer has a good segmentation effect on the mobile power bank, it lacks the segmentation of the scissors part that overlaps with it. This also shows that the model has not achieved satisfactory results in the face of the multi-scale problem of contraband in security X-ray images, and also confirms the superiority of the improved and optimized algorithm in dealing with multi-scale contraband and the accuracy of contour shape segmentation.
[0094] In summary, the present invention provides a method for segmenting and identifying target products in security inspection X-ray images based on deep learning, the method comprising:
[0095] Obtain a security X-ray image semantic segmentation dataset and crop the dataset to a uniform size; divide the dataset into a training set and a test set, and subdivide the test set into subsets of different difficulty levels.
[0096] A neural network structure based on the HRNet backbone network is constructed, and a de-occlusion module is embedded in the neural network structure to perform feature extraction on the input security inspection X-ray image; the de-occlusion module is constructed based on the attention mechanism, and generates a material perception map and a boundary enhancement map through multi-branch feature extraction and regional information aggregation.
[0097] The X-ray image input to the de-occlusion module is forwarded in two branches. On the first branch, after double boundary enhancement, an edge image is obtained, and the final boundary enhancement map is obtained after processing by N1 neural network blocks; on the other branch, the output image is spliced and fused with the edge image obtained by the first branch, and regional information aggregation is performed after processing by N2 neural network blocks, where N1 and N2 are both positive integers. Through the regional information aggregation network, the most suitable feature map is selected from these feature maps as the final material perception map.
[0098] Next, the selected material perception map and the boundary enhancement map extracted by the first branch are spliced into a fused feature map; the fused feature map is used to generate an attention map through the attention map generation module; this attention map and the spliced image obtained before processing by the N2 neural network block are matrix multiplied to obtain the output feature map.
[0099] The output feature map is input into a multi-level feature aggregation module, and layer-by-layer data-dependent upsampling is used to reduce information loss. The spatial resolution is increased through feature vector transformation and spatial rearrangement operations, while the number of channels is reduced to retain image detail information.
[0100] The neural network is trained using the training set and the performance is verified using the test set.
[0101] The steps in the present invention can be adjusted in order, combined or deleted according to actual needs.
[0102] The units in the device of the present invention can be combined, divided and deleted according to actual needs.
[0103] Although the present invention is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present invention. The scope of protection of the present invention is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present invention.
Claims
1. A method for segmenting and identifying target products in security inspection X-ray images based on deep learning, characterized in that: The method comprises: Obtain a semantic segmentation dataset of security X-ray images and crop the dataset to a uniform size; divide the dataset into a training set and a test set, and subdivide the test set into subsets of different difficulty levels; A neural network structure based on the HRNet backbone network is constructed, a de-occlusion module is embedded in the neural network structure, and features are extracted from the input security inspection X-ray image; the de-occlusion module is constructed based on the attention mechanism, and a material perception map and a boundary enhancement map are generated through multi-branch feature extraction and regional information aggregation; The X-ray image input to the de-occlusion module is forwarded in two branches. In the first branch, after double boundary enhancement, an edge image is obtained, and the final boundary enhancement map is obtained after processing by N1 neural network blocks. In the other branch, the output image is spliced and fused with the edge image obtained by the first branch, and regional information aggregation is performed after processing by N2 neural network blocks, where N1 and N2 are both positive integers. Through the regional information aggregation network, the most appropriate feature map is selected from these feature maps as the final material perception map. Next, the selected material perception map and the boundary enhancement map extracted by the first branch are spliced into a fused feature map; the fused feature map is used to generate an attention map through the attention map generation module; the attention map and the spliced image before being processed by the N2 neural network blocks are matrix multiplied to obtain the output feature map; The output feature map is input into a multi-level feature aggregation module, and data-dependent upsampling, feature vector transformation and spatial rearrangement operations are performed layer by layer on the output feature map; The neural network is trained using the training set and the performance is verified using the test set.
2. The method for segmenting and identifying target products in security inspection X-ray images based on deep learning as claimed in claim 1, characterized in that: The double boundary enhancement is implemented by a double boundary enhancement module, and the workflow of the module includes: The boundary information of the feature map is enhanced in four directions: up, down, left, and right. Directional boundary extraction is used to ensure that the boundary information of each edge area is not lost, and it is aggregated in space to form a complete boundary map shape. Taking the left boundary information extraction as an example, the representation of the feature map before and after boundary enhancement is and It is calculated using the following formula: Where c is the number of channels of the feature map, i and j represent the ordinate and abscissa of the pixel points in the feature map, respectively, W represents the width of the feature map, and n is an arbitrary natural number used to control the range of boundary extraction; Through the above calculation method, the significant information of the left boundary of the feature map can be extracted. The calculation methods for the upper, lower and right directions are similar. After completing the directional boundary enhancement, the Sobel operator is applied to the feature map of each channel to obtain the edge information in the horizontal and vertical directions, generate edge images in two directions, merge the edge images in these two directions, generate a boundary information enhancement map and send it backward.
3. The method for segmenting and identifying target products in security inspection X-ray images based on deep learning as claimed in claim 1, characterized in that: The regional information aggregation is achieved through a regional information aggregation network, and the workflow of the network includes: The input image is subjected to average pooling of different scales, and then the pooling result is extended and re-expanded to the same size as the original size. The expanded image is spliced with the original image. The splicing integrates information at multiple scales and increases the number of channels of the feature map. The spliced feature map is passed through a convolutional neural network with a convolution kernel of 3×3 and gated to select the most appropriate feature map from the feature maps finally obtained by average pooling of different scales as the output of the regional information aggregation network.
4. The method for segmenting and identifying target products in security inspection X-ray images based on deep learning according to any one of claims 1 to 3, characterized in that: The work content of the attention map generation module includes: First, the feature maps output by the dual boundary information enhancement module and the local material information perception module are spliced to fuse the feature information of the two modules; the spliced feature maps are compressed in the channel dimension using a 1×1 convolution kernel to generate feature maps for calculating the attention score. The calculation formula of this process is as follows: <h2 style=";text-align:left;direction:ltr">F3=W<h2 style=";text-align:left;direction:ltr"> l <h2 style=";text-align:left;direction:ltr"> (F1‖F2)+b<h2 style=";text-align:left;direction:ltr"> l <h2 style=";text-align:left;direction:ltr"> ; In the formula, They represent the feature maps output by the dual boundary information enhancement module and the feature maps output by the local material information perception module; ‖ represents the splicing operation, W l and b l are the parameters of the convolutional layer, Represents a concatenated and compressed image; Then input F3 into the Sigmoid function (σ) to generate the attention score map S: Where S∈R H×W ; Finally, the inner product of the attention score map S and the connection image P is calculated to obtain the final attention map Where P is the image after channel concatenation of the original input image and the boundary feature image.
5. The method for segmenting and identifying target products in security inspection X-ray images based on deep learning as claimed in claim 1, characterized in that: The workflow of the multi-level feature aggregation module includes: Construct four branches with different resolutions, first upsample the feature map of the lowest resolution, then perform feature concatenation and fusion with the feature map of the adjacent resolution, then continue to upsample the concatenated feature map, and continue to perform concatenation and fusion with the feature map of the previous layer for the second time until the highest resolution. In this way, a total of three feature fusion operations are performed between feature maps of adjacent resolutions to avoid introducing too much information loss in a one-time upsampling.