X-ray image contraband detection method based on multi-scale feature fusion

Abstract

The invention relates to an X-ray image contraband detection method based on multi-scale feature fusion, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an X-ray contraband data set, and dividing the data set into a training set and a test set; constructing an X-ray image contraband target detection model based on multi-scale feature fusion, wherein the model comprises an image branch, a self-adaptive high-low pass filter module, a text branch, a Neck layer and a Head head; images in the training set are input into the model to train the model; optimizing the model by adopting a loss function to obtain a trained model; and inputting the images in the test set into the trained model to obtain a contraband detection result. According to the method, through image and text joint training, the performance is improved in real-time task monitoring, the efficiency is higher, the calculation amount and memory occupation are reduced, and the problem that image boundary details are fuzzy in a complex scene is solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an X-ray image contraband detection method based on multi-scale feature fusion. Background Art

[0002] During the security check, an X-ray scanner is usually used to perform X-ray imaging of the passenger's luggage, and then professional security personnel will check whether there are any prohibited items. However, this link is very time-consuming to manually screen dangerous goods, and security personnel need to maintain long-term high-intensity work. In the long run, it is difficult to efficiently and accurately identify prohibited items in a large number of X-ray images, and human errors are prone to occur, resulting in reduced stability and accuracy of the security inspection system. In addition, due to the overlap of covered objects during X-ray scanning, there are serious overlapping occlusions, extreme clutter, coupling of foreground and background information, and edge blurring, so it is difficult to professionally distinguish dangerous objects. In addition, since the staff is prone to fatigue and distraction, manual identification is boring and prone to errors, not to mention catching up with the peak traffic hours and identifying various prohibited items within the specified time. Studies have shown that manual operators can only achieve a success rate of 80-90%. However, the detection of prohibited items requires detailed inspections, which requires both professionalism and vigilance. In recent years, people are in urgent need of proposing a real-time, accurate and efficient security inspection method. Therefore, conducting research on X-ray contraband target detection has important theoretical and practical significance.

[0003] Several automatic contraband detection systems have been proposed to simplify the baggage security screening process. However, unlike the superior results of traditional object detectors in RGB images, the performance of X-ray detectors is limited due to the inherent differences between X-ray and RGB images. In addition to the differences with natural images, other issues include severe class imbalance, limited datasets, severe occlusions, and little prior knowledge. Another issue is that the high classification variance within the contraband category further hinders the performance of contraband detection as these items are barely visible in X-ray scans. Eliminating and concealing these contraband items can lead to serious consequences. Current state-of-the-art methods are not powerful enough to cope with real-life challenges and stay ahead of these evolving contraband.

[0004] Researchers are committed to developing powerful and efficient systems to overcome these loopholes and provide reliable solutions for baggage contraband screening. However, these algorithms have not yet achieved perfect results in baggage security detection. Contraband security detection remains an open challenge. Therefore, it is imperative to develop more powerful, flexible, and real-time methods for baggage scanning contraband identification. The research community must design suitable methods to help effectively detect contraband during the security inspection process under various complex and changing situations. Therefore, faced with the unique image features of X-ray images and the evolving types of contraband, X-ray image contraband target detection is a more challenging and urgent problem to be solved. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a method for detecting contraband in X-ray images based on multi-scale feature fusion.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: The present invention provides a method for detecting contraband in X-ray images based on multi-scale feature fusion, comprising the following steps: S1. Obtain a data set of X-ray contraband items, and divide the data set into a training set and a test set; S2. Construct an X-ray image contraband target detection model based on multi-scale feature fusion, the model includes an image branch, an adaptive high-pass and low-pass filter module, a text branch, a Neck layer and a Head; X-ray images in the training set are input into the model to train the model; S3. Use the loss function to optimize the model and obtain a trained model; S4. The X-ray images in the test set are input into the trained model to obtain the contraband detection results.

[0007] Further, step S1 includes: Get an X-ray contraband dataset with N data images , each picture The composition is as follows: , , in, Indicates the number of images, Represents a vector that matches the number of categories in the dataset, index represents the specified class that appears in the dataset, =1 means the category appears in middle, =0 means not present middle, Represents the semi-transparent sub-images corresponding to different categories in the image, Target object categories considered as total contraband, Represents all objects in the image that are not related to contraband, called background classes. , Indicates the number of categories in the data set; Representing images Whether the prohibited object category exists, 1 means it exists, and 0 means it does not exist; Indicates the total prohibited items category Individual contraband target object categories in ; Representing images Individual contraband target object categories in ; Represents the total background class A single background class in ; Representing images A single background category in .

[0008] Furthermore, in step S2, in the image branch, the training set X-ray images are input into the improved YOLOv8 backbone network for feature extraction to obtain multi-scale features, specifically: The improved YOLOv8 backbone network adopts the C3k2 structure of YOLO11 to replace the C2f structure, and adds the C2PSA module of YOLO11 after the last layer of SPPF module of the backbone network; replaces the C3k convolution operation in the C3k2 structure with a deformable convolution, and the deformable convolution is a deformable convolution network DCN; The X-ray images in the training set are input into the improved YOLOv8 backbone network for feature extraction to obtain the first feature , Second feature and the third feature .

[0009] Furthermore, the adaptive high-pass and low-pass filter modules in step S2 are specifically: The adaptive high-pass and low-pass filter module includes an adaptive low-pass filter generator and an adaptive high-pass filter generator; the adaptive low-pass filter generator includes a convolution layer, a depth-separable convolution layer, and a reshaping and normalization layer; the adaptive high-pass filter generator includes a convolution layer, a reshaping and normalization layer, and a unit kernel subtraction layer; The first feature , Second feature and the third feature After the adaptive high-pass and low-pass filter modules are used for feature fusion enhancement, the first enhanced feature is obtained. , the second enhanced feature And the third enhancement feature , the specific process is: The third feature As a high-level feature and a second feature Perform feature fusion enhancement to obtain the second enhanced feature The specific steps are as follows: In the adaptive low-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive low-pass filter generator are obtained. , the features after convolution A depth-wise separable inverse convolution operation is performed through a depth-wise separable convolution layer to generate a feature map group, which is then reshaped and normalized to obtain a low-pass filter kernel. ; The third feature With the generated low-pass filter kernel Perform convolution operation to obtain intermediate features , intermediate features After the depth-separable convolution operation, the enhanced third feature is obtained ; In the adaptive high-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive high-pass filter generator are obtained. , the second feature after convolution After reshaping and normalizing the layer, the low-pass filter kernel in the adaptive high-pass filter generator is obtained , low-pass filter kernel After the unit kernel subtraction layer, the unit kernel E is used for subtraction operation to obtain the anti-kernel, and then the high-pass filter kernel is obtained. , the weight of the unit core E is [[0,0,0], [0,1,0], [0,0,0]]; the second feature With the generated high-pass filter kernel Perform convolution operation and then residual operation to obtain the enhanced second feature ; The enhanced third feature and the enhanced second feature Fusion is performed to obtain the second enhanced feature Similarly, the second enhancement feature As an advanced feature and a first feature Perform feature fusion enhancement to obtain the first enhanced feature ; The third enhanced feature The third feature without any treatment .

[0010] Furthermore, in the text branch of step S2, the training set is passed through the rich text generation module to obtain a rich text representation, and the rich text representation is input into the language model to generate text features, specifically: In the rich text generation module, the positioning target center of the prohibited item category in each image in the training set is extracted, and the position of the prohibited item in the entire image is indicated by words with directional expressions, and the directional expressions include upper left, upper middle, upper right, middle left, middle, middle right, lower left, lower middle, and lower right; the rich text of the prohibited items in each image is generated through the prescribed sentence template format, and the rich text representation is obtained according to the feature description of the prohibited items in each image and the sentence template filled with category and location information. T ; Rich text representation T Input into the pre-trained RoBERTa model to generate text features W, the formula is as follows: , in, Indicates A rich text representation, Indicates the RoBERTa language model input i Rich text representation Generated text features, represents the pre-trained RoBERTa language model, express The number of nouns, Represents the embedding dimension.

[0011] Furthermore, the Neck layer in step S2 includes an upsampling layer, a downsampling layer and a T-CSPLayer module; the first enhanced feature , the second enhanced feature , the third enhanced feature And the text feature W is input into the Neck layer for fusion to obtain the first fusion feature , the second fusion feature , the third fusion feature , the process is as follows: The third enhancement feature Perform a 2x upsampling operation, The upsampled features and the second enhanced features Perform splicing in the channel dimension to obtain the first splicing feature , the first splicing feature The text feature W is fused through the T-CSPLayer module to obtain the fused feature , the formula is as follows: , in, represents the Sigmoid function, Represents the text feature vectors, max means finding the maximum value; for fusion features Perform a 2x upsampling operation, Upsampled features and first enhanced features Perform splicing in the channel dimension to obtain the second splicing feature , the second splicing feature The text feature W is fused through the T-CSPLayer module to obtain the first fusion feature ; For the first fusion feature Perform downsampling operation, Downsampled features and fused features Splicing is performed on the channel dimension to obtain the third splicing feature , the third splicing feature The text feature W is fused through the T-CSPLayer module to obtain the second fusion feature ; For the second fusion feature Perform downsampling operation, Downsampled features and third enhanced features Splicing is performed on the channel dimension to obtain the fourth splicing feature , the fourth splicing feature The text feature W is fused through the T-CSPLayer module to obtain the third fusion feature .

[0012] Furthermore, the Head in step S2 includes a text comparison head and a bounding box detection; The first fusion feature , the second fusion feature and the third fusion feature Input into the bounding box detection to perform bounding box prediction and obtain the bounding box prediction result; the first fusion feature , the second fusion feature , the third fusion feature And text features Input the text into the text comparison head to calculate the text similarity and obtain the similarity result. The text category with the highest similarity result is the detection category corresponding to the bounding box, and then the contraband detection result is obtained. The formula is as follows: , in, represents normalization processing, Indicates that the fused features are convolved to obtain the image feature embedding vector , Indicates the number of targets, represents a learnable scaling factor used to adjust the scale of the score, Represents a zero-initialized learnable offset.

[0013] Furthermore, step S3 is specifically as follows: The loss function includes the regional text contrast loss and bounding box regression loss; the bounding box regression loss includes IoU loss and distribution focal loss , the regional text contrast loss , IoU loss and distribution focal loss Combine them to get the overall training loss function, which is expressed as follows: ， in, Represents the weight factor.

[0014] The advantages of the present invention are: The present invention can better capture and express the detailed information in the image through joint training of image and text, and is more efficient while maintaining performance in real-time monitoring tasks, reducing the amount of calculation and memory usage. It solves the problem of blurred image boundary details in complex scenes. In addition, in addition to the basic category set of the training model, the model is more generalized and robust when detecting new contraband objects; it achieves excellent results in the relevant X-ray contraband data set, with higher recognition performance and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0016] Figure 1 is a flow chart of the steps of the method of the present invention; Figure 2 The model architecture of the adaptive high-pass and low-pass filter module in the method of the present invention; Figure 3 The adaptive low-pass filter generator model architecture in the adaptive high- and low-pass filter module of the method of the present invention; Figure 4 The adaptive high-pass filter generator model architecture in the adaptive high- and low-pass filter module of the method of the present invention; Figure 5 This is a visualization result diagram of the method of the present invention in the experiment. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1 In this embodiment, Figure 1 As shown, the present invention provides a method for detecting contraband in X-ray images based on multi-scale feature fusion, and the specific steps include: S1. Obtain the X-ray contraband dataset and divide it into a training set and a test set.

[0019] Specifically, obtain an X-ray contraband dataset with N data images , each picture The composition is as follows: , , in, Indicates the number of images, Represents a vector that matches the number of categories in the dataset, index represents the specified class that appears in the dataset, =1 means the category appears in middle, =0 means not present middle, Represents the semi-transparent sub-images corresponding to different categories in the image, Target object categories considered as total contraband, Represents all objects in the image that are not related to contraband, called background classes. , Indicates the number of categories in the data set; Representing images Whether the prohibited object category exists, 1 means it exists, and 0 means it does not exist; Indicates the total prohibited items category Individual contraband target object categories in ; Representing images Individual contraband target object categories in ; Represents the total background class A single background class in ; Representing images A single background category in .

[0020] S2. Construct an X-ray image contraband target detection model based on multi-scale feature fusion, the model includes an image branch, an adaptive high-pass and low-pass filter module, a text branch, a Neck layer and a Head; the X-ray images in the training set are input into the model to train the model.

[0021] Specifically, in the image branch, an improved YOLOv8 backbone network is used to extract features to obtain multi-scale features: The improved YOLOv8 backbone network adopts the C3k2 structure of YOLO11 to replace the C2f structure, and adds the C2PSA module of YOLO11 after the last layer of SPPF module in the backbone network; replaces the C3k convolution operation in the C3k2 structure with a deformable convolution, and the deformable convolution is a deformable convolution network DCN; the X-ray image in the training set is input into the improved YOLOv8 backbone network for feature extraction, and the first feature , Second feature and the third feature .

[0022] Specifically, the adaptive high-pass and low-pass filter module includes an adaptive low-pass filter generator and an adaptive high-pass filter generator; the adaptive low-pass filter generator includes a convolution layer, a depth-separable convolution layer, and a reshaping and normalization layer; the adaptive high-pass filter generator includes a convolution layer, a reshaping and normalization layer, and a unit kernel subtraction layer; The first feature , Second feature and the third feature After the adaptive high-pass and low-pass filter modules are used for feature fusion enhancement, the first enhanced feature is obtained. , the second enhanced feature And the third enhancement feature , the specific process is: The third feature As a high-level feature and a second feature Perform feature fusion enhancement to obtain the second enhanced feature The specific steps are as follows: In the adaptive low-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive low-pass filter generator are obtained. , the features after convolution A depth-wise separable inverse convolution operation is performed through a depth-wise separable convolution layer to generate a feature map group, which is then reshaped and normalized to obtain a low-pass filter kernel. ; The third feature With the generated low-pass filter kernel Perform convolution operation to obtain intermediate features , intermediate features After the depth-separable convolution operation, the enhanced third feature is obtained ; In the adaptive high-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive high-pass filter generator are obtained. , the second feature after convolution After reshaping and normalizing the layer, the low-pass filter kernel in the adaptive high-pass filter generator is obtained , low-pass filter kernel After the unit kernel subtraction layer, the unit kernel E is used for subtraction operation to obtain the anti-kernel, and then the high-pass filter kernel is obtained. , the weight of the unit core E is [[0,0,0], [0,1,0], [0,0,0]]; the second feature With the generated high-pass filter kernel Perform convolution operation and then residual operation to obtain the enhanced second feature ; The enhanced third feature and the enhanced second feature Fusion is performed to obtain the second enhanced feature Similarly, the second enhancement feature As an advanced feature and a first feature Perform feature fusion enhancement to obtain the first enhanced feature .

[0023] The third enhanced feature The third feature without any treatment .

[0024] Specifically, in the text branch, the training set is passed through the rich text generation module to obtain a rich text representation, which is then input into the language model to generate text features: In the rich text generation module, the positioning target center of the prohibited item category in each image in the training set is extracted, and the position of the prohibited item in the entire image is indicated by words with directional expressions, and the directional expressions include upper left, upper middle, upper right, middle left, middle, middle right, lower left, lower middle, and lower right; the rich text of the prohibited items in each image is generated through the prescribed sentence template format, and the rich text representation is obtained according to the feature description of the prohibited items in each image and the sentence template filled with category and location information. T ; Rich text representation T Input into the pre-trained RoBERTa model to generate text features W, the formula is as follows: , in, Indicates A rich text representation, Indicates the RoBERTa language model input i Rich text representation Generated text features, represents the pre-trained RoBERTa language model, express The number of nouns, Represents the embedding dimension.

[0025] Specifically, the Neck layer includes an upsampling layer, a downsampling layer and a T-CSPLayer module; the first enhanced feature , the second enhanced feature , the third enhanced feature And the text feature W is input into the Neck layer for fusion to obtain the first fusion feature , the second fusion feature , the third fusion feature , the process is as follows: The third enhancement feature Perform a 2x upsampling operation, The upsampled features and the second enhanced features Perform splicing in the channel dimension to obtain the first splicing feature , the first splicing feature The text feature W is fused through the T-CSPLayer module to obtain the fused feature , the formula is as follows: , in, represents the Sigmoid function, Represents the text feature vectors, max means finding the maximum value; for fusion features Perform a 2x upsampling operation, Upsampled features and first enhanced features Perform splicing in the channel dimension to obtain the second splicing feature , the second splicing feature The text feature W is fused through the T-CSPLayer module to obtain the first fusion feature .

[0026] For the first fusion feature Perform downsampling operation, Downsampled features and fused features Splicing is performed on the channel dimension to obtain the third splicing feature , the third splicing feature The text feature W is fused through the T-CSPLayer module to obtain the second fusion feature .

[0027] For the second fusion feature Perform downsampling operation, Downsampled features and third enhanced features Splicing is performed on the channel dimension to obtain the fourth splicing feature , the fourth splicing feature The text feature W is fused through the T-CSPLayer module to obtain the third fusion feature .

[0028] Specifically, the Head includes a text comparison header and a bounding box detection: The first fusion feature , the second fusion feature and the third fusion feature Input into the bounding box detection to perform bounding box prediction and obtain the bounding box prediction result; the first fusion feature , the second fusion feature , the third fusion feature And text features Input the text into the text comparison head to calculate the text similarity and obtain the similarity result. The text category with the highest similarity result is the detection category corresponding to the bounding box, and then the contraband detection result is obtained. The formula is as follows: , in, represents normalization processing, Indicates that the fused features are convolved to obtain the image feature embedding vector , Indicates the number of targets, represents a learnable scaling factor used to adjust the scale of the score, Represents a zero-initialized learnable offset.

[0029] S3. Use the loss function to optimize the model and obtain a trained model.

[0030] Specifically, the classification score, bounding box prediction, bounding box distribution prediction and true label predicted by the model are matched, and then the classification loss, bounding box regression loss and distribution regression loss are calculated based on the matching results. The loss function includes regional text contrast loss and bounding box regression loss; the bounding box regression loss includes IoU loss and distribution focal loss , the regional text contrast loss , IoU loss and distribution focal loss Combine them to get the overall training loss function, which is expressed as follows: ， in, Represents the weight factor.

[0031] S4. The X-ray images in the test set are input into the trained model to obtain the contraband detection results.

[0032] Example 2 The present invention uses the PIXray dataset, which contains 8885 images of various contraband categories, of which the training set is 7109. The proposed X-ray image contraband detection method based on multi-scale feature fusion is evaluated on the dataset.

[0033] Table 1 Comparison of the performance of the SOTA method and the proposed method on the PIXray dataset The SOTA method for this dataset refers to the foreground-background specific feature learning framework ForkNet proposed by Tong Jia et al. in 2024, which is used to solve the overlap and occlusion problems of X-ray images. As can be seen from Table 1, the method of the present invention is superior to the initial architecture method in the dataset evaluation. The average detection accuracy of the method of the present invention is significantly higher than that of the SOTA. The experimental results show that the method of the present invention has a good detection effect, and the extracted features have strong discrimination and generalization capabilities.

[0034] Figure 5 This is a display diagram of the results obtained by the method of the present invention. An input image is subjected to the method of the present invention to obtain a predicted result diagram. From the predicted result diagram obtained by the method of the present invention, it can be seen that the method of the present invention can detect contraband very well, and there is not much difference from the real image. This proves that the method of the present invention can achieve the expected effect.

[0035] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting contraband in X-ray images based on multi-scale feature fusion, characterized in that: The following steps are involved: S1. Obtain a data set of X-ray contraband items, and divide the data set into a training set and a test set; S2. Construct an X-ray image contraband target detection model based on multi-scale feature fusion, the model includes an image branch, an adaptive high-pass and low-pass filter module, a text branch, a Neck layer and a Head; X-ray images in the training set are input into the model to train the model; S3. Use the loss function to optimize the model and obtain a trained model; S4. The X-ray images in the test set are input into the trained model to obtain the contraband detection results.

2. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 1 is characterized in that: Step S1 includes: Get an X-ray contraband dataset with N data images , each picture The composition is as follows: ， ， in, Indicates the number of images, Represents a vector that matches the number of categories in the dataset, index represents the specified class that appears in the dataset, =1 means the category appears in middle, =0 means not present middle, Represents the semi-transparent sub-images corresponding to different categories in the image, Target object categories considered as total contraband, Represents all objects in the image that are not related to contraband, called background classes. , Indicates the number of categories in the data set; Representing images Whether the prohibited object category exists, 1 means it exists, and 0 means it does not exist; Indicates the total prohibited items category Individual contraband target object categories in ; Representing images Individual contraband target object categories in ; Represents the total background class A single background class in ; Representing images A single background category in .

3. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 1 is characterized in that: Step S2, in the image branch, the training set X-ray images are input into the improved YOLOv8 backbone network for feature extraction to obtain multi-scale features, specifically: The improved YOLOv8 backbone network adopts the C3k2 structure of YOLO11 to replace the C2f structure, and adds the C2PSA module of YOLO11 after the last layer of SPPF module of the backbone network; replaces the C3k convolution operation in the C3k2 structure with a deformable convolution, and the deformable convolution is a deformable convolution network DCN; The X-ray images in the training set are input into the improved YOLOv8 backbone network for feature extraction to obtain the first feature , Second feature and the third feature .

4. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 3 is characterized in that: Step S2: The adaptive high-pass and low-pass filter modules are specifically: The adaptive high-pass and low-pass filter module includes an adaptive low-pass filter generator and an adaptive high-pass filter generator; the adaptive low-pass filter generator includes a convolution layer, a depth-separable convolution layer, and a reshaping and normalization layer; the adaptive high-pass filter generator includes a convolution layer, a reshaping and normalization layer, and a unit kernel subtraction layer; The first feature , Second feature and the third feature After the adaptive high-pass and low-pass filter modules are used for feature fusion enhancement, the first enhanced feature is obtained. , the second enhanced feature And the third enhancement feature , the specific process is: The third feature As a high-level feature and a second feature Perform feature fusion enhancement to obtain the second enhanced feature The specific steps are as follows: In the adaptive low-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive low-pass filter generator are obtained. , the features after convolution A depth-wise separable inverse convolution operation is performed through a depth-wise separable convolution layer to generate a feature map group, which is then reshaped and normalized to obtain a low-pass filter kernel. ; The third characteristic With the generated low-pass filter kernel Perform convolution operation to obtain intermediate features , intermediate features After the depth-separable convolution operation, the enhanced third feature is obtained ; In the adaptive high-pass filter generator, the second feature After the convolution operation is performed on the convolution layer, the features after convolution in the adaptive high-pass filter generator are obtained. , the second feature after convolution After reshaping and normalizing the layer, the low-pass filter kernel in the adaptive high-pass filter generator is obtained , low-pass filter kernel After the unit kernel subtraction layer, the unit kernel E is used for subtraction operation to obtain the anti-kernel, and then the high-pass filter kernel is obtained. , the weights of the unit core E are [[0,0,0], [0,1,0], [0,0,0]]; Second feature With the generated high-pass filter kernel Perform convolution operation and then residual operation to obtain the enhanced second feature ; The enhanced third feature and the enhanced second feature Fusion is performed to obtain the second enhanced feature ; Similarly, the second enhancement feature As an advanced feature and a first feature Perform feature fusion enhancement to obtain the first enhanced feature ; The third enhanced feature The third feature without any treatment .

5. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 4 is characterized in that: In step S2 text branch, the training set is passed through the rich text generation module to obtain a rich text representation, and the rich text representation is input into the language model to generate text features, specifically: In the rich text generation module, the positioning target center of the prohibited item category in each image in the training set is extracted, and the position of the prohibited item in the entire image is indicated by words with directional expressions, and the directional expressions include upper left, upper middle, upper right, middle left, middle, middle right, lower left, lower middle, and lower right; the rich text of the prohibited items in each image is generated through the prescribed sentence template format, and the rich text representation is obtained according to the feature description of the prohibited items in each image and the sentence template filled with category and location information. T ; Rich text representation T Input into the pre-trained RoBERTa model to generate text features W, the formula is as follows: ， in, Indicates A rich text representation, Indicates the RoBERTa language model input i Rich text representation Generated text features, represents the pre-trained RoBERTa language model, express The number of nouns, Represents the embedding dimension.

6. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 5 is characterized in that: The Neck layer in step S2 is specifically: The Neck layer includes an upsampling layer, a downsampling layer and a T-CSPLayer module; the first enhanced feature , the second enhanced feature , the third enhanced feature And the text feature W is input into the Neck layer for fusion to obtain the first fusion feature , the second fusion feature , the third fusion feature , the process is as follows: The third enhancement feature Perform a 2x upsampling operation, The upsampled features and the second enhanced features Perform splicing in the channel dimension to obtain the first splicing feature , the first splicing feature The text feature W is fused through the T-CSPLayer module to obtain the fused feature , the formula is as follows: ， in, represents the Sigmoid function, Represents the text feature vectors, max means finding the maximum value; for fusion features Perform a 2x upsampling operation, Upsampled features and first enhanced features Perform splicing in the channel dimension to obtain the second splicing feature , the second splicing feature The text feature W is fused through the T-CSPLayer module to obtain the first fusion feature ; For the first fusion feature Perform downsampling operation, Downsampled features and fused features Splicing is performed on the channel dimension to obtain the third splicing feature , the third splicing feature The text feature W is fused through the T-CSPLayer module to obtain the second fusion feature ; For the second fusion feature Perform downsampling operation, Downsampled features and third enhanced features Splicing is performed on the channel dimension to obtain the fourth splicing feature , the fourth splicing feature The text feature W is fused through the T-CSPLayer module to obtain the third fusion feature .

7. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 6 is characterized in that: The Head in step S2 is specifically: The Head includes a text comparison head and a bounding box detection; The first fusion feature , the second fusion feature and the third fusion feature Input into the bounding box detection to perform bounding box prediction and obtain the bounding box prediction result; the first fusion feature , the second fusion feature , the third fusion feature And text features Input the text into the text comparison head to calculate the text similarity and obtain the similarity result. The text category with the highest similarity result is the detection category corresponding to the bounding box, and then the contraband detection result is obtained. The formula is as follows: ， in, represents normalization processing, Indicates that the fused features are convolved to obtain the image feature embedding vector , Indicates the number of targets, represents a learnable scaling factor used to adjust the scale of the score, Represents a zero-initialized learnable offset.

8. The X-ray image contraband detection method based on multi-scale feature fusion according to claim 7 is characterized in that: Step S3 is specifically as follows: The loss function includes the regional text contrast loss and bounding box regression loss; the bounding box regression loss includes IoU loss and distribution focal loss , the regional text contrast loss , IoU loss and distribution focal loss Combine them to get the overall training loss function, which is expressed as follows: ， in, Represents the weight factor.

Images