Original resolution-based x-ray image package detection training method and device

Abstract

The application relates to an original resolution-based X-ray image package detection training method and device. A plurality of constraint conditions are set through a data loader, the key adaptability problem of an existing X-ray image package detection network in an X-ray security inspection scene can be solved, that is, the physical size distortion caused by discarding the fixed size input, the material property deviation caused by the conventional interpolation, and the data enhancement means contrary to the X-ray physical imaging principle are abandoned, the real physical structure and material characteristic information contained in the original image data are preserved to the greatest extent, the detection precision and the generalization ability of small size and multi-form contraband under a stacking shielding scene are significantly improved, and a reliable technical foundation is provided for high-precision and interpretable automatic security inspection.

Description

Technical Field

[0001] This application relates to the field of security inspection technology, and in particular to a training method and apparatus for package detection based on X-ray images at original resolution. Background Technology

[0002] As a key non-contact screening method in the public safety defense line, X-ray security inspection has the core task of real-time, accurate and interpretable detection and classification of prohibited items (such as knives, firearms, explosives, lithium batteries, liquid containers, etc.) in luggage / cargo through X-ray imaging and multi-energy spectral analysis (including pseudo-color, atomic number, high-energy / low-energy dual-energy spectra, etc.).

[0003] With the development of deep learning, object detection algorithms based on convolutional neural networks (CNNs) (such as Mask R-CNN and the YOLO series) have become mainstream solutions and have been rapidly transferred to security inspection scenarios due to their maturity in the field of natural images. However, the training paradigm commonly used by such models—forced fixed input size preprocessing (such as uniform resampling to 640x640 or 1024x1024) and some data augmentation techniques (such as MixUp and random rotation)—shows significant incompatibility when transferred to X-ray security inspection scenarios. In particular, forced scaling interferes with the real physical size information of the object contained in the image, and conventional interpolation algorithms are unable to accurately restore the material attenuation characteristics unique to X-ray imaging. Moreover, some data augmentation techniques destroy the inherent physical characteristics of X-ray projection data, which can easily exacerbate false detection and false negative problems when processing extra-long or small packages, thus weakening the practical application value of the system. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] The main objective of this disclosure is to propose a training method and apparatus for package detection based on original resolution X-ray images, which can significantly improve the detection accuracy and generalization ability of packages.

[0006] A first aspect of this application provides a training method for package detection based on raw resolution X-ray images, the method comprising:

[0007] In response to the training instruction of the X-ray image package detection model, multiple first X-ray images to be trained are acquired; the first X-ray images are original resolution X-ray images with package areas provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold.

[0008] A data loader is configured; the data loader is configured with at least the following three constraints:

[0009] The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and add an image data enhancement process obtained by stitching based on the original size of the first X-ray image.

[0010] The second constraint includes maintaining the physical information and geometric structure in the first X-ray image unchanged;

[0011] The third constraint includes: dynamically filling the plurality of first X-ray images;

[0012] The X-ray image package detection network is trained by loading the multiple first X-ray images according to the data loader, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items inside packages in X-ray images is obtained.

[0013] The X-ray image package detection training method based on the original resolution provided in this embodiment has at least the following beneficial effects:

[0014] This method sets multiple constraints through a data loader, which can solve the key adaptability problem of existing X-ray image package detection networks in X-ray security inspection scenarios. Specifically, by abandoning physical size distortion caused by fixed-size input, material property offset caused by conventional interpolation, and data augmentation methods that violate the physical imaging principles of X-rays, it can preserve the true physical structure and material property information contained in the original image data to the greatest extent. This can significantly improve the detection accuracy and generalization ability of small-sized and multi-shaped contraband in stacked and obscured scenarios, providing a reliable technical foundation for high-precision and interpretable automated security inspection.

[0015] A second aspect of this application provides a training apparatus for package detection based on raw resolution X-ray images, the apparatus comprising:

[0016] The image acquisition module is used to respond to the training instruction of the X-ray image package detection model and acquire multiple first X-ray images to be trained; the first X-ray image is an original resolution X-ray image with a package area provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold.

[0017] The loader setting module is used to configure the data loader; the data loader is configured with at least the following three constraints:

[0018] The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and add an image data enhancement process obtained by stitching based on the original size of the first X-ray image.

[0019] The second constraint includes maintaining the physical information and geometric structure in the first X-ray image unchanged;

[0020] The third constraint includes: dynamically filling the plurality of first X-ray images;

[0021] The model training module is used to train the corresponding X-ray image package detection network by loading the multiple first X-ray images according to the data loader, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items inside packages in X-ray images is obtained.

[0022] A third aspect of this application provides an electronic device including at least one controller and a memory for communicatively connecting to the controller; the memory stores instructions executable by the at least one controller to cause the at least one controller to perform a training method for detecting wrapping in an X-ray image based on the original resolution as described above.

[0023] A fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform a training method for detecting envelopes in X-ray images based on the original resolution, as described above.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic flowchart of an embodiment of a training method for package detection based on original resolution X-ray images provided in this application;

[0027] Figure 2This is a schematic diagram of the structure of the improved YoLov5 embodiment provided in this application;

[0028] Figure 3 This is a schematic diagram of an embodiment of an X-ray image package detection training device based on the original resolution provided in this application;

[0029] Figure 4 This is a schematic diagram of the structure of an embodiment of an electronic device provided in this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0031] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0032] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or function in a specific orientation, and therefore should not be construed as a limitation of this application.

[0033] like Figure 1 As shown in one embodiment of this application, a method for detecting packages in X-ray images based on the original resolution is provided. The method includes the following steps S100 to S300:

[0034] Step S100: In response to the training instruction of the X-ray image package detection model, acquire multiple first X-ray images to be trained;

[0035] Step S200: Set up the data loader; the data loader must have at least the following three constraints:

[0036] The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and to add an image data enhancement process obtained by stitching based on the original size of the first X-ray image.

[0037] The second constraint includes maintaining the physical information and geometric structure of the first X-ray image unchanged;

[0038] The third constraint includes: dynamically filling multiple first X-ray images;

[0039] Step S300: Load multiple first X-ray images according to the data loader to train the corresponding X-ray image package detection network, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items inside packages in X-ray images is obtained.

[0040] In this step S100, taking a computer as an example, after the user inputs the training instruction for the X-ray image package detection model, the system responds to the instruction and acquires multiple first X-ray images to be trained. The first X-ray images are original resolution X-ray images with package areas provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold.

[0041] The security inspection device here can be a security screening machine, such as a subway security screening machine or an airport security screening machine. After the user's package passes through the security screening machine, the machine generates an image of the package. This image is a raw resolution X-ray image provided by the security screening machine. The raw resolution here is determined by the model of the security screening machine or the security parameters set by the user, and there are no restrictions here.

[0042] The packaged region in a raw resolution X-ray image refers to the area occupied by the package within the image. Because in security inspection scenarios, there are often extremely long package images (i.e., X-ray images formed by stacked packages or packages blocking the conveyor belt and causing prolonged obstruction of the X-ray scanner's light beam), a first threshold is set to classify X-ray images into a first X-ray image and other X-ray images. This embodiment requires the first X-ray image as training data. Typically, the first threshold is a width threshold; that is, only when the width of the packaged region in an X-ray image is less than the first threshold is it considered a first X-ray image. The purpose here is that if the width is greater than the first threshold, the model might perform scaling operations on the X-ray image during training to avoid distortion of the X-ray image's geometry.

[0043] The multiple first X-ray images here can be used as multiple batches of images for training the model.

[0044] In step S200, the data loader is the core tool in PyTorch for loading and managing training data. PyTorch is an open-source machine learning library based on Python.

[0045] The key point of this embodiment lies in setting the constraints on the data loader, so that the data loader can call data for processing during model training, and the processing is subject to pre-set conditions.

[0046] The constraints of this embodiment include three points, which are described below:

[0047] The first point is to set up an improved mosaic enhancement process. This improved process removes the resizing operation (using interpolation algorithms to scale the first X-ray image to a fixed pixel size) from the traditional mosaic enhancement process. Mosaic enhancement is a common process in many object detection models (such as YOLO and Mask R-CNN). To avoid physical size distortion caused by the model discarding fixed-size inputs, material property shifts caused by conventional interpolation, and data enhancement methods that violate the principles of X-ray physical imaging, this embodiment improves the traditional mosaic enhancement process by removing the resizing operation caused by mosaic enhancement and directly stitching images based on the original size or sub-images of the original size.

[0048] The second point is to maintain the physical information and geometric structure of the first X-ray image unchanged, that is, to disable all enhancement methods that destroy the physical information and geometric structure of the X-ray image, such as Random HSV (color distortion), RandomPerspective (perspective distortion), Horizontal Flip (horizontal flip, destroying the semantics of the conveyor belt position), ScaleJitter (scale jitter, indirect scaling), and Mix Up (image mixing).

[0049] The third point includes: dynamically filling multiple first X-ray images, for example:

[0050] For a batch of N first X-ray images [ ] Calculate the maximum height of the batch =max( ) and maximum batch width =max( );

[0051] Pad each first X-ray image in the batch to a uniform size. The fill method can be selected from center fill, bottom right fill, or top left fill strategies (which must be consistent), and the fill value is the background value. Here, the background values ​​are: 255 for pseudo-color images, 65535 for high-energy / low-energy images, and 0 for atomic number images.

[0052] The dynamic filling here will not affect the real physical structure and material properties information contained in the original image data.

[0053] This embodiment adds constraints to the data loader. These steps can preserve the real physical structure and material properties information contained in the original image data to the greatest extent, thereby significantly improving the detection accuracy and generalization ability of small package images, and providing a reliable technical foundation for high-precision and interpretable automatic security inspection.

[0054] In step S300, an X-ray image package detection network is determined, such as the YOLOv5 model. Multiple first X-ray images are loaded using a data loader to train the model until the model training is completed, resulting in an X-ray image package detection model that can detect items inside packages in X-ray images.

[0055] The X-ray image package detection training method based on the original resolution provided in this embodiment has at least the following beneficial effects:

[0056] This method sets multiple constraints through a data loader, which can solve the key adaptability problem of existing X-ray image package detection networks in X-ray security inspection scenarios. Specifically, by abandoning physical size distortion caused by fixed-size input, material property offset caused by conventional interpolation, and data augmentation methods that violate the physical imaging principles of X-rays, it can preserve the true physical structure and material property information contained in the original image data to the greatest extent. This can significantly improve the detection accuracy and generalization ability of small-sized and multi-shaped contraband in stacked and obscured scenarios, providing a reliable technical foundation for high-precision and interpretable automated security inspection.

[0057] In some embodiments of this application, step S100, acquiring multiple first X-ray images to be trained, includes:

[0058] Step S110: Obtain the original resolution X-ray image of the package area provided by the security inspection device;

[0059] Step S120: If the size of the enclosed region in the original resolution X-ray image with the enclosed region is greater than a first threshold, determine the geometric center of the region.

[0060] Step S130: At the geometric center of the region, mark the coordinates of the contraband in the package and find the vertical column where there are no contraband items as a candidate column for slicing.

[0061] Step S140: Based on the underlying data of the original resolution X-ray image collected by the security inspection device, calculate the information content of the candidate columns that can be cut, and select the candidate columns with the least information content and closest to the geometric center of the region as the best cut columns.

[0062] Step S150: Based on the optimal slice column, the original resolution X-ray image with the wrapped region is divided into two first X-ray images.

[0063] In this embodiment, the X-ray sensor of the security inspection device S110 acquires X-ray data after penetrating the package and reconstructs a digital X-ray image. This image is at its original resolution, for example, 8000 pixels (width) × 6000 pixels (height).

[0064] In step S120, it is first determined whether the size of the packaged area is too large, requiring segmentation. The first threshold is a preset value, determined based on the maximum image size that the subsequent image analysis algorithm can efficiently process without scaling. If segmentation is required, the geometric center of the packaged area is calculated. The steps for calculating the geometric center are: first, determine the coordinates of the upper left corner of the bounding rectangle of the packaged area; then, determine the coordinates of the lower right corner; and finally, calculate the geometric center based on the coordinates of the upper left and lower right corners.

[0065] In step S130, firstly, it is necessary to obtain the location information of the contraband items marked within the package area. These markings can come from various sources:

[0066] Pre-marked: Coordinates of suspicious objects manually marked on the image by security personnel.

[0067] Automatic preliminary identification: A lightweight and fast preliminary contraband detection algorithm scans the entire image and outputs the bounding box coordinates of suspicious targets. This preliminary detection does not require high accuracy; its purpose is to quickly locate the approximate area where possible objects may exist.

[0068] Finding candidate columns for tiling: Using the coordinates of the geometric center point C as a reference, scan left and right to find a complete vertical column of pixels that runs from top to bottom through the entire package area, without crossing any marked areas of contraband. These found vertical columns are the "candidate columns for tiling".

[0069] In step S140, information content is an indicator of the complexity of image content. A column with less information content means that the image content in that area is simpler and more uniform, and cutting at that location would cause minimal damage to the image content. The calculation of information content is based on the image's underlying data.

[0070] After calculating the information content of all candidate columns for tiling, the columns with the lowest information content are first selected. If multiple columns have the same minimum information content, the column closest to the X-coordinate of the region's geometric center is chosen as the optimal tiling column. This is done to ensure that the two segmented sub-images are as symmetrical in size as possible, which is beneficial for load balancing.

[0071] In step S150, the original high-resolution X-ray image is divided into two sub-images (the first X-ray image) along the vertical line containing the determined optimal slice column. These two sub-images can then be fed into the subsequent model for processing in parallel, significantly improving processing efficiency while ensuring recognition accuracy.

[0072] This method actively seeks out regions with sparse distribution of contraband and simple image content for segmentation, effectively avoiding the target truncation problem caused by traditional simple segmentation. While improving the processing speed of large package images, it can segment extremely wide and long package images into at least two smaller package images, avoiding physical size distortion caused by the model discarding fixed-size input.

[0073] In some embodiments of this application, the underlying data includes at least one of a high-energy map, a low-energy map, and an atomic number map;

[0074] Information content includes at least one of information entropy and variance.

[0075] Here, the underlying data is set to at least one of high-energy, low-energy, and atomic number maps, corresponding to the category of the first X-ray image. For example, if the first X-ray image is a high-energy image, then the underlying data is also a high-energy image; if the first X-ray image is a low-energy image, then the underlying data is also a low-energy image. Information entropy and variance can fully reflect the information content of the X-ray image to improve the accuracy of segmentation.

[0076] In some embodiments of this application, the dynamic filling of multiple first X-ray images in step S200 includes:

[0077] Step S210: Determine the maximum height and maximum width among the multiple first X-ray images;

[0078] Step S220: Fill multiple first X-ray images to the maximum height and maximum width; wherein, the fill value is the background value of the first X-ray image; the multiple first X-ray images are filled in the same way, and the fill method includes any one of center fill, lower right fill, and upper left fill.

[0079] This embodiment first determines the maximum height and maximum width among multiple first X-ray images, and then fills each first X-ray image in the same way, which can ensure the consistency of training samples and improve the accuracy of model training.

[0080] In some embodiments of this application, the X-ray image package detection network is an improved YOLOv5 network;

[0081] The improved YOLOv5 network includes: an improved Backbone section, Neck section, and Head section; the network structure of the Neck and Head sections in the improved YOLOv5 network is consistent with the YOLOv5 network.

[0082] The first module in the improved Backbone section is the improved SPPF module. The network structure of the other modules is the same as that of the YOLOv5 network. The improved SPPF module includes: one 1x1 convolution, 3x3, 5x5, and 7x7 max pooling average concatenation, and one 1x1 convolution.

[0083] The original SPPF module in the Backbone section performs one 1x1 convolution + three serial 5x5 MaxPool operations + concatenation + one 1x1 convolution operation. When the feature map is small, the feature map results of the three serial 5x5 MaxPool operations will be repeated, increasing useless information and computation.

[0084] The improvement to the SPPF module in this embodiment is that the three consecutive 5x5 MaxPools are replaced with cascaded 3x3, 5x5, and 7x7 MaxPools. This setup retains a multi-scale receptive field while providing a finer-grained receptive field under dynamic input size, thereby improving the accuracy of model detection.

[0085] In some embodiments of this application, the calculation of grid coordinates and anchor grids in the Detect layer of the Head portion is moved forward to the forward propagation of the improved YOLOv5 network.

[0086] In this embodiment, since each change in the input image size causes a change in the feature map size, the grid coordinates... and anchor point grid The dimensions will also change, therefore they cannot be fixed during initialization. This embodiment will... and The calculations are moved to the inference process in real time, which allows the image size to be fixed during initialization, so as to preserve the real physical structure and material properties information contained in the original image data to the greatest extent.

[0087] In some embodiments of this application, after obtaining an X-ray image package detection model for detecting items inside a package in an X-ray image, the method further includes:

[0088] Deploy an X-ray image package detection model using a dynamic Shape model.

[0089] Compared to fixed-shape model deployments, package images generated by X-ray machines have arbitrary aspect ratios. Forcing a long, narrow suitcase (e.g., 2000x800 pixels) and a square cardboard box (e.g., 1200x1200 pixels) to be scaled down to 640x640 pixels would result in severe distortion. Dynamic-shape models can directly accept images of their original size, preserving the original aspect ratio. This ensures that the images input to the model are as close as possible to the real physical world, retaining the most complete feature information and thus maximizing the accuracy achieved during model training.

[0090] In this embodiment, deploying a dynamic shape model (such as YOLOv6's dynamic shape inference technology) to an X-ray image package detection model can significantly improve the efficiency and flexibility of X-ray image package detection.

[0091] For ease of understanding, such as Figure 2 As shown in one embodiment of this application, a method for detecting packages in X-ray images based on the original resolution is provided. This method includes the following:

[0092] Taking the YOLOv5 framework as an example, the specific solution is as follows:

[0093] Step S910, ultra-long package image processing; which includes steps S9110 to S9150:

[0094] In X-ray security inspection applications, especially in subway stations during peak hours, package stacking is a common phenomenon. Large numbers of packages clog the conveyor belt, causing the X-ray scanner's light beam to be obscured for extended periods, resulting in extremely wide and long package images. To address the memory / GPU overflow issues caused during training or deployment and to avoid resizing, an intelligent segmentation strategy is proposed. The specific steps are as follows:

[0095] Step S9110: Determine whether the original resolution X-ray image with package area provided by the security inspection machine is an extra-long package. The determination condition is that the image width of the package area is greater than the set threshold (e.g., 800).

[0096] Step S9120: Focus on the central area of ​​the image, and based on the coordinates of the prohibited items, find the vertical column where there are no prohibited items (background) and mark it as a candidate column for slicing.

[0097] Step S9130: Using the underlying data (high-energy map, low-energy map, and atomic number map, etc.), calculate the information entropy and variance of the candidate columns. Select the column with the least information (both low information entropy and variance) and closest to the geometric center as the optimal tiling column.

[0098] In step S9140, the image is divided into two sub-images along the optimal column, and the corresponding annotation information is modified. The sub-image serves as the first X-ray image.

[0099] In step S9150, if there are no suitable candidate columns in the central region (i.e., all contraband items are present in the central region), the search is expanded to non-central regions, with the search rules referring to S9130. If the found cutting column is located near the leftmost or rightmost boundary of the image, only one valid sub-image is cut out (the other side is an invalid small fragment, which can be discarded).

[0100] Step S920: Disable destructive data augmentation to ensure original resolution data loading;

[0101] Step S9210: Disable all enhancement methods that damage the physical information and geometry of X-ray images, such as RandomHSV, Random Perspective, Horizontal Flip, Scale Jitter, and Mix Up.

[0102] Step S9220, Modify Mosaic Enhancement: Remove the Resize operation caused by Mosaic enhancement, and directly stitch the images based on the original size or the original size sub-images, using the specified background value when filling.

[0103] Step S930, Custom Raw Resolution Data Loader: Within a training batch, each image is allowed to retain its original size. Dynamic filling is performed after analyzing the dimensions of all images within a batch, avoiding forced scaling;

[0104] Step S9310, for a batch Zhang Image [ ] Calculate the maximum height of the batch =max( ) and maximum batch width =max( );

[0105] Step S9320: Pad each image in the batch to a uniform size. The fill method can be selected from strategies such as center fill, bottom right fill, or top left fill (they must be consistent), and the fill value is the background value.

[0106] at last Mosaic enhancements are disabled during each training epoch. At this point, a custom data loader is used to ensure that the original size map is used as input, thus avoiding potential noise introduced by Mosaic from affecting model convergence.

[0107] Step S940: Determine the YOLOv5 network;

[0108] The YOLOv5 network architecture mainly includes: an improved Backbone (CSPDarknet), a Neck (FPN / PANet), and an improved Head (Detect head);

[0109] The CSP DarkNet architecture, as an improved backbone, offers advantages such as enhanced feature extraction capabilities, increased computational efficiency, improved accuracy, enhanced learning capabilities of CNNs, and mitigation of gradient vanishing and information loss. CSP DarkNet primarily consists of Conv, C2f, and SPPF modules.

[0110] Conv module: The foundational module in the backbone, it extracts features and passes them to the next layer. The module includes convolutional layers, batch normalization (BN), and activation functions (such as SiLU).

[0111] Input: The feature map of the previous layer or the original input of the model (e.g., a 3x640x640 image); Output: The feature map after convolution.

[0112] The C2f module is used to perform feature fusion in Backbone. This module contains the Conv module and the Concat layer for cross-layer connections.

[0113] Input: Features from the previous layer; Output: Feature maps after feature fusion. These feature maps will be retained and passed to subsequent upsampling or downsampling operations, or they may be directly entered into the Detect layer.

[0114] The SPPF module performs pooling operations at different scales and combines the feature maps before and after each pooling to achieve multi-scale fusion. This reduces computational cost, increases the receptive field, and preserves more spatial information. This module includes the Conv module and pooling layers.

[0115] Input: Features from the previous layer; Output: A low-dimensional feature map that incorporates features from multiple scales.

[0116] This feature map preserves spatial information while incorporating context at different scales, making it highly suitable for further use in object detection tasks for classification and localization.

[0117] Step S940: Modify the assumptions and dependencies in the network layer regarding the fixed input size to enable it to handle ( The input tensor is dynamically changing. Where:

[0118] The SPPF module's process is: one 1x1 convolution + three consecutive 5x5 MaxPool operations + concatenation + one 1x1 convolution. When the feature map is small, the feature map results of the three consecutive 5x5 MaxPool operations will be duplicated (for example, when the training data size is less than 416x416, the results of the last two MaxPool operations are the same; when the training data size is less than 160x160, the results of the last three operations are the same), increasing useless information and computation.

[0119] The improvement to the SPPF module is that the three-sequential 5x5 MaxPool is replaced with a cascaded 3x3, 5x5, and 7x7 MaxPool. This setup retains the multi-scale receptive field while providing a finer-grained receptive field even with dynamic input sizes.

[0120] The Detect layer is preset during initialization. Grid ( The size of this grid is preset based on the input size, thus fixing the input size during training and inference.

[0121] The improvement of the Detect layer lies in: (Grid coordinates) and The calculation of the (anchor mesh) is moved from initialization to forward propagation.

[0122] The Detect layer is the detection head of YOLOv5, which is responsible for converting multi-scale feature maps from Neck into bounding box predictions.

[0123] Input: 3 feature maps at different scales (corresponding to downsampling rates of 8, 16, and 32 respectively); Output (inference stage): bounding box coordinates (bbox coordinates) ), prediction box confidence and class; output (training phase): convolution of three feature maps at three different scales to output three feature maps.

[0124] The model predicts the offset of the target in a grid coordinate system, when the training data size is... The dimensions of P3.1 are [ ].

[0125] in, Number of prior bounding boxes (anchors), usually 3;

[0126] and : These represent the height and width of the grid coordinate system, where each index points to a grid (for example, [0,0] points to the first grid).

[0127] : Number of predicted categories +5 ( ); Based on the current grid, predict the x-offset of the target's center point; Based on the current grid, predict the y-offset of the target's center point; Width scaling factor; : Height scaling factor. Target score; : Score for each category.

[0128] Although the training phase yields intermediate results, decoding is subsequently performed in the loss function. Decoding involves converting the feature map into predicted bounding boxes, their confidence scores, and categories. The decoding formula is as follows:

[0129] (Multiplying by 2 and subtracting 0.5 in the formula is to expand the offset range to beyond ±0.5 grids);

[0130] ;

[0131] ;

[0132] ;

[0133] : Downsampling scale (8, 16, 32 respectively);

[0134] and :from The coordinates of the top-left corner of the current grid are retrieved from the data.

[0135] and From That is, the width and height of the prior box corresponding to the current grid cell.

[0136] Since each change in the size of the input image causes a change in the size of the feature map, and The size will also change, so it cannot be fixed during initialization. It must be... and The calculations are moved to the inference process and performed in real time.

[0137] Step S950, Dynamic Shape Model Deployment;

[0138] Step S9510: Using the PyTorch ONNX export interface, set the `dynamic_axes` parameter to dynamically specify the batch, height, and width dimensions of the input tensor (`dynamic_axes = {'input': {0:'batch', 2: 'input_h', 3:'input_w'}}`). Additionally, when exporting the model, only export the convolutions up to the Detect layer. Subsequent code dynamically calculates the values ​​based on the input size and feature layer level. and .

[0139] Step S9520 involves converting the exported dynamic Shape ONNX model to the target inference engine / hardware platform (such as TensorRT, RKNN, etc.). The converter needs to correctly support dynamic input dimensions. During the conversion process, it may be necessary to explicitly specify the dynamic dimension range in the configuration file.

[0140] Step S9530, C++ inference implementation: The preprocessing stage is the same as the training stage, only performing padding operations and strictly prohibiting any resize operations.

[0141] In step S960, for hardware platforms that do not support dynamic shape input, a solution can be adopted to export multiple fixed-shape models (such as exporting multiple fixed-size models of 320x320, 640x640, 720x720, etc.). During inference, multiple models are managed, and the closest fixed-size model that is greater than or equal to the input image size is selected for inference (preprocessing also only involves padding operations).

[0142] The method provided in this embodiment has at least the following beneficial effects:

[0143] This method sets multiple constraints through a data loader, which can solve the key adaptability problem of existing X-ray image package detection networks in X-ray security inspection scenarios. Specifically, by abandoning physical size distortion caused by fixed-size input, material property offset caused by conventional interpolation, and data augmentation methods that violate the physical imaging principles of X-rays, it can preserve the true physical structure and material property information contained in the original image data to the greatest extent. This can significantly improve the detection accuracy and generalization ability of small-sized and multi-shaped contraband in stacked and obscured scenarios, providing a reliable technical foundation for high-precision and interpretable automated security inspection.

[0144] like Figure 3 One embodiment of this application provides a training device for package detection based on raw resolution X-ray images, the device comprising:

[0145] The image acquisition module 1100 is used to respond to the training instruction of the X-ray image package detection model and acquire multiple first X-ray images to be trained; the first X-ray image is an original resolution X-ray image with a package area provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold.

[0146] The loader setting module 1200 is used to set the data loader; the data loader must have at least the following three constraints set:

[0147] The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and add an image data enhancement process obtained by stitching based on the original size of the first X-ray image.

[0148] The second constraint includes maintaining the physical information and geometric structure of the first X-ray image unchanged;

[0149] The third constraint includes: dynamically filling multiple first X-ray images;

[0150] The model training module 1300 is used to train the corresponding X-ray image package detection network by loading multiple first X-ray images according to the data loader, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items in packages in X-ray images is obtained.

[0151] It should be noted that the X-ray image package detection training device based on the original resolution provided in this embodiment and the X-ray image package detection training method based on the original resolution described above are based on the same inventive concept. Therefore, the content of the X-ray image package detection training device based on the original resolution described in this embodiment is also applicable to the content of the X-ray image package detection training method based on the original resolution described above, and will not be repeated here.

[0152] like Figure 4 One embodiment of this application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described training method for X-ray image package detection based on original resolution. The electronic device includes:

[0153] At least one battery;

[0154] At least one memory;

[0155] At least one processor;

[0156] At least one program;

[0157] The program is stored in memory, and the processor executes at least one program to implement the above-described method for training X-ray image package detection based on raw resolution, as described in this disclosure.

[0158] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0159] The electronic devices according to embodiments of this application will now be described in detail.

[0160] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0161] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to perform a training method for X-ray image package detection based on original resolution according to an embodiment of this disclosure.

[0162] The input / output interface 1800 is used to implement information input and output.

[0163] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0164] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0165] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0166] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described training method for X-ray image package detection based on original resolution.

[0167] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0169] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0170] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0172] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0173] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A training method for package detection in X-ray images based on original resolution, characterized in that, The method includes: In response to the training instruction of the X-ray image package detection model, multiple first X-ray images to be trained are acquired; the first X-ray images are original resolution X-ray images with package areas provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold. A data loader is configured; the data loader is configured with at least the following three constraints: The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and add an image data enhancement process obtained by stitching based on the original size of the first X-ray image. The second constraint includes maintaining the physical information and geometric structure in the first X-ray image unchanged; The third constraint includes: dynamically filling the plurality of first X-ray images; The X-ray image package detection network is trained by loading the multiple first X-ray images according to the data loader, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items inside packages in X-ray images is obtained.

2. The method for training X-ray image package detection based on original resolution according to claim 1, characterized in that, The acquisition of multiple first X-ray images to be trained includes: Acquire the raw resolution X-ray image of the packaged area provided by the security inspection device; If the size of the enclosed region in the original resolution X-ray image with the enclosed region is greater than a first threshold, the geometric center of the region is determined. At the geometric center of the area, the coordinates of the contraband in the package are marked, and the vertical column where there are no contraband is marked as a candidate column for slicing. Based on the underlying data of the original resolution X-ray image collected by the security inspection device, the information content of the candidate slice column is calculated, and the candidate slice column with the least information content and closest to the geometric center of the region is selected as the best slice column. Based on the optimal slice column, the original resolution X-ray image with the enclosed region is divided into two first X-ray images.

3. The method for training X-ray image package detection based on original resolution according to claim 2, characterized in that, The underlying data includes at least one of a high-energy map, a low-energy map, and an atomic number map; The amount of information includes at least one of information entropy and variance.

4. The method for training X-ray image package detection based on original resolution according to claim 2, characterized in that, The dynamic filling of the plurality of first X-ray images includes: Determine the maximum height and maximum width among the plurality of first X-ray images; The plurality of first X-ray images are filled to the maximum height and the maximum width; wherein the fill value is the background value of the first X-ray image; the plurality of first X-ray images are filled in the same way, and the fill method includes any one of center fill, lower right fill and upper left fill.

5. The method for training X-ray image package detection based on original resolution according to claim 4, characterized in that, The X-ray image package detection network is an improved YOLOv5 network; The improved YOLOv5 network includes an improved Backbone section, Neck section, and Head section; the network structure of the Neck and Head sections in the improved YOLOv5 network is consistent with that of the YOLOv5 network. The first module in the improved Backbone section is the improved SPPF module. The network structure of the other modules is the same as that of the YOLOv5 network. The improved SPPF module includes: a 1x1 convolution, 3x3, 5x5, and 7x7 max pooling average concatenation, and a 1x1 convolution.

6. The method for training X-ray image package detection based on original resolution according to claim 5, characterized in that, The calculation of grid coordinates and anchor grids in the Detect layer of the Head section is moved forward to the forward propagation of the improved YOLOv5 network.

7. The method for training X-ray image package detection based on original resolution according to claim 1, characterized in that, After obtaining the X-ray image package detection model for detecting items inside a package in an X-ray image, the method further includes: The X-ray image package detection model is deployed using a dynamic Shape model.

8. A training device for package detection based on raw resolution X-ray images, characterized in that, The device includes: The image acquisition module is used to respond to the training instruction of the X-ray image package detection model and acquire multiple first X-ray images to be trained; the first X-ray image is an original resolution X-ray image with a package area provided by the security inspection device, and the size of the package area in the first X-ray image is smaller than a first threshold. The loader setting module is used to configure the data loader; the data loader is configured with at least the following three constraints: The first constraint includes: setting an improved mosaic enhancement process; the improved mosaic enhancement process is to remove the operation of scaling the first X-ray image to a fixed pixel size using an interpolation algorithm in the mosaic enhancement process, and add an image data enhancement process obtained by stitching based on the original size of the first X-ray image. The second constraint includes maintaining the physical information and geometric structure in the first X-ray image unchanged; The third constraint includes: dynamically filling the plurality of first X-ray images; The model training module is used to train the corresponding X-ray image package detection network by loading the multiple first X-ray images according to the data loader, so that after the X-ray image package detection network is trained, an X-ray image package detection model for detecting items inside packages in X-ray images is obtained.

9. An electronic device, characterized in that, It includes at least one controller and a memory for communicatively connecting with the controller; the memory stores instructions executable by the at least one controller, the instructions being executed by the at least one controller to cause the at least one controller to perform a training method for X-ray image package detection based on raw resolution as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a training method for detecting X-ray images based on raw resolution as described in any one of claims 1 to 7.

Images