Model training method, contraband detection method, electronic equipment and storage medium

Abstract

The invention discloses a model training method, a contraband detection method, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: inputting a training sample and a training label into a to-be-trained target detection model; each training sample is a target ray image obtained by superposing at least one perspective shielding block on a wrapped initial ray image, and performing the following operations through the target detection model: performing feature extraction processing and feature fusion on each training sample to obtain a fusion feature map of multiple scales; carrying out material feature extraction on the fusion feature map of the multiple scales to obtain a material feature map; performing shielding degree feature extraction on the material feature graph to obtain a space shielding probability graph; obtaining a shielding weighted feature map according to the fusion feature map and the space shielding probability map; and analyzing the shielding weighted feature map to obtain and determine a total training loss value based on a prediction result and a training label, so that a high-precision and high-robustness target detection model for a complex scene can be trained.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a model training method, a contraband detection method, an electronic device, and a storage medium. Background Technology

[0002] In the field of security inspection, packages are typically scanned using X-rays or other optical signals to identify target objects (mainly contraband) within. However, the stacking of items within a package can partially or severely obscure contraband, making it difficult for the model to identify contraband in the X-ray image. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in related technologies. To this end, this application proposes a model training method, a contraband detection method, an electronic device, and a storage medium, which can achieve high-precision and robust target detection and recognition in security inspection scenarios with complex obstructions and mixed materials.

[0004] Firstly, this application provides a model training method, which includes: Input the training samples and training labels into the target detection model to be trained; the training samples are target ray images obtained by overlaying at least one perspective occlusion block on the initial ray image of the package, and the perspective occlusion block is an occlusion block with material category generated by simulating ray imaging; the training labels include the package's contraband category label, material category label, and bounding box label of the contraband location; perform the following operations through the target detection model: Feature extraction and feature fusion processing are performed on each training sample to obtain fused feature maps at multiple scales; Material feature extraction is performed on the fused feature map at multiple scales to obtain a material feature map; The material feature map is processed by occlusion degree feature extraction to obtain a spatial occlusion probability map; the spatial occlusion probability map includes the relative occlusion severity of each pixel after being occluded by a perspective occlusion block; Based on the fused feature map and the spatial occlusion probability map, an occlusion-weighted feature map is obtained; The occlusion-weighted feature map is analyzed to obtain and determine the total training loss value based on the prediction results and training labels. The target detection model is then trained by backpropagation based on the total training loss value until the training termination condition is met, resulting in a trained target detection model. The prediction results include at least a portion of the following: the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence score.

[0005] According to the model training method of this application, the target ray image is obtained as a training sample by superimposing at least one perspective occlusion block on the initial ray image of the package. This can achieve low-cost, realistic and effective acquisition of occlusion training samples that fit the actual application cost. The three tasks of material perception, occlusion perception and target detection are deeply integrated into a single target detection model training process for joint optimization. This can achieve high-precision and high-robust target detection and recognition in security inspection scenarios with complex occlusion and mixed materials.

[0006] According to one embodiment of this application, before inputting training samples and training labels into the target detection model to be trained, the method further includes: For each initial training sample, obtain and generate a transparent occlusion block corresponding to each polygon pattern based on at least one randomly selected polygon pattern, the transparency set of each polygon pattern, and the color selected from the corresponding color system of each material category; Each transparent occlusion block is superimposed on the initial ray image to obtain the target ray image as a training sample.

[0007] According to one embodiment of this application, the object detection model includes a feature extraction module; it performs feature extraction and feature fusion processing on each training sample to obtain a fused feature map at multiple scales, including: The backbone network of the feature extraction module performs a series of convolution and downsampling operations on the training samples to obtain multi-scale feature maps of the training samples. Multi-scale feature maps are fused using a feature pyramid network and / or path aggregation network in the feature extraction module to obtain fused feature maps of multiple scales.

[0008] According to one embodiment of this application, the target detection model includes a material perception module; after performing material feature extraction processing on the fused feature map at multiple scales to obtain a material feature map, the method further includes: The material feature map is reduced in dimensionality by using a convolution kernel of at least one target size from the material perception module to reduce the number of channels, thus obtaining a dimensionality-reduced feature map. The reduced feature map is processed by nonlinear transformation using the activation function of the material perception module to obtain the predicted probability of each pixel in each material category.

[0009] According to one embodiment of this application, the target detection model includes an occlusion perception module; occlusion degree feature extraction processing is performed on the material feature map to obtain a spatial occlusion probability map, including: The occlusion degree feature map is obtained by extracting the occlusion degree feature from the material feature map through at least one convolutional layer of the occlusion perception module. The occlusion degree feature map is processed by nonlinear transformation using the activation function of the occlusion perception module to obtain the spatial occlusion probability map.

[0010] According to one embodiment of this application, an occlusion-weighted feature map is obtained based on the fused feature map and the spatial occlusion probability map, including: The occlusion perception module multiplies the fused feature map and the spatial occlusion probability map point by point to obtain the occlusion weighted feature map.

[0011] According to one embodiment of this application, the object detection model includes a detection head; parsing the occlusion-weighted feature map to obtain and determine the total training loss value based on the prediction result and training labels, including: The prediction result is obtained by analyzing the occlusion-weighted feature map using the detection head; The first loss value of the first classification loss function is determined by predicting the probability and material category label of each pixel based on the detection head. The second loss value of the second classification loss function is determined by using the detection head to predict the probability of prohibited items and the prohibited item category label. Based on the predicted bounding box and its label, the detection head determines the third loss value of the bounding box localization loss function and the fourth loss value of the distribution focus loss function. The total training loss value is determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0012] Secondly, this application provides a method for detecting prohibited items, the method comprising: The ray image of the package is input into a trained target detection model to obtain the detection result for the package; the detection result is either the first detection result or the second detection result. The first test result is used to indicate that there are no prohibited items in the package; The second test result is used to indicate the presence of prohibited items in the package, the type of prohibited items, and their location; The object detection model was trained using the aforementioned model training method.

[0013] Thirdly, this application provides a model training apparatus, which includes: The input module is used to input training samples and training labels into the target detection model to be trained. The training samples are target ray images obtained by superimposing at least one perspective occlusion block on the initial ray image of the package. The perspective occlusion block is an occlusion block with material category generated by simulating ray imaging. The training labels include the package's contraband category label, material category label, and bounding box label where the contraband is located. The processing module is used to perform the following operations using the object detection model: Feature extraction and feature fusion processing are performed on each training sample to obtain fused feature maps at multiple scales; Material feature extraction is performed on the fused feature map at multiple scales to obtain a material feature map; The material feature map is processed by occlusion degree feature extraction to obtain a spatial occlusion probability map; the spatial occlusion probability map includes the relative occlusion severity of each pixel after being occluded by a perspective occlusion block; Based on the fused feature map and the spatial occlusion probability map, an occlusion-weighted feature map is obtained; The occlusion-weighted feature map is analyzed to obtain and determine the total training loss value based on the prediction results and training labels. The target detection model is then trained by backpropagation based on the total training loss value until the training termination condition is met, resulting in a trained target detection model. The prediction results include at least a portion of the following: the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence score.

[0014] The model training device provided in this application obtains the target ray image as a training sample by superimposing at least one perspective occlusion block on the initial ray image of the package. This enables low-cost, realistic, and effective acquisition of occlusion training samples that are in line with the actual application cost. It deeply integrates the three tasks of material perception, occlusion perception, and target detection into a single target detection model training process for joint optimization. This allows for high-precision and robust target detection and recognition in security inspection scenarios with complex occlusion and mixed materials.

[0015] Fourthly, this application provides a contraband detection device, which includes: The detection module is used to input the ray image of the package into the trained target detection model to obtain the detection result for the package; the detection result is either a first detection result or a second detection result. The first test result is used to indicate that there are no prohibited items in the package; The second test result is used to indicate the presence of prohibited items in the package, the type of prohibited items, and their location; The object detection model was trained using the aforementioned model training method.

[0016] Fifthly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the model training method provided in the first aspect or the contraband detection method provided in the second aspect.

[0017] Sixthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model training method provided in the first aspect or the contraband detection method provided in the second aspect.

[0018] In a seventh aspect, this application provides a chip, the chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the model training method provided in the first aspect or the contraband detection method provided in the second aspect.

[0019] Eighthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the model training method provided in the first aspect or the contraband detection method provided in the second aspect.

[0020] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: By superimposing at least one perspective occlusion block on the initial ray image of the package to obtain the target ray image as a training sample, it is possible to obtain occlusion training samples that are in line with the actual application cost in a low-cost, realistic and effective manner. The three tasks of material perception, occlusion perception and target detection are deeply integrated into a single target detection model training process for joint optimization. This enables high-precision and robust target detection and recognition in security inspection scenarios with complex occlusion and mixed materials.

[0021] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the model training method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the architecture of the target detection model provided in the embodiments of this application; Figure 3 This is one of the flowcharts illustrating the prohibited substance detection method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the model training device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the contraband detection device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] In the field of X-ray security inspection, deep learning-based target detection algorithms have become a core technology. However, related technologies face serious challenges: 1. Severe occlusion problem: Items inside the package are stacked on top of each other, causing the target object to be partially or severely occluded, making it difficult for the model to recognize.

[0026] 2. Material confusion problem: In X-ray imaging, different materials (organic, mixed, and inorganic) have different penetrability and imaging characteristics, but the models of related technologies have failed to effectively utilize this prior physical knowledge, leading to misjudgment of objects with similar shapes but different materials.

[0027] 3. Data annotation bottleneck: Obtaining a large amount of accurate labeled data covering various complex occlusion situations is costly, and the occlusions generated by traditional data augmentation methods (such as random cropping, CutOut, CutMix) do not conform to the physical characteristics of X-rays, resulting in limited effectiveness.

[0028] The related technologies failed to fundamentally simulate and utilize the physical characteristics of X-ray imaging, resulting in poor detection accuracy and robustness in real complex package scenarios.

[0029] The model training method, contraband detection method, electronic device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0030] like Figure 1 As shown, the model training method includes steps 110, 120, 121, 122, 123, 124 and 125.

[0031] Step 110: Input the training samples and training labels into the target detection model to be trained; the training samples are target ray images obtained by superimposing at least one perspective occlusion block on the initial ray image of the package, and the perspective occlusion block is an occlusion block with material category generated by simulating ray imaging; the training labels include the package's contraband category label, material category label, and bounding box label where the contraband is located.

[0032] This application embodiment has a large number of training samples. Each training sample is a target ray image obtained by superimposing at least one perspective occlusion block on the initial ray image of the package. The initial ray image and the target ray image are generally X-ray images. Superimposing perspective occlusion blocks on the initial ray image is used to simulate the X-ray image obtained due to the stacking of packages in real-world scenes.

[0033] One or more perspective occlusion blocks are superimposed on the initial ray image to create more and more realistic occlusion samples.

[0034] The training samples are specifically divided into positive training samples and negative training samples.

[0035] The packages corresponding to positive training samples contain contraband, while the packages corresponding to negative training samples do not contain contraband.

[0036] The target detection model to be trained in this application embodiment is used for the identification of prohibited items, aiming to identify whether there are prohibited items in the package, the type of prohibited items, and the location of the prohibited items. Therefore, the training labels need to include the prohibited item category label, the material category label, and the bounding box label of the location of the prohibited items in the package.

[0037] Among them, the prohibited item category label is either no prohibited items (label of negative training samples) or the true category of prohibited items (part of the label of positive training samples), such as firearms, ammunition, strong acid, etc.

[0038] The prohibited material category label (part of the label on the positive training sample) can be any of the following: organic, inorganic, or mixtures.

[0039] The bounding box label (part of the label of the positive training sample) of the location of the contraband represents the true location of the contraband. It can be represented by pixel coordinate pairs, such as the bottom left and top right pixel coordinates, or the top left and bottom right pixel coordinates.

[0040] Step 120: Perform the following operations using the object detection model: Step 121: Perform feature extraction and feature fusion processing on each training sample to obtain fused feature maps at multiple scales.

[0041] Step 121 is mainly performed by the feature extraction module of the object detection model. The feature extraction module is the core backbone of the object detection system. Its core function is to extract feature information of different scales and different levels of abstraction from the input X-ray training samples, such as color features, texture features, brightness features, etc., and then perform feature fusion to obtain a fused feature map, which provides rich feature representation for subsequent occlusion perception, material classification and object detection.

[0042] Step 122: Extract material features from the fused feature maps of multiple scales to obtain a material feature map, which includes the material features of each pixel.

[0043] Step 122 is mainly performed by the material perception module of the target detection model. The material perception module is one of the core intelligent components of the entire system. It receives the high-level fusion feature map generated by the feature extraction module and is specifically responsible for analyzing the physical material properties corresponding to different regions in the image.

[0044] Its core function is to act as the system's "physical property analyzer." It performs in-depth analysis of the fused feature map and outputs a classification judgment of the material category. The material categories are specifically divided into three categories: organic materials (such as plastics, fabrics, and food), inorganic materials (mainly referring to metal products), and mixtures (such as composite materials and glass). This judgment provides key physical dimension information for the recognition process and can output a material feature map.

[0045] In addition, the material perception module can also process the material feature map and output the predicted probability of the material category of each pixel. It can be represented by a three-dimensional vector [P_organic, P_inorganic, P_mixture], which represents the probability that the point belongs to organic matter, inorganic matter and mixture respectively. The sum of the three probability values ​​is 1.

[0046] Step 123: Perform occlusion degree feature extraction processing on the material feature map to obtain a spatial occlusion probability map; the spatial occlusion probability map includes the relative occlusion severity of each pixel after being occluded by the perspective occlusion block.

[0047] Steps 1, 2, and 3 are primarily executed by the occlusion perception module of the target detection model. In X-ray imaging, different materials inherently differ in their occlusion capabilities. Inorganic materials strongly absorb X-rays and are almost opaque, causing severe occlusion of objects behind them. Mixtures are translucent and cause moderate occlusion. Organic materials weakly absorb X-rays and are highly transparent, typically causing only mild occlusion. Therefore, a physical prior bridge can be established using material information, allowing the model to learn to infer the degree of occlusion. Essentially, it learns that when a certain material is present, its occlusion effect is most likely to make the detection task difficult.

[0048] The occlusion perception module can extract the occlusion degree feature from the material feature map to obtain a spatial occlusion probability map. This spatial occlusion probability map includes the relative occlusion severity of each pixel after being occluded by the perspective occlusion block, and its value range is [0, 1].

[0049] Step 124: Obtain the occlusion-weighted feature map based on the fused feature map and the spatial occlusion probability map.

[0050] Step 124 is also executed by the occlusion perception module. The occlusion perception module performs point-by-point multiplication of the fused feature map and the spatial occlusion probability map to achieve occlusion perception modulation of the original fused feature map, thereby highlighting the feature response of the occluded area and obtaining the occlusion weighted feature map.

[0051] Step 125: Analyze the occlusion weighted feature map to obtain and determine the total training loss value based on the prediction results and training labels. Perform backpropagation training on the target detection model based on the total training loss value until the training termination condition is met to obtain the trained target detection model. The prediction results include at least a portion of the following: the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence score.

[0052] Step 125 is executed by the detection head of the object detection model. The detection head is similar to the detection head structure of common object detection (e.g., YOLOv8), and is responsible for the final parsing of the occlusion weighted feature map and outputting the prediction result.

[0053] The prediction results include at least a portion of the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence level.

[0054] The predicted probability of a prohibited item category represents the probability that the package is located under each prohibited item category; the predicted bounding box of the prohibited item location represents the predicted location; the confidence level represents the accuracy of the predicted probability of the prohibited item category and / or the accuracy of the predicted bounding box detection.

[0055] The detection head can determine the total loss value of the target detection model based on the prediction results and training labels. The total loss value mainly consists of four sub-loss values: the first loss value between the predicted probability of the material category of each pixel and the material category label; the second loss value between the predicted probability of the prohibited item category and the prohibited item category label; the third loss value for determining the bounding box localization loss function; and the fourth loss value for the distribution focus loss function. The total loss value can be obtained by summing the first, second, third, and fourth loss values.

[0056] The target detection model can be trained by backpropagation based on the total loss value until the training termination condition is met. The training termination condition is adjusted to the total loss value being less than the target loss value (preset value) or reaching the target number of training iterations (preset value). The target detection model obtained at the end of training is the trained target detection model.

[0057] This application embodiment obtains a target ray image as a training sample by superimposing at least one perspective occlusion block on the initial ray image of the package. This can achieve low-cost, realistic and effective acquisition of occlusion training samples that fit the actual application cost. It deeply integrates the three tasks of material perception, occlusion perception and target detection into a single target detection model training process for joint optimization. This can achieve high-precision and high-robust target detection and recognition in security inspection scenarios with complex occlusion and mixed materials.

[0058] In some embodiments, the method further includes, before inputting training samples and training labels into the object detection model to be trained: For each initial training sample, obtain and generate a transparent occlusion block corresponding to each polygon pattern based on at least one randomly selected polygon pattern, the transparency set of each polygon pattern, and the color selected from the corresponding color system of each material category; Each transparent occlusion block is superimposed on the initial ray image to obtain the target ray image as a training sample.

[0059] In this embodiment, for each initial X-ray image, one or more occlusion blocks are randomly generated. Each occlusion block P is defined by the following key attributes to simulate X-ray fluoroscopic imaging: Shape and position: The occlusion block is an arbitrary polygon, and its center point (x, y) and the number of vertices are randomly generated within the effective range of the image.

[0060] Transparency: The occlusion block is not completely opaque; its transparency α is randomly sampled within a uniformly distributed range, for example, α ~ Uniform(0.3, 0.9). This simulates the property that X-rays can partially penetrate objects.

[0061] Color scheme for each material category: The color C of the occlusion block is not grayscale or a random RGB value, but is randomly selected from a set of preset colors corresponding to X-ray materials. This set of colors includes: Orange tones (simulating organic materials such as plastics and fabrics), for example, randomly sampled from the range [R:200-255, G:100-150, B:0-50] in the RGB color space.

[0062] Green series (simulating mixtures, such as glass, composite materials), for example, randomly sampled from the range [R:50-150, G:150-255, B:50-150].

[0063] Blue tones (simulating metallic inorganic objects such as knives, guns, and thermos cups), for example, randomly sampled from the range [R:0-100, G:100-200, B:200-255].

[0064] Based on the selection of the above information, one or more transparent occlusion blocks can be generated. Then, each transparent occlusion block is superimposed on the initial ray image to obtain the target ray image as a training sample, thereby creating more and more realistic occlusion samples at low cost.

[0065] In some embodiments, the target detection model includes a feature extraction module; it performs feature extraction and feature fusion processing on each training sample to obtain a fused feature map at multiple scales, including: The backbone network of the feature extraction module performs a series of convolution and downsampling operations on the training samples to obtain multi-scale feature maps of the training samples. Multi-scale feature maps are fused using a feature pyramid network and / or path aggregation network in the feature extraction module to obtain fused feature maps of multiple scales.

[0066] The feature extraction module typically consists of a modern convolutional neural network (such as CSPDarknet) and includes a Feature Pyramid Network (FPN) or Path Aggregation Network (PANet) as the "neck". Its processing flow can be broken down into two stages: Phase 1: Multi-scale Feature Extraction (Backbone). The backbone network progressively transforms the input image into feature maps of different scales through a series of convolutions and downsampling operations.

[0067] As the number of network layers increases, the spatial dimensions (height and width) of the feature maps gradually decrease, but the number of channels gradually increases, thereby capturing more abstract semantic information. For example, an input 640x640 image, after passing through the backbone, will successively generate feature maps of scales such as 80x80, 40x40, and 20x20.

[0068] Phase Two: Multi-Scale Feature Fusion (Neck - FPN / PANet). In order to simultaneously utilize low-level detailed information (which is beneficial for locating small targets) and high-level semantic information (which is beneficial for recognizing large targets), the neck network fuses the above multi-scale features through top-down and bottom-up paths to obtain a fused feature map.

[0069] The "top-down" approach refers to upsampling deep, low-resolution strong semantic features and fusing them with shallow, high-resolution features.

[0070] Bottom-up refers to downsampling shallow features and fusing them with deep features to enhance localization capabilities. After the above processing, the module finally outputs three (the aforementioned 80x80, 40x40, 20x20, etc.) or four (160x160, 80x80, 40x40, 20x20, etc.) different scales of fused feature maps. These fused feature maps constitute the shared basis for all subsequent tasks.

[0071] In some embodiments, the target detection model includes a material-aware module; after performing material feature extraction processing on the fused feature map at multiple scales to obtain a material feature map, the method further includes: The material feature map is reduced in dimensionality by using a convolution kernel of at least one target size from the material perception module to reduce the number of channels, thus obtaining a dimensionality-reduced feature map. The reduced feature map is processed by nonlinear transformation using the activation function of the material perception module to obtain the predicted probability of each pixel in each material category.

[0072] The material perception module performs in-depth analysis of the feature map and outputs a classification judgment of the target material.

[0073] The material-aware module is a lightweight network structure that first extracts material features from the input fused feature map through one or more convolutional layers to obtain the material feature map.

[0074] Then, instead of using a multilayer perceptron to transform the number of channels in the entire feature map, the material perception module uses a target convolution kernel (1x1 convolution) to adjust the number of channels at each position in the feature map (e.g., 1024 channels) to a dimension suitable for material classification (3 channels, corresponding to organic matter, inorganic matter and mixtures respectively). This dimensionality reduction process is performed to better mine material features.

[0075] Finally, a Softmax activation function is applied to perform a non-linear transformation, converting this 3D vector into a probability distribution (originally, each dimension of the vector was not a probability), so that the sum of the three probability values ​​is 1. For example, for each pixel on the feature map, a three-dimensional vector [P_organic, P_inorganic, P_mixture] is output, representing the probability that the point belongs to organic matter, inorganic matter, and a mixture, respectively.

[0076] In some embodiments, the target detection model includes an occlusion perception module; occlusion degree feature extraction processing is performed on the material feature map to obtain a spatial occlusion probability map, including: The occlusion degree feature map is obtained by extracting the occlusion degree feature from the material feature map through at least one convolutional layer of the occlusion perception module. The occlusion degree feature map is processed by nonlinear transformation using the activation function of the occlusion perception module to obtain the spatial occlusion probability map.

[0077] In X-ray imaging, different materials inherently differ in their occlusion capabilities. Inorganic materials strongly absorb X-rays and are almost opaque, causing severe occlusion of objects behind them. Mixtures, with their semi-transparent properties, cause moderate occlusion. Organic materials weakly absorb X-rays and are highly transparent, typically causing only mild occlusion. Therefore, a physical prior can be established using material information, allowing the model to learn and infer the degree of occlusion. Essentially, it learns that when a certain material is present, its occlusion effect is most likely to make the detection task difficult.

[0078] The occlusion perception module is also a lightweight network structure. It first takes the material feature map output by the material perception module as input through one or more convolutional layers and extracts material features to obtain the material feature map.

[0079] Then, a Sigmoid activation function is used to perform a non-linear transformation on the material feature map, outputting a spatial occlusion probability map with a value range of [0, 1], which is used to characterize the relative occlusion severity of each region of the image.

[0080] In some embodiments, an occlusion-weighted feature map is obtained based on the fused feature map and the spatial occlusion probability map, including: The occlusion perception module multiplies the fused feature map and the spatial occlusion probability map point by point to obtain the occlusion weighted feature map.

[0081] Specifically, the pointwise multiplication between the two can be represented by the following formula (1): (1), Where F1 represents the fused feature map of the feature extraction module, F2 represents the spatial occlusion probability map of the occlusion perception module, and F represents the occlusion-weighted feature map after dot product. This represents the Sigmoid activation function. This represents the dot product of matrices.

[0082] By multiplying the fused feature map and the spatial occlusion probability map point by point, the occlusion perception modulation of the original features is achieved, thereby highlighting the feature response of the occluded area.

[0083] In some embodiments, the object detection model includes a detection head; parsing the occlusion-weighted feature map to obtain and determine the total training loss value based on the prediction results and training labels, including: The prediction result is obtained by analyzing the occlusion-weighted feature map using the detection head; The first loss value of the first classification loss function is determined by predicting the probability and material category label of each pixel based on the detection head. The second loss value of the second classification loss function is determined by using the detection head to predict the probability of prohibited items and the prohibited item category label. Based on the predicted bounding box and its label, the detection head determines the third loss value of the bounding box localization loss function and the fourth loss value of the distribution focus loss function. The total training loss value is determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0084] During the training phase, the output of the material-aware module also includes the predicted probability of each pixel in each material category. This is compared with the material category labels in the training labels to calculate and determine the first loss value L of the first classification loss function. m Formula (2) is as follows: (2), Wherein, formula (2) is the first loss value L of the first classification loss function (specifically, the cross-entropy classification loss function). m The calculation is as follows: N represents the number of anchor points of the positive training samples for which the loss needs to be calculated, and C represents the number of material categories, where C=3 (organic, inorganic, and mixture). This represents the one-hot encoding of the material category label. For sample i, if its true category is c, then... Otherwise, it is 0. This represents the probability that sample i belongs to category c as predicted by the material-aware module (i.e., the output after the Softmax activation function).

[0085] The detection head outputs two branches in parallel: a classification branch and a regression branch. The classification branch outputs the class probability for each anchor point; the model independently determines whether it belongs to that class, rather than outputting a multi-class probability distribution. This helps handle potential label ambiguity and overlap issues in the data, corresponding to the second loss value L of the second classification loss function. bce The following (3) calculation is performed: (3), in, This indicates the number of positive training samples. The one-hot code indicating the category of prohibited items on the label. This represents the predicted probability for each category of prohibited items, where nc represents the number of prohibited item categories.

[0086] The regression branch outputs the bounding box parameters for each anchor point, transforming the traditional bounding box coordinate regression problem into a discrete probability distribution classification problem. That is, each coordinate is modeled as a classification problem with reg_max = 16 bins. For example, for a given coordinate point, the model predicts its probability distribution across 16 bins in the interval [0, reg_max-1]. The bounding box regression loss value L... bbox It can be determined based on the following formula (4): (4), Where α and β are the weights of two loss functions, L IoU L represents the third loss value of the localization loss function for the bounding box. DFL This represents the fourth loss value of the focal loss function for bounding box distribution.

[0087] Among them, L IoU It can be determined by the following formula (5): (5), in, Let b represent the predicted bounding box coordinates of the i-th positive training sample. i Let CIoU(·) represent the bounding box label (true bounding box coordinates) of the i-th positive training sample, and let CIoU(·) represent the Complete-IoU metric.

[0088] In addition, L DFL It can be determined by the following formula (6): (6), This represents the discrete distribution of the j-th edge (left, top, bottom, right) of the bounding box predicted by the model. represents the normalized distance of the j-th edge of the true bounding box. DFL(·) represents the distribution focus loss function, which can be determined by the following formula (7): (7), in, It is the integer part of the true coordinate value s after rounding down, which indicates the index of the left bin of s in the discrete distribution.

[0089] Finally, the total training loss value of the entire training process can be represented by the following formula (8): (8), in, The weights for each loss function.

[0090] See Figure 2This application provides an architectural diagram of an object detection model, which includes a feature extraction module, a material perception module, an occlusion perception module, and a detection head.

[0091] The feature extraction module takes training samples and training labels as input and outputs a fused feature map.

[0092] The input to the material perception module is the fused feature map, and the output is the material feature map and the predicted probability of each pixel in the categories of organic matter, inorganic matter, and mixture obtained after processing by a 1x1 convolution kernel.

[0093] The input to the occlusion perception module is the fused feature map output by the feature extraction module and the material feature map output by the material perception module, and the output is a spatial occlusion probability map.

[0094] The input to the detection head is a dot product of the fused feature map output by the feature extraction module and the spatial occlusion probability map. The resulting occlusion-weighted feature map is then used as the output for prediction. The prediction results include the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence level.

[0095] The training labels, the predicted probabilities of each pixel in the categories of organic matter, inorganic matter, and mixture, and the prediction results can be used to calculate the total training loss value of the loss function. For details, please refer to the aforementioned embodiments, which will not be repeated here.

[0096] This application provides a method for detecting prohibited items; see [link / reference]. Figure 3 This application provides a flowchart illustrating a method for detecting contraband, including: Step S310: Obtain a ray image of the package; Step S320: Input the ray image of the package into the trained target detection model to obtain the detection result for the package; the detection result is either the first detection result or the second detection result. The first test result is used to indicate that there are no prohibited items in the package; The second test result is used to indicate the presence of prohibited items in the package, the type of prohibited items, and their location; The object detection model was trained using the aforementioned model training method.

[0097] In this embodiment, after obtaining the trained target detection model, the target detection model processes the ray image of the package through various modules to output the detection result. Since the training process superimposes at least one perspective occlusion block on the initial ray image of the package to obtain the target ray image as a training sample, it can achieve low-cost, realistic, and effective acquisition of occlusion training samples that fit the actual application cost. The three tasks of material perception, occlusion perception, and target detection are deeply integrated into a single target detection model training process for joint optimization. After the target detection model is put into operation, it can achieve high-precision and robust target detection and recognition in security inspection scenarios with complex occlusion and mixed materials.

[0098] The model training method provided in this application can be executed by a model training device. This application uses an example of a model training device executing the model training method to illustrate the model training device provided in this application.

[0099] This application also provides a model training device.

[0100] like Figure 4 As shown, the model training device includes a model training module 410 and a processing module 420.

[0101] The input module 410 is used to input training samples and training labels into the target detection model to be trained; the training samples are target ray images obtained by superimposing at least one perspective occlusion block on the initial ray image of the package, and the perspective occlusion block is an occlusion block with material category generated by simulating ray imaging; the training labels include the package's contraband category label, material category label, and bounding box label where the contraband is located. Processing module 420 is used to perform the following operations using the object detection model: Feature extraction and feature fusion processing are performed on each training sample to obtain fused feature maps at multiple scales; Material feature extraction is performed on the fused feature map at multiple scales to obtain a material feature map; The material feature map is processed by occlusion degree feature extraction to obtain a spatial occlusion probability map; the spatial occlusion probability map includes the relative occlusion severity of each pixel after being occluded by a perspective occlusion block; Based on the fused feature map and the spatial occlusion probability map, an occlusion-weighted feature map is obtained; The occlusion-weighted feature map is analyzed to obtain and determine the total training loss value based on the prediction results and training labels. The target detection model is then trained by backpropagation based on the total training loss value until the training termination condition is met, resulting in a trained target detection model. The prediction results include at least a portion of the following: the predicted probability of the contraband category, the predicted bounding box of the contraband location, and the confidence score.

[0102] According to the model training device provided in the embodiments of this application, by superimposing at least one perspective occlusion block on the initial ray image of the package to obtain the target ray image as a training sample, it is possible to obtain occlusion training samples that are in line with the actual application cost in a low-cost, realistic and effective manner. The three tasks of material perception, occlusion perception and target detection are deeply integrated into a target detection model training process for joint optimization. In the security inspection scenario with complex occlusion and mixed materials, high-precision and high-robust target detection and recognition can be achieved.

[0103] In some embodiments, the model training apparatus further includes: The preprocessing module is used for: For each initial training sample, obtain and generate a transparent occlusion block corresponding to each polygon pattern based on at least one randomly selected polygon pattern, the transparency set of each polygon pattern, and the color selected from the corresponding color system of each material category; Each transparent occlusion block is superimposed on the initial ray image to obtain the target ray image as a training sample.

[0104] In some embodiments, the target detection model includes a feature extraction module; the processing module 420 is used for: The backbone network of the feature extraction module performs a series of convolution and downsampling operations on the training samples to obtain multi-scale feature maps of the training samples. Multi-scale feature maps are fused using a feature pyramid network and / or path aggregation network in the feature extraction module to obtain fused feature maps of multiple scales.

[0105] In some embodiments, the target detection model includes a material perception module; the processing module 420 is used for: The material feature map is reduced in dimensionality by using a convolution kernel of at least one target size from the material perception module to reduce the number of channels, thus obtaining a dimensionality-reduced feature map. The reduced feature map is processed by nonlinear transformation using the activation function of the material perception module to obtain the predicted probability of each pixel in each material category.

[0106] In some embodiments, the target detection model includes an occlusion perception module; the processing module 420 is used to: perform occlusion degree feature extraction processing on the material feature map through at least one convolutional layer of the occlusion perception module to obtain an occlusion degree feature map; The occlusion degree feature map is processed by nonlinear transformation using the activation function of the occlusion perception module to obtain the spatial occlusion probability map.

[0107] In some embodiments, the processing module 420 is configured to: multiply the fused feature map and the spatial occlusion probability map point by point through the occlusion perception module to obtain an occlusion weighted feature map.

[0108] In some embodiments, the target detection model includes a detection head; the processing module 420 is used for: The prediction result is obtained by analyzing the occlusion-weighted feature map using the detection head; The first loss value of the first classification loss function is determined by predicting the probability and material category label of each pixel based on the detection head. The second loss value of the second classification loss function is determined by using the detection head to predict the probability of prohibited items and the prohibited item category label. Based on the predicted bounding box and its label, the detection head determines the third loss value of the bounding box localization loss function and the fourth loss value of the distribution focus loss function. The total training loss value is determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0109] The prohibited substance detection method provided in this application can be executed by a prohibited substance detection device. This application uses a model training device to execute the prohibited substance detection method as an example to illustrate the prohibited substance detection device provided in this application.

[0110] This application also provides a prohibited item detection device.

[0111] like Figure 5 As shown, the contraband detection includes an input module 510 and a detection module 520.

[0112] Input module 510 is used to input a ray image of the package; The detection module 520 is used to input the ray image of the package into the trained target detection model to obtain the detection result for the package; the detection result is either a first detection result or a second detection result. The first test result is used to indicate that there are no prohibited items in the package; The second test result is used to indicate the presence of prohibited items in the package, the type of prohibited items, and their location; The object detection model was trained using the aforementioned model training method.

[0113] The model training device and contraband detection device in this application embodiment can be electronic devices or components within electronic devices, such as integrated circuits or chips. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific devices.

[0114] The model training device and the contraband detection device in this application embodiment can be devices with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0115] The model training device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiments of this application, and the contraband detection device provided in the embodiments of this application, can achieve... Figure 3 To avoid repetition, the various processes implemented in the method implementation examples will not be described again here.

[0116] In some embodiments, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, it implements the various processes of the above-described model training method and / or model training method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0117] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0118] The memory 602 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0119] The memory 602 is used to store computer programs that execute the embodiments of this application, and the execution is controlled by the processor 601. The processor 601 is used to execute the computer programs stored in the memory 602 to implement the steps shown in the foregoing method embodiments.

[0120] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0121] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described model training method and / or model training method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0122] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0123] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described model training method and / or model training method.

[0124] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0125] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described model training method and / or model training method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0126] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0129] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0130] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0131] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A model training method, characterized in that, The method comprises the following steps: inputting training samples and training labels into a target detection model to be trained; the training sample is a target radiograph obtained by superimposing at least one perspective occlusion block on an initial radiograph of a parcel, the perspective occlusion block is an occlusion block with a material category generated by simulation radiography; the training label comprises a contraband category label, a material category label and a bounding box label of a contraband position; the target detection model performs the following operations: performing feature extraction processing and feature fusion processing on each training sample to obtain a plurality of scales of fusion feature maps; performing material feature extraction processing on the plurality of scales of fusion feature maps to obtain a material feature map; performing occlusion degree feature extraction processing on the material feature map to obtain a spatial occlusion probability map; the spatial occlusion probability map comprises the relative occlusion severity of each pixel point after being occluded by a perspective occlusion block; obtaining an occlusion weighted feature map according to the fusion feature map and the spatial occlusion probability map; analyzing the occlusion weighted feature map to obtain a prediction result and determine a total training loss value based on the training label, performing back propagation training on the target detection model based on the total training loss value, until a training end condition is met, and obtaining a trained target detection model; the prediction result comprises at least part of a contraband category prediction probability, a predicted bounding box of a contraband position and a confidence.

2. The model training method of claim 1, wherein, Before the step of inputting training samples and training labels into a target detection model to be trained, the method further comprises the following steps: for each initial training sample, generating a transparent occlusion block corresponding to each polygonal pattern based on at least one randomly selected polygonal pattern, the transparency of each polygonal pattern and the color selected from the corresponding color system of each material category; superimposing each transparent occlusion block on the initial radiograph of the parcel to obtain a target radiograph as a training sample.

3. The model training method of claim 1, wherein, The target detection model comprises a feature extraction module; the step of performing feature extraction processing and feature fusion processing on each training sample to obtain a plurality of scales of fusion feature maps comprises the following steps: performing a series of convolution operations and down-sampling operations on the training sample through the backbone network of the feature extraction module to obtain a multi-scale feature map of the training sample; performing multi-scale feature map fusion processing on the multi-scale feature map through the feature pyramid network and / or path aggregation network of the feature extraction module to obtain a plurality of scales of fusion feature maps.

4. The model training method of claim 1, wherein, The target detection model comprises a material perception module; after the step of performing material feature extraction processing on the plurality of scales of fusion feature maps to obtain a material feature map, the method further comprises the following steps: performing channel number dimension reduction processing on the material feature map through at least one target size convolution kernel of the material perception module to obtain a dimension reduction feature map; performing non-linear transformation processing on the dimension reduction feature map through an activation function of the material perception module to obtain the prediction probability of each pixel point in each material category.

5. The model training method of claim 1, wherein, The target detection model comprises an occlusion perception module; the step of performing occlusion degree feature extraction processing on the material feature map to obtain a spatial occlusion probability map comprises the following steps: The material feature map is subjected to occlusion degree feature extraction processing through at least one convolutional layer of the occlusion perception module, to obtain an occlusion degree feature map; The occlusion degree feature map is subjected to non-linear transformation processing through an activation function of the occlusion perception module, to obtain the spatial occlusion probability map.

6. The model training method of claim 5, wherein, The method further includes: The fusion feature map and the spatial occlusion probability map are subjected to point-by-point multiplication through the occlusion perception module, to obtain the occlusion weighted feature map.

7. The model training method of claim 4, wherein, The target detection model includes a detection head. The method further includes: The occlusion weighted feature map is analyzed through the detection head, to obtain a prediction result and determine a total training loss value based on the prediction result and the training label. The prediction result is obtained through the detection head. A first loss value of a first classification loss function is determined through the detection head based on the material category prediction probability of each pixel point and the material category label. A second loss value of a second classification loss function is determined through the detection head based on the contraband category prediction probability and the contraband category label.

8. A contraband detection method characterized by, A third loss value of a bounding box positioning loss function and a fourth loss value of a distribution focus loss function are determined through the detection head based on the prediction bounding box and the bounding box label. The total training loss value is determined based on the first loss value, the second loss value, the third loss value, and the fourth loss value. The method further includes: A wrapped ray image is input to a trained target detection model, to obtain a detection result for the wrapped package. The detection result is a first detection result or a second detection result. The first detection result is used to indicate that there is no contraband in the wrapped package.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The second detection result is used to indicate that there is contraband in the wrapped package, a contraband category, and a location.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The target detection model is trained through the model training method of any one of claims 1-7. The processor implements the model training method of any one of claims 1-7 or the contraband detection method of claim 8 when executing the program. The computer program, when executed by the processor, implements the model training method of any one of claims 1-7 or the contraband detection method of claim 8.

Images