Contraband detection model training method and device based on image edge enhancement
Through image edge enhancement technology, image edge features are extracted and learned, and the problems of insufficient training samples and poor detection generalization capabilities of contraband detection models are solved, and high-precision and high-generalization contraband detection are achieved.
Patent Information
- Application Number
- CN202411173750.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, insufficient training samples of contraband detection models lead to low detection accuracy, complex background of items, occlusion, stacking, etc., resulting in poor generalization of contraband detection capabilities.
Using the contraband detection model training method based on image edge enhancement, the model's learning ability is improved through image preprocessing, edge feature extraction, deep semantic feature guidance learning and edge feature adaptive learning, combined with data augmentation and adaptive edge detection operators. and detection generalization ability.
It improves the accuracy and generalization ability of contraband detection, enhances the positioning performance and classification accuracy of contraband, and ensures the real-time nature of the algorithm.
Smart Images

Figure CN119027879A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image detection technology, and in particular to a method and device for training a contraband detection model based on image edge enhancement. Background Art
[0002] Security inspection is an indispensable part of the public transportation industry. In order to determine the safety of luggage items, it is usually necessary to manually identify the luggage images under X-rays to detect and block prohibited items from entering the public transportation link. However, due to factors such as mental fatigue, visual fatigue, and lack of concentration, manual screening has a certain risk of missed detection. In order to overcome manual missed detection, intelligent security detection came into being. Its core is to automatically identify X-ray images of prohibited items through algorithms. At present, the X-ray images of prohibited items identified by algorithms mainly use two-stage algorithms and one-stage algorithms. The advantage of the two-stage algorithm is high accuracy, but low efficiency and low real-time performance; the one-stage algorithm is proposed on the basis of the two-stage algorithm to solve the problem of low efficiency of the two-stage algorithm; but there are still problems such as insufficient model training samples leading to low detection accuracy, and poor generalization ability of prohibited item detection due to complex background, occlusion, and stacking of items. Therefore, in view of the defects of the above-mentioned one-stage algorithm, a contraband detection model training method and device based on image edge enhancement are urgently needed to overcome the above defects. Summary of the invention
[0003] In order to solve the problems of insufficient model training samples leading to low detection accuracy in X-ray images using algorithms to identify contraband, and poor generalization ability of contraband detection due to complex background, occlusion, and stacking of objects, this application provides the following technical solutions:
[0004] In a first aspect, a contraband detection model training method based on image edge enhancement is provided, comprising:
[0005] Image preprocessing is performed based on the original training image to obtain edge weak labels corresponding to the original training image and a data augmented image.
[0006] Based on the data enhancement image, the edge feature image is determined.
[0007] A feature map is determined based on the data augmented image and the edge feature image.
[0008] Based on the feature map, deep semantic feature guided learning and edge feature adaptive learning are performed to obtain feature learning results corresponding to the feature map.
[0009] Based on the feature learning result, at least one dimensional feature map corresponding to the feature learning result is obtained.
[0010] Based on the at least one dimensional feature map, at least one scale feature corresponding to the at least one dimensional feature is obtained.
[0011] Based on at least one phase feature, at least one prediction intermediate result and a prediction result are obtained, and the parameters of the contraband detection model based on image edge enhancement are updated according to the at least one prediction intermediate result and the prediction result.
[0012] Image preprocessing is performed based on the original training image to obtain the edge weak labels corresponding to the original training image and the data augmented image, including:
[0013] An original training image and a label corresponding to the original training image are obtained, wherein the label at least includes a position label.
[0014] Contraband images are identified based on the original training images and location labels.
[0015] Sharpen the edges of the contraband image.
[0016] Based on the sharpened contraband image, a contraband edge binary map is determined.
[0017] In the original training image, the contraband image is replaced with the contraband edge binary map as the weak edge label.
[0018] A random transformation operation is performed on the original training image to obtain a corresponding transformed image, wherein the random transformation operation includes at least one of the following: random scaling, random rotation, and random folding.
[0019] The transformed image is resized to a preset size, and the image mean and image variance of the transformed image after the size adjustment are determined.
[0020] Based on the image mean and image variance, the resized transformed image is standardized to obtain a data augmented image.
[0021] Further, based on the data enhanced image, an edge feature image is determined, including:
[0022] Based on the data enhanced image, a lateral gradient image of the data enhanced image is determined.
[0023] Based on the data enhanced image, a longitudinal gradient image of the data enhanced image is determined.
[0024] The horizontal gradient image and the vertical gradient image are fused to obtain the edge feature image.
[0025] Further, determining a feature map based on the data enhanced image and the edge feature image includes:
[0026] The data augmented image and the edge feature image are spliced to obtain a spliced image.
[0027] Perform convolution layer, regularization layer, and activation layer operations on the edge feature image to obtain an edge feature sampling map.
[0028] Perform convolution layer, regularization layer, and activation layer operations on the spliced image to obtain a spliced sampling image.
[0029] The edge feature sampling map is concatenated with the concatenated sampling map to obtain a feature map.
[0030] Furthermore, based on the feature learning result, at least one dimensional feature map corresponding to the feature learning result is obtained, including:
[0031] Perform at least one downsampling edge-guided convolution and adaptive guided learning on the feature learning result to obtain at least one dimensional feature map corresponding to the feature learning result.
[0032] Further, based on the at least one dimensional feature map, obtaining at least one scale feature corresponding to the at least one dimensional feature includes:
[0033] Separate features of at least one dimensional feature map to obtain corresponding at least one semantic feature and at least one adaptive edge feature.
[0034] Perform Gaussian filtering on at least one adaptive edge feature to obtain at least one Gaussian filtering feature.
[0035] Path fusion is performed on at least one semantic feature to obtain at least one deep semantic feature.
[0036] At least one Gaussian filter feature is correspondingly multiplied with at least one deep semantic feature to obtain at least one scale feature corresponding to at least one dimensional feature.
[0037] Further, based on at least one phase feature, at least one prediction intermediate result and a prediction result are obtained, and parameters of a contraband detection model based on image edge enhancement are updated according to the at least one prediction intermediate result and the prediction result, including:
[0038] Inputting at least one scale feature into a contraband detection model based on image edge enhancement to obtain at least one prediction intermediate result and a prediction result;
[0039] Determine a marginal weak label loss based on at least one prediction intermediate result and the marginal weak label;
[0040] Determine the target supervision loss based on the prediction results and labels;
[0041] Determine the training loss based on the marginal weak label loss and the target supervision loss;
[0042] In response to the training loss being greater than a preset value, the parameters of the contraband detection model based on image edge enhancement are reversely updated.
[0043] In a second aspect, a contraband detection model training device based on image edge enhancement is provided, comprising:
[0044] The preprocessing module is used to perform image preprocessing based on the original training image to obtain the edge weak label corresponding to the original training image and the data enhanced image.
[0045] The edge feature determination module is used to determine the edge feature image based on the data enhancement image.
[0046] The feature map determination module is used to determine the feature map based on the data enhancement image and the edge feature image.
[0047] The learning result determination module is used to perform deep semantic feature guided learning and edge feature adaptive learning based on the feature map to obtain feature learning results corresponding to the feature map.
[0048] The dimensional feature determination module is used to obtain at least one dimensional feature map corresponding to the feature learning result based on the feature learning result.
[0049] The scale feature determination module is used to obtain at least one scale feature corresponding to at least one dimensional feature based on at least one dimensional feature map.
[0050] The prediction module is used to obtain at least one prediction intermediate result and a prediction result based on at least one phase feature, and update the parameters of the contraband detection model based on image edge enhancement according to the at least one prediction intermediate result and the prediction result.
[0051] In a third aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the contraband detection model training method based on image edge enhancement as described in the first aspect.
[0052] In a fourth aspect, a computer-readable storage medium is provided, on which a contraband detection model training program based on image edge enhancement is stored. When the contraband detection model training program based on image edge enhancement is executed by a processor, the contraband detection model training method based on image edge enhancement recorded in the first aspect is implemented.
[0053] In a fifth aspect, a computer device is provided, comprising a memory, a processor, and a contraband detection model training program based on image edge enhancement stored in the memory and executable on the processor, wherein when the processor executes the contraband detection model training program based on image edge enhancement, the contraband detection model training method based on image edge enhancement described in the first aspect is implemented.
[0054] In a sixth aspect, a contraband detection model detection method based on image edge enhancement is provided, comprising:
[0055] Get the image to be detected.
[0056] A contraband detection model based on image edge enhancement is used to detect the image to be detected to obtain a contraband detection result, wherein the contraband detection model based on image edge enhancement is trained by the contraband detection model training method based on image edge enhancement recorded in the first aspect.
[0057] The beneficial effects of the technical solution provided by the embodiment of the present application are:
[0058] By implementing the contraband detection model training method, device, system and detection method based on image edge enhancement disclosed in the embodiment of the present application, a new X-ray image contraband detection method based on deep learning is established through an adaptive edge feature strategy and an edge feature guidance strategy, so as to realize intelligent detection and identification of contraband. Compared with the conventional two-stage and one-stage contraband detection algorithms, the X-ray image contraband detection method based on adaptive edge feature guidance performs preprocessing through data enhancement to improve the learning ability of the model and the generalization ability of detection; the edge information of the image is extracted by using the edge detection operator and the adaptive edge detection operator using the adaptive edge feature strategy, and the multi-scale deep semantic information of the image is adaptively guided by the CNN model through the edge feature guidance strategy, so that the detection of contraband of different sizes, different categories and different scenes can be realized, and the accuracy and generalization ability of contraband detection are improved, and the positioning performance and classification accuracy of contraband are also improved, and the real-time performance of the algorithm is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0060] Figure 1 It is a schematic diagram of a contraband detection model training method based on image edge enhancement provided in an embodiment of the present application;
[0061] Figure 2 is a schematic diagram of an adaptive guided learning module provided in an embodiment of the present application;
[0062] Figure 3 It is a schematic diagram of the training process of a contraband detection model based on image edge enhancement provided in an embodiment of the present application;
[0063] Figure 4 is a schematic diagram of the model structure of a contraband detection model based on image edge enhancement provided in an embodiment of the present application;
[0064] Figure 5 It is a schematic diagram of a contraband detection model training device based on image edge enhancement provided in an embodiment of the present application;
[0065] Figure 6 It is a schematic diagram of a contraband detection model detection method based on image edge enhancement provided in an embodiment of the present application;
[0066] Figure 7 It is a schematic diagram of the detection process of the contraband detection model based on image edge enhancement provided in an embodiment of the present application. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in the implementation mode of this application will be clearly and completely described below in conjunction with the drawings in the implementation mode of this application. Obviously, the described implementation mode is only a part of the embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0068] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure shall have the usual meanings understood by persons with ordinary skills in the field to which the present disclosure belongs. The words "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one", "one" or "the" do not indicate a quantity limitation, but indicate that there is at least one. The numbers in the drawings of the specification only indicate the distinction between various functional components or modules, and do not indicate the logical relationship between components or modules. Words such as "include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0069] Hereinafter, various embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that in the accompanying drawings, the same reference numerals are given to components having substantially the same or similar structures and functions, and repeated descriptions thereof will be omitted.
[0070] In order to solve the problems of low detection accuracy caused by insufficient model training samples in X-ray images for algorithm identification of contraband, and poor generalization ability of contraband detection caused by complex background, occlusion, and stacking of objects, the present application provides the following embodiments.
[0071] In some embodiments, Figure 1 As shown, a contraband detection model training method based on image edge enhancement includes:
[0072] A100: Perform image preprocessing based on the original training image to obtain the edge weak labels corresponding to the original training image and the data augmented image.
[0073] A200: Determine edge feature images based on data-enhanced images.
[0074] A300: Determines feature maps based on data-enhanced images and edge feature images.
[0075] A400: Based on the feature map, deep semantic feature guided learning and edge feature adaptive learning are performed to obtain feature learning results corresponding to the feature map.
[0076] A500: Based on the feature learning result, obtain at least one dimensional feature map corresponding to the feature learning result.
[0077] A600: Based on at least one dimensional feature graph, obtain at least one scale feature corresponding to the at least one dimensional feature.
[0078] A700: Based on at least one phase feature, obtain at least one prediction intermediate result and a prediction result, and update parameters of a contraband detection model based on image edge enhancement according to the at least one prediction intermediate result and the prediction result.
[0079] Image preprocessing is performed based on the original training image to obtain the edge weak labels corresponding to the original training image and the data augmented image, including:
[0080] A110: Obtain an original training image and a label corresponding to the original training image, wherein the label includes at least a position label.
[0081] The original training image refers to an X-ray image of the object to be detected containing contraband. The object to be detected referred to in the embodiment of the present application is an object with a storage space inside and the contents of which cannot be identified from the outside in the visible light band. Schematically, the object to be detected can be a suitcase, a backpack, a handbag, a logistics transfer box, etc.
[0082] The tag includes at least the location tag and category labels The position label indicates the position of the contraband in the original training image; the category label indicates the type of contraband, such as flammable and explosive materials, explosives, poisons, corrosive materials, radioactive materials, firearms and ammunition, controlled devices, and items containing lithium batteries. The position label and category label are obtained by annotation. This application does not limit the process of annotating the position label and type label. Usually, in order to ensure the accuracy of the information recorded in the original training image and improve the accuracy of model training, the position label and category label are manually annotated.
[0083] A120: Identify contraband images based on original training images and location labels.
[0084] Determining the prohibited item image is to extract a separate prohibited item image from the original training image according to the indication of the position label. The determined prohibited item image is a part of the original training image.
[0085] A130: Sharpen the edges of the contraband image.
[0086] Edge sharpening refers to processing the edges of an image through specific techniques or tools to enhance the clarity and focus of the image. Edge sharpening improves the edge clarity and contrast of the contraband image area, thereby improving the local quality of the contraband image, which is beneficial to improving the quality of the training samples.
[0087] A140: Determine a binary image of the contraband edge based on the sharpened contraband image.
[0088] The binary image of prohibited items edge refers to a binary image in which the edge of prohibited items is set to white or black, and the non-edge part of prohibited items is set to black or white. The binary image of prohibited items edge can help the prohibited items detection model based on image edge enhancement to more accurately identify and process prohibited items in the image.
[0089] A150: In the original training image, the contraband image is replaced with the contraband edge binary map as the weak edge label.
[0090] Preferably, in the original training image, after replacing the contraband image with the contraband edge binary image, the pixels outside the contraband edge binary image are set to 0 as weak edge labels. Among them, the edge weak label is a binary image including the outline of contraband.
[0091] A160: Perform a random transformation operation on the original training image to obtain the corresponding transformed image X′.
[0092] The random transformation operation includes at least one of the following: random scaling, random rotation, and random folding. It means that the original training image is scaled, rotated, and folded with random probability to obtain a transformed image. Schematically, during a random transformation operation, the random probability of scaling is 0, the random probability of rotation is 50%, and the random probability of folding is 100%. Then, scaling is not performed on the original training image, and there is a 50% probability of rotating the original training image, and the corresponding transformed image is obtained after folding. The order of scaling, rotation, and folding can be pre-set or random, and the present application does not limit the execution order of scaling, rotation, and folding.
[0093] A170: Adjust the transformed image X′ to a preset size, and determine the image mean and image variance of the transformed image after the size adjustment.
[0094] The preset size refers to a preset resolution of the transformed image, which is expressed as H*W, where H represents the vertical resolution of the transformed image, and W represents the horizontal resolution of the transformed image.
[0095] Image mean refers to the average value of the image pixel values. It reflects the average level of the overall brightness or color of the image. The larger the image mean, the more the image's measurement or color tends to be higher; conversely, it tends to be lower. The image mean can be obtained by calculating the average of all pixel values in the entire image.
[0096] Image variance refers to the average value of the sum of squares of the difference between the image pixel values and the image mean. It reflects the degree of dispersion of the image pixel values relative to the image mean. That is, the distribution range or degree of variation of the image pixel values. The larger the image variance, the more dispersed the distribution of the image pixel values, and the richer the image details and contrast; the smaller the image variance, the more concentrated the image pixel values, and the image is uniform or consistent.
[0097] Optionally, the image mean is obtained by presetting; and the image variance is calculated based on the image mean.
[0098] A180: Based on the image mean and image variance, the resized transformed image X′ is standardized to obtain the data augmented image X a .
[0099] Image standardization is achieved by subtracting the image mean and dividing it by the image standard deviation. This process can accelerate the convergence of the contraband detection model based on image edge enhancement. The image standard deviation is the arithmetic square root of the image variance.
[0100] By preprocessing the original training image, the data enhanced image X can be obtained. a , marginal weak labels Combined with location tags and category labels Provide a basis for subsequent implementation methods.
[0101] Based on data augmentation image X a , determine the edge feature image X e ,include:
[0102] A210: Data-based Image Augmentation X a , determine the lateral gradient image D of the data augmented image x .
[0103] Alternatively, the transverse gradient may be a transverse Sobel gradient.
[0104] Specifically, according to:
[0105] D x (i,j)=f(i-1,j-1)+2f(i-1,j)+f(i-1,j+1)-f(i+1,j-1)-2f(i+1,j)-f(i+1,j+1), determine the data augmented image X a The lateral Sobel gradient image D x Among them, i and j represent the horizontal and vertical positions of the pixel respectively, D x (i,j) represents the data augmented image X a In the figure, the lateral Sobel gradient at the pixel point (i, j), f() represents the data augmented image X a The image pixel value corresponding to the pixel in .
[0106] A220: Data-based Image Augmentation X a , determine the longitudinal gradient image D of the data augmented image y .
[0107] Optionally, the longitudinal gradient may be a Sobel gradient longitudinally.
[0108] Specifically, according to:
[0109] D y(i,j)=f(i-1,j+1)+2f(i,j+1)+f(i+1,j+1)-f(i-1,j+1)-2f(i,j-1)-f(i+1,j-1), determine the data augmented image X a The longitudinal Sobel gradient image D y Among them, D y (i,j) represents the data augmented image X a , the longitudinal Sobel gradient at the pixel (i, j).
[0110] A230: Fusion of lateral gradient images D x And the longitudinal gradient image D y , get the edge feature image X e .
[0111] Specifically, according to:
[0112] Determine the edge feature image X e Among them, X e (i, j) represents the gradient value at point (i, j) in the edge feature image.
[0113] Based on data augmentation image X a , and edge feature image X e Determine the feature map, including:
[0114] A310: Data augmentation image X a , and edge feature image X e Stitching to get the stitched image X 0 .
[0115] A320: Edge feature image X e Perform convolutional layer, regularization layer, and activation layer operations (CBL operations) to obtain an edge feature sampling map.
[0116] In the edge feature image X e After performing the convolutional layer, regularization layer, and activation layer operations, the H*W size image is downsampled to obtain an edge feature sampling map.
[0117] Optionally, the size of the edge feature sampling map is
[0118] A330: stitching images X 0 Perform convolutional layer, regularization layer, and activation layer operations to obtain a spliced sampling graph.
[0119] In stitching image X 0 After performing the convolutional layer, regularization layer, and activation layer operations, the H*W size image is downsampled to obtain a spliced sampling image.
[0120] Optionally, the size of the spliced sampling map is
[0121] A340: Concatenate the edge feature sampling map with the splicing sampling map to obtain the feature map P 0 .
[0122] Based on the feature map, deep semantic feature guided learning and edge feature adaptive learning are performed to obtain feature learning results corresponding to the feature map.
[0123] The feature map P 0 Input the adaptive guided learning module to conduct deep semantic feature guided learning and edge feature adaptive learning. Figure 2 As shown, the adaptive guided learning module includes: an edge guided convolution submodule, a separation semantic feature submodule, a separation adaptive edge feature submodule and a Bottleneck submodule.
[0124] The edge-guided convolution submodule separates the input feature map (H×W×k) by channel, then performs convolution, adaptively extracts edge features, and uses edge features to guide the learning of deep semantic features. Schematically, the input image is first expanded along the channel, and the last channel is copied to form a (H×W×k+1)-dimensional feature map. Then, a separation operation is performed to separate the (H×W×k+1)-dimensional feature map into a (H×W×k)-dimensional feature map and a (H×W×1)-dimensional feature map. Then, a convolution operation is performed on the (H×W×k)-dimensional feature map, and an adaptive edge convolution operation is performed on the (H×W×1)-dimensional feature map. A semantic feature map with edge information and an adaptively learned edge feature map are obtained.
[0125] The semantic feature separation submodule separates the semantic features obtained in the edge-guided convolution. Schematically, the first (H×W×k)-dimensional semantic features of the edge-guided convolution result are taken out.
[0126] The separation adaptive edge feature submodule is to separate the adaptive edge features obtained in the edge-guided convolution. Schematically, the rear (H×W×1)-dimensional edge features of the edge-guided convolution result are taken out.
[0127] As well as the Bottleneck submodule, it performs deep residual learning on the semantic feature map with edge information, and learns high-level semantic information more delicately according to the guidance of edge information.
[0128] Based on the feature learning result, at least one dimensional feature map corresponding to the feature learning result is obtained, including:
[0129] Perform at least one downsampling edge-guided convolution and adaptive guided learning on the feature learning result to obtain at least one dimensional feature map.
[0130] Schematically, this part is explained by taking the execution of three down-sampling edge-guided convolutions and adaptive guided learning as an example:
[0131] A510: Input the feature learning results into the downsampled edge-guided convolution, adaptively extract edge features and use edge features to guide the learning of deep semantic features, and reduce the input feature size to 1 / 4, resulting in a size of .
[0132] A520: Input the graph obtained in A510 into the adaptive guided learning module for processing, and obtain a graph of size .
[0133] A530: Downsample the image input obtained in A520 and use edge-guided convolution to adaptively extract edge features and use edge features to guide the learning of deep semantic features, and reduce the input feature size to 1 / 4. The size is .
[0134] A540: The graph obtained in A530 is input into the adaptive guided learning module for processing, and the size is .
[0135] A550: Downsample the image input obtained by A540 and use edge-guided convolution to adaptively extract edge features and use edge features to guide the learning of deep semantic features, and reduce the input feature size to 1 / 4. The size is .
[0136] A560: The graph obtained in A550 is input into the adaptive guided learning module for processing, and the size is .
[0137] A520, A540 and A560 are used to obtain feature maps of large, medium and small dimensions.
[0138] Based on the at least one dimensional feature graph, obtaining at least one scale feature corresponding to the at least one dimensional feature includes:
[0139] A610: Separate features of at least one dimensional feature map to obtain at least one corresponding semantic feature and at least one adaptive edge feature.
[0140] A620: Perform Gaussian filtering on at least one adaptive edge feature to obtain at least one Gaussian filtering feature.
[0141] A630: Perform path fusion on at least one semantic feature to obtain at least one deep semantic feature.
[0142] A640: Multiply at least one Gaussian filter feature with at least one deep semantic feature to obtain at least one scale feature corresponding to at least one dimensional feature.
[0143] Continue to perform three downsampling edge-guided convolutions and the results obtained by adaptive guided learning to explain this part:
[0144] Separate the features of the three-dimensional feature maps to obtain the corresponding three semantic features (M0, M1, M2) and three adaptive edge features
[0145] Gaussian filtering of three adaptive edge features Get the corresponding Gaussian filter feature
[0146] Specifically, according to: Gaussian filtering is performed on the three adaptive edge features, where G(p) represents the Gaussian filtering result, p represents the coordinates of the pixel point, and σ represents the standard deviation of the Gaussian function. The Gaussian filter convolves the adaptive edge features with the Gaussian kernel, so that the high-frequency components in the adaptive edge features are reduced, and the high-frequency noise in the adaptive edge features is removed, while the gradient information in the adaptive edge features is retained, the edge blur is avoided, and more detail information is retained.
[0147] Perform path fusion on the three semantic features (M0, M1, M2) to obtain deep semantic features
[0148] Optionally, feature pyramid strategy and path fusion strategy are used to fuse information from top to bottom and bottom to obtain deep semantic features.
[0149] Gaussian filter feature With deep semantic features Corresponding multiplication, we get three scale features The purpose of this operation is to spatially highlight the important information in the multi-scale deep semantic feature map, which is conducive to detecting key information.
[0150] Based on at least one scale feature, at least one prediction intermediate result and a prediction result are obtained, and parameters of a contraband detection model based on image edge enhancement are updated according to the at least one prediction intermediate result and the prediction result, including:
[0151] A710: inputting at least one scale feature into a contraband detection model based on image edge enhancement to obtain at least one prediction intermediate result and a prediction result;
[0152] A720: Determine a marginal weak label loss based on at least one prediction intermediate result and a marginal weak label;
[0153] A730: Determine the target supervision loss based on the prediction results and labels;
[0154] A740: Determine the training loss based on the marginal weak label loss and the target supervision loss;
[0155] A750: In response to the training loss being greater than a preset value, the parameters of the contraband detection model based on image edge enhancement are reversely updated.
[0156] The explanation of this part still uses three scale characteristics: Instance of .
[0157] The three scale features The contraband detection model based on image edge enhancement is input, and the above three scale features can obtain at least one prediction intermediate result and a prediction result through its prediction network.
[0158] At least one prediction intermediate result is the result obtained by processing the above three scale features through each layer of the network except the last layer of the prediction network, which can be in the form of a MAP graph. The number of prediction intermediate results is the difference between the number of layers of the prediction network and 1. The prediction result is the result obtained by processing the above three scale features through all layers of the prediction network, in the form of Prediction results Contains the predicted location information and predicted category information
[0159] The information output by the prediction network is subjected to non-maximum suppression to remove the target boxes that do not meet the requirements and retain the target boxes that meet the requirements.
[0160] Preferably, the category information is predicted according to the Softmax function.
[0161] Specifically, according to Predict category information. N represents the number of all categories, n represents the category number, and k represents the specific category.
[0162] Preferably, the positioning loss LOSS is calculated according to the intersection-over-union function b , the cross entropy function calculates the classification loss LOSS c .
[0163] The localization loss reflects the difference between the predicted bounding box localization position and the annotated bounding box position, i.e., the position label The classification loss reflects the difference between the predicted contraband category and the labeled contraband category, that is, the category label The difference between.
[0164] According to: LOSS ob =αLOSS b +βLOSS c , determine the target supervision loss. Among them, LOSS ob is the target supervision loss, α is the positioning loss weight, and β is the classification loss weight.
[0165] According to: LOSS total =LOSS ob +LOSS e Determine the training loss. Utilize marginal weak labels The prediction network results are supervised and guided to ensure that the deep network model can accurately hold the image edge information in the deep semantic information. total is the training loss, LOSS e is the marginal weak label loss.
[0166] Edge weak label loss LOSS e ,according to: Of these, LOSS m is the loss of the corresponding result obtained by processing a certain layer of the prediction network, when m=M, it refers to the prediction result; w m It is the weight of the loss of the corresponding result obtained after a certain layer of the prediction network is processed, m indicates a certain layer of the prediction network, and M is the number of network layers of the prediction network.
[0167] Illustratively, for a three-layer network, w1=1, w2=1, w3=1; or w1=0.5, w2=0.5, w3=0.2 are often selected.
[0168] The embodiment of the present application discloses constraining the training loss with dual constraints of target supervision loss and edge weak label loss. Under such dual constraints, the trained contraband detection model based on image edge enhancement improves the recognition accuracy of the location and type of contraband.
[0169] The preset value determines when the training loss is large and the model needs to be trained further; when the model reaches the accuracy requirement, the training can be stopped. A contraband detection model based on image edge enhancement with optimized parameters can be obtained.
[0170] Optionally, the preset value is preset.
[0171] The training process of the contraband detection model based on image edge enhancement described above is as follows Figure 3 shown.
[0172] The model structure of the contraband detection model based on image edge enhancement described above is as follows Figure 4 As shown, it corresponds to the training process of the contraband detection model based on image edge enhancement described above.
[0173] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0174] In other embodiments, Figure 5 As shown, a contraband detection model training device based on image edge enhancement includes:
[0175] The preprocessing module is used to perform image preprocessing based on the original training image to obtain the edge weak label corresponding to the original training image and the data enhanced image.
[0176] The edge feature determination module is used to determine the edge feature image based on the data enhancement image.
[0177] The feature map determination module is used to determine the feature map based on the data enhancement image and the edge feature image.
[0178] The learning result determination module is used to perform deep semantic feature guided learning and edge feature adaptive learning based on the feature map to obtain feature learning results corresponding to the feature map.
[0179] The dimensional feature determination module is used to obtain at least one dimensional feature map corresponding to the feature learning result based on the feature learning result.
[0180] The scale feature determination module is used to obtain at least one scale feature corresponding to at least one dimensional feature based on at least one dimensional feature map.
[0181] The prediction module is used to obtain at least one prediction intermediate result and a prediction result based on at least one phase feature, and update the parameters of the contraband detection model based on image edge enhancement according to the at least one prediction intermediate result and the prediction result.
[0182] For the specific limitations of the above-mentioned contraband detection model training device based on image edge enhancement, please refer to the above-mentioned limitations on the contraband detection model training method based on image edge enhancement, which will not be repeated here. Each module in the above-mentioned contraband detection model training device based on image edge enhancement can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0183] In some other embodiments, a computer program product includes a computer program, which, when executed by a processor, implements the above-mentioned contraband detection model training method based on image edge enhancement.
[0184] In other embodiments, a computer-readable storage medium stores a contraband detection model training program based on image edge enhancement. When the contraband detection model training program based on image edge enhancement is executed by a processor, the above-mentioned contraband detection model training method based on image edge enhancement is implemented.
[0185] In other embodiments, a computer device includes a memory, a processor, and a contraband detection model training program based on image edge enhancement stored in the memory and executable on the processor. When the processor executes the contraband detection model training program based on image edge enhancement, the above-mentioned contraband detection model training method based on image edge enhancement is implemented.
[0186] In other embodiments, an X-ray machine examines an object to be detected containing contraband, and collects the image of the object to be detected under the X-ray machine as an original image. In the original image, the image of the contraband is labeled as an original training image. The label includes a position label and a category label, which respectively indicate the position of the contraband in the original image and the type of the contraband. The original training image is input into the above-mentioned computer device for training the contraband detection model based on image edge enhancement.
[0187] The computer device may be a personal computer or a server or other device capable of training a contraband detection model based on image edge enhancement.
[0188] When the computer device is used to perform training of the contraband detection model based on image edge enhancement, the contraband detection model training program based on image edge enhancement stored in the memory is run by the processor of the computer device to execute:
[0189] A100: Perform image preprocessing based on the original training image to obtain the edge weak labels corresponding to the original training image and the data augmented image.
[0190] A200: Determine edge feature images based on data-enhanced images.
[0191] A300: Determines feature maps based on data-enhanced images and edge feature images.
[0192] A400: Based on the feature map, deep semantic feature guided learning and edge feature adaptive learning are performed to obtain feature learning results corresponding to the feature map.
[0193] A500: Based on the feature learning result, obtain at least one dimensional feature map corresponding to the feature learning result.
[0194] A600: Based on at least one dimensional feature graph, obtain at least one scale feature corresponding to the at least one dimensional feature.
[0195] A700: Based on at least one phase feature, obtain at least one prediction intermediate result and a prediction result, and update parameters of a contraband detection model based on image edge enhancement according to the at least one prediction intermediate result and the prediction result.
[0196] In some other embodiments, a contraband detection model detection method based on image edge enhancement, such as Figure 6 As shown, including:
[0197] B100: Get the image to be detected.
[0198] B200: Use a contraband detection model based on image edge enhancement to detect the image to be detected to obtain a contraband detection result, wherein the contraband detection model based on image edge enhancement is trained by the contraband detection model training method based on image edge enhancement recorded in the first aspect.
[0199] The contraband detection model training method based on image edge enhancement is used to obtain optimized model parameters. The process of using the parameter-optimized contraband detection model based on image edge enhancement to detect contraband is as follows: Figure 7 As shown. The image to be detected is input into the model, and finally the bounding box positioning information and contraband classification information are obtained through the prediction network. The generated bounding box positioning information and contraband classification information are displayed on the image to be detected, and the final contraband detection result Figure Y is obtained. pre .
[0200] By implementing the contraband detection model training method, device, system and detection method based on image edge enhancement disclosed in the embodiment of the present application, a new X-ray image contraband detection method based on deep learning is established through adaptive edge feature strategy and edge feature guidance strategy to realize intelligent detection and recognition of contraband. Compared with the conventional two-stage and one-stage contraband detection algorithms, the X-ray image contraband detection method based on adaptive edge feature guidance performs preprocessing through data enhancement to improve the learning ability of the model and the generalization ability of detection; the edge information of the image is extracted by using the adaptive edge feature strategy using the edge detection operator and the adaptive edge detection operator, and at the same time, the multi-scale deep semantic information of the image is adaptively guided by the CNN model through the edge feature guidance strategy, so that the detection of contraband of different sizes, different categories and different scenes can be realized, which improves the accuracy and generalization ability of contraband detection, and also improves the positioning performance and classification accuracy of contraband, and ensures the real-time performance of the algorithm. By adopting the dual constraints of target supervision loss and edge weak label loss for loss, the recognition accuracy of contraband positioning and classification is further improved.
[0201] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0202] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program loaded on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a memory, or installed from a ROM. When the computer program is executed by an external processor, the above-mentioned functions defined in the method of the embodiment of the present application are executed.
[0203] It should be noted that the computer-readable medium of the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In an embodiment of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In an embodiment of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0204] The computer-readable medium may be included in the server; or it may exist independently without being installed in the server. The computer-readable medium carries one or more programs. When the one or more programs are executed by the server, the server: in response to detecting that the peripheral mode of the terminal is not activated, obtains the frame rate of the application on the terminal; when the frame rate meets the screen-off condition, determines whether the user is obtaining the screen information of the terminal; in response to the judgment result that the user is not obtaining the screen information of the terminal, controls the screen to enter the immediate dimming mode.
[0205] Computer program code for performing the operation of the embodiments of the present application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).
[0206] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0207] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present application.
[0208] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A contraband detection model training method based on image edge enhancement, characterized in that: include: Performing image preprocessing based on the original training image to obtain a weak edge label corresponding to the original training image and a data augmented image; Determine an edge feature image based on the data enhanced image; Determine a feature map based on the data enhanced image and the edge feature image; Based on the feature graph, deep semantic feature guided learning and edge feature adaptive learning are performed to obtain feature learning results corresponding to the feature graph; Based on the feature learning result, obtaining at least one dimensional feature map corresponding to the feature learning result; Based on the at least one dimensional feature map, obtaining at least one scale feature corresponding to the at least one dimensional feature; Based on the at least one scale feature, at least one prediction intermediate result and a prediction result are obtained, and the parameters of the contraband detection model based on image edge enhancement are updated according to the at least one prediction intermediate result and the prediction result.
2. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: The image preprocessing is performed based on the original training image to obtain the edge weak label corresponding to the original training image and the data enhanced image, including: Acquire the original training image and a label corresponding to the original training image, wherein the label at least includes a position label; Determining a contraband image according to the original training image and the location label; sharpening the edge of the contraband image; Based on the sharpened contraband image, a binary image of the contraband edge is determined; In the original training image, replacing the contraband image with the contraband edge binary image as the edge weak label; Performing a random transformation operation on the original training image to obtain a corresponding transformed image, wherein the random transformation operation includes at least one of the following: random scaling, random rotation, and random folding; Adjusting the transformed image to a preset size, and determining an image mean and an image variance of the transformed image after the size adjustment; The resized transformed image is standardized based on the image mean and the image variance to obtain the data enhanced image.
3. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: The step of determining an edge feature image based on the data enhanced image comprises: Based on the data enhanced image, determining a lateral gradient image of the data enhanced image; Based on the data enhanced image, determining a longitudinal gradient image of the data enhanced image; The transverse gradient image and the longitudinal gradient image are fused to obtain the edge feature image.
4. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: The determining of the feature map based on the data enhanced image and the edge feature image comprises: Splicing the data enhanced image and the edge feature image to obtain a spliced image; Performing convolution layer, regularization layer, and activation layer operations on the edge feature image to obtain an edge feature sampling map; Performing convolution layer, regularization layer, and activation layer operations on the spliced image to obtain a spliced sampling image; The edge feature sampling map is spliced with the splicing sampling map to obtain the feature map.
5. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: Based on the feature learning result, obtaining at least one dimensional feature map corresponding to the feature learning result includes: Perform at least one downsampling edge-guided convolution and adaptive guided learning on the feature learning result to obtain at least one dimensional feature map corresponding to the feature learning result.
6. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: The obtaining, based on the at least one dimensional feature map, at least one scale feature corresponding to the at least one dimensional feature comprises: Separating features of the at least one dimensional feature map to obtain corresponding at least one semantic feature and at least one adaptive edge feature; Performing Gaussian filtering on the at least one adaptive edge feature to obtain at least one Gaussian filtering feature; Performing path fusion on the at least one semantic feature to obtain at least one deep semantic feature; The at least one Gaussian filter feature is correspondingly multiplied with the at least one deep semantic feature to obtain at least one scale feature corresponding to the at least one dimensional feature.
7. The contraband detection model training method based on image edge enhancement according to claim 1 is characterized in that: The obtaining, based on the at least one phase feature, at least one intermediate prediction result and a prediction result, and updating the parameters of the contraband detection model based on image edge enhancement according to the at least one intermediate prediction result and the prediction result, comprises: Inputting the at least one scale feature into a contraband detection model based on image edge enhancement to obtain at least one prediction intermediate result and a prediction result; Determine an edge weak label loss based on the at least one prediction intermediate result and the edge weak label; Determine a target supervision loss based on the prediction results and the labels; Determining a training loss based on the edge weak label loss and the target supervision loss; In response to the training loss being greater than a preset value, the parameters of the contraband detection model based on image edge enhancement are reversely updated.
8. A contraband detection model training device based on image edge enhancement, characterized in that: include: A preprocessing module, used for performing image preprocessing based on the original training image to obtain an edge weak label corresponding to the original training image and a data enhanced image; An edge feature determination module, used to determine an edge feature image based on the data enhanced image; A feature map determining module, configured to determine a feature map based on the data enhanced image and the edge feature image; A learning result determination module, used to perform deep semantic feature guided learning and edge feature adaptive learning based on the feature graph to obtain a feature learning result corresponding to the feature graph; A dimensional feature determination module, configured to obtain, based on the feature learning result, at least one dimensional feature graph corresponding to the feature learning result; A scale feature determination module, configured to obtain at least one scale feature corresponding to the at least one dimensional feature based on the at least one dimensional feature map; A prediction module is used to obtain at least one prediction intermediate result and a prediction result based on the at least one phase feature, and to update the parameters of the contraband detection model based on image edge enhancement according to the at least one prediction intermediate result and the prediction result.
9. A computer-readable storage medium, characterized in that A contraband detection model training program based on image edge enhancement is stored thereon. When the contraband detection model training program based on image edge enhancement is executed by a processor, the contraband detection model training method based on image edge enhancement according to any one of claims 1 to 7 is implemented.
10. A contraband detection model detection method based on image edge enhancement, characterized in that: include: Get the image to be detected; The image to be detected is detected using a contraband detection model based on image edge enhancement to obtain a contraband detection result, wherein the contraband detection model based on image edge enhancement is trained by the contraband detection model training method based on image edge enhancement described in any one of claims 1 to 7.
Citation Information
Patent Citations
Remote sensing target detection method based on single-stage full convolutional network and multi-feature fusion
CN113177456A
RGBD image semantic segmentation method based on boundary attention
CN113822284A
Thermal infrared electrical equipment image semantic segmentation method based on edge auxiliary learning
CN114140480A
Method and device for detecting forbidden articles in security check image
CN114676759A
Prohibited article detection method and system based on natural guidance and data enhancement
CN114898131A
Cited By
Parcel contraband detection method, system and device and storage medium
CN120876480A
