Deep learning-based millimeter wave image hidden target detection method and device and storage medium

Abstract

This disclosure relates to a method, apparatus, and storage medium for cloaked target detection in millimeter-wave images based on deep learning. The method includes: preprocessing the acquired image to statistically analyze the target distribution patterns; segmenting the target contours according to the target distribution patterns and fusing them with a background sample set; detecting specific target objects based on a target detection network; and detecting cloaked targets in millimeter-wave images based on training an improved target detection model. Through the various embodiments of this disclosure, the generalization ability of the model and its ability to detect targets at different scales are effectively improved, meeting the needs of practical applications.

Description

Technical Field

[0001] This disclosure relates to the field of concealed target detection in millimeter-wave images, specifically to a deep learning-based method for concealed target detection in millimeter-wave images, a deep learning-based device for concealed target detection in millimeter-wave images, and a computer-readable storage medium. Background Technology

[0002] Current object detection algorithms struggle to extract features from targets at different scales, resulting in significant differences in detection performance across scales, particularly for small targets where accuracy is low. Furthermore, model training relies on large amounts of data, and while commonly used data augmentation algorithms can significantly expand the data sample size, they do not adequately consider the diverse target requirements of millimeter-wave images in practical applications. Summary of the Invention

[0003] This disclosure aims to provide a deep learning-based method for detecting cloaked targets in millimeter-wave images, a deep learning-based device for detecting cloaked targets in millimeter-wave images, and a computer-readable storage medium, which effectively improves the model's generalization ability and its ability to detect targets at different scales, thus meeting the needs of practical applications.

[0004] According to one of the solutions disclosed herein, a deep learning-based method for detecting cloaked targets in millimeter-wave images is provided, comprising:

[0005] The acquired images are preprocessed, and the distribution patterns of the targets are statistically analyzed.

[0006] Based on the distribution pattern of the target, the target outline is segmented and then fused with the background sample set;

[0007] Detection of specific target objects based on object detection networks;

[0008] Based on the training of an improved target detection model, hidden targets in millimeter-wave images are detected.

[0009] In some embodiments, the preprocessing of the acquired images to statistically analyze the distribution patterns of the targets includes:

[0010] Construct a single-target sample set, while the total sample set contains multiple specific target objects;

[0011] Determine the location and type of the concealed target;

[0012] Assign different location category information to different categories;

[0013] Statistical analysis was performed on the labeled images to identify locations where hidden targets frequently appeared, and a distribution map was constructed.

[0014] In some embodiments, the process of segmenting the target contour and fusing it with the background sample set according to the target distribution pattern includes:

[0015] Read the sample set and the corresponding label set, read the images sequentially, and crop and save the images as the target sub-image sample set according to the label coordinates;

[0016] A portion of the samples in the target sub-image sample set is randomly selected to construct the training set of the target segmentation network, and the target segmentation network is trained to obtain the trained target segmentation network.

[0017] The target sub-image set is input for prediction, and based on the target contour mask image, an image set containing only the target contour is obtained;

[0018] Based on the distribution statistics chart, the area to be expanded is delineated for the background sample set;

[0019] The overlap ratio of the target bounding boxes in the image set containing only the target outline and the background sample set is obtained to obtain the fused sample set.

[0020] In some embodiments, the detection of a specific target object based on a target detection network includes:

[0021] Construct parallel convolutional branches to cover targets of large, medium and small sizes, forming convolutional blocks;

[0022] Construct three convolutional branches with identical structures, whose convolutional kernels are consistent with those in the original residual block;

[0023] Based on replacing the convolutional blocks, weight sharing of convolutional layers is set up so that convolutional branches can share weights for joint training, and convolutional branches can be automatically matched based on the target size.

[0024] In some embodiments, detecting cloaked targets in millimeter-wave images based on training an improved target detection model includes:

[0025] Construct a training sample set, which is taken from the sample set and the fused sample set and the corresponding label set, and divide it into training set, test set and validation set according to the proportion;

[0026] The detection of millimeter-wave cloaked targets is based at least on the configuration and adjustment of training parameters;

[0027] Untrained millimeter-wave images were randomly selected as test data, and a trained target detection model was loaded to detect millimeter-wave cloaked targets.

[0028] In some embodiments, the detection of cloaked targets in millimeter-wave images based on the training of an improved target detection model further includes:

[0029] The model output results are used to mark the location of hidden objects on the image for visualization and are generated into a file. The file content includes information on the type of hidden object and the coordinates of its carrier location.

[0030] In some embodiments, the detection of millimeter-wave cloaked targets, based at least on the configuration and adjustment of training parameters, includes:

[0031] Set the number of training target categories and the names of the target categories;

[0032] Feature extraction;

[0033] Input images sequentially into the backbone network and modify the network;

[0034] Candidate bounding boxes are generated, and the classification branch is used to filter the bounding boxes, while the regression branch is used to optimize them.

[0035] Define a loss function to monitor the network's learning process;

[0036] The candidate bounding boxes are accurately classified and located. The output candidate bounding box regions are used as input and processed by the region of interest pooling layer to obtain a fixed-size feature map. The bounding box category is predicted by the classification branch network and the bounding box localization prediction is achieved by the regression branch network.

[0037] The non-maximum suppression algorithm is used to filter out redundant anchor points generated by a target, while retaining the target bounding box.

[0038] Modify the training hyperparameters and start training until the preset number of rounds are repeated, and you will get the trained object detection network.

[0039] Detection of millimeter-wave concealed targets.

[0040] In some embodiments, Faster R-CNN is used as the base network for hidden object detection, wherein the backbone network is changed to ResNeXt 101, and the ResNeXt 101 convolutional modules are improved to enhance feature detection of targets at multiple scales.

[0041] According to one of the solutions disclosed herein, a deep learning-based millimeter-wave image concealed target detection device is provided, comprising:

[0042] The preprocessing module is configured to preprocess the acquired images and statistically analyze the distribution patterns of the targets.

[0043] The fusion module is configured to segment the target contour according to the target distribution pattern and then fuse it with the background sample set;

[0044] The first detection module is configured to detect specific target objects based on a target detection network.

[0045] The second detection module is configured to detect hidden targets in millimeter-wave images based on the training of an improved target detection model.

[0046] According to one of the solutions of this disclosure, a computer-readable storage medium is provided having computer-executable instructions stored thereon, which, when executed by a processor, implement:

[0047] Based on the above-mentioned deep learning-based method for cloaked target detection in millimeter-wave images.

[0048] The various embodiments of the deep learning-based millimeter-wave image cloaked target detection method, device, and computer-readable storage medium disclosed herein at least preprocess the acquired images to statistically analyze target distribution patterns; segment the target contours according to the target distribution patterns and fuse them with a background sample set; detect specific target objects based on a target detection network; and detect cloaked targets in millimeter-wave images based on the training of an improved target detection model. The aim is to provide a novel convolutional module to extract features of targets of different sizes, guide the matching of convolutional branches with different receptive fields for different images through target size distribution patterns, and improve model training efficiency; simultaneously, image fusion is performed based on distribution characteristics as prior information to generate new samples, enriching the number of samples and the diversity of targets, effectively improving the model's generalization ability and detection capability for targets of different scales, and meeting practical application needs. Taking full account of the features of millimeter-wave images and the distribution of hidden targets, the distribution characteristics of targets are first statistically analyzed. Based on the distribution characteristic map, image fusion is guided to accurately expand the diversity of targets. The convolutional module of the target detection network is redesigned, and the original convolutional block is replaced with three parallel convolutional branches, so that the features of large, medium and small targets can be fully learned, thereby improving the detection accuracy of hidden targets in millimeter-wave images.

[0049] It should be understood that the foregoing general description and the following detailed description are exemplary and illustrative only, and are not intended to limit the scope of this disclosure. Attached Figure Description

[0050] In drawings that are not necessarily drawn to scale, similar reference numerals in different views may indicate similar components. Similar reference numerals with letter suffixes or similar reference numerals with different letter suffixes may indicate different instances of similar components. The drawings are generally used to illustrate various embodiments by way of example rather than limitation, and are used together with the specification and claims to explain the disclosed embodiments.

[0051] Figure 1 A flowchart of a deep learning-based millimeter-wave image cloaked target detection method according to an embodiment of the present disclosure is shown;

[0052] Figure 2 A target distribution statistics chart of an embodiment of this disclosure is shown;

[0053] Figure 3 The following are flowcharts illustrating the image fusion process of various embodiments of this disclosure;

[0054] Figure 4 An improved backbone structure diagram according to an embodiment of this disclosure is shown;

[0055] Figure 5 A structural diagram of a single convolutional block according to an embodiment of this disclosure is shown;

[0056] Figure 6 A flowchart of the target detection process according to an embodiment of this disclosure is shown;

[0057] Figures 7(a) to 7(c) The results of target detection at different scales according to embodiments of this disclosure are shown;

[0058] Figure 8 An architecture diagram of a deep learning-based millimeter-wave image cloaked target detection device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0060] To protect public safety in densely populated areas, security checks on people coming and going are crucial. However, traditional methods such as security gates and cameras are no longer sufficient to meet the security needs of large crowds. Millimeter-wave security equipment stands out among many new security inspection devices due to its omnidirectional, penetrating, high-resolution imaging, and non-ionizing radiation characteristics.

[0061] Deep learning technology can be applied to the automatic detection of concealed targets in millimeter-wave images, improving security inspection efficiency. However, current target detection methods for millimeter-wave images do not perform well when faced with complex background information and diverse target scales, especially for small targets where the detection accuracy is poor.

[0062] In conjunction with the preceding background section, this disclosure provides illustrative examples of solutions to address the deficiencies in the prior art, but these are not intended to limit the scope of patent protection claimed in this disclosure.

[0063] As one of the solutions, such as Figure 1 As shown, embodiments of this disclosure provide a deep learning-based method for detecting cloaked targets in millimeter-wave images, including:

[0064] The acquired images are preprocessed, and the distribution patterns of the targets are statistically analyzed.

[0065] Based on the distribution pattern of the target, the target outline is segmented and then fused with the background sample set;

[0066] Detection of specific target objects based on object detection networks;

[0067] Based on the training of an improved target detection model, hidden targets in millimeter-wave images are detected.

[0068] To address the problems mentioned above, the embodiments of this disclosure propose a deep learning-based method for cloaked target detection in millimeter-wave images. This method fully considers the features of millimeter-wave images and the distribution of cloaked targets. First, it statistically analyzes the distribution characteristics of targets and uses this distribution characteristic map to guide image fusion, thereby accurately expanding the diversity of targets. The convolutional module of the target detection network is redesigned, replacing the original convolutional block with three parallel convolutional branches. This ensures that features of targets of all sizes are fully learned, improving the accuracy of cloaked target detection in millimeter-wave images. The convolutional modules involved in the embodiments of this disclosure extract features of targets of different sizes. The distribution patterns of target sizes guide the matching of convolutional branches with different receptive fields to different images, improving model training efficiency. Simultaneously, image fusion is performed using distribution characteristics as prior information to generate new samples, enriching the number of samples and the diversity of targets. This invention can effectively improve the model's generalization ability and its ability to detect targets at different scales, meeting the needs of practical applications.

[0069] For example, a specific detection process can be implemented using a technical solution represented by the following steps.

[0070] In some implementations, the method of this disclosure may include: preprocessing the acquired image and statistically analyzing the distribution patterns of the target, including:

[0071] Construct a single-target sample set, while the total sample set contains multiple specific target objects;

[0072] Determine the location and type of the concealed target;

[0073] Assign different location category information to different categories;

[0074] Statistical analysis was performed on the labeled images to identify locations where hidden targets frequently appeared, and a distribution map was constructed.

[0075] For example, the method in this embodiment may include:

[0076] Step S1: Preprocess the acquired images and statistically analyze the distribution patterns of the targets;

[0077] Step S11: Construct a single-target sample set X = {x1, x2, ..., x}i}(i < N), the total sample set contains multiple types of specific target objects, such as prohibited items, denoted as Y = {X1, X2, …, X j}(j < N);

[0078] Step S12: Determine the position and category of the hidden target, assign different tag names P = {p1, p2, …, pi}(i < N) to different categories, and use the rectangular box method for annotation. Its coordinate information is expressed as: tr: the vertical coordinate of the upper left corner; tl: the horizontal coordinate of the lower left corner; br: the horizontal coordinate of the upper right corner; bl: the vertical coordinate of the lower right corner. If the image does not contain any target, extract and construct the background sample set B = {b1, b2, …, b k}(k < N);

[0079] Step S13: Statistically analyze the annotated images to reveal the positions where hidden targets often appear; construct a distribution statistical chart T, and the process is as Figure 2 shown;

[0080] Generate a binary mask map M = {m1, m2, …, m j}(j < N) for the sample set Y

[0081]

[0082]

[0083] where M is a binary mask image obtained by superimposing the pixel values of N images. That is, for N randomly selected images, when the i-th image contains a target, the pixel values of the rectangular box are set to 1, and the pixel values of other areas are set to 0.

[0084] In some embodiments, the method of the present disclosure may be:

[0085] According to the target distribution law, segment the target contour and fuse it with the background sample set, including:

[0086] Read the sample set and the corresponding label set, read the images in sequence, and crop and save the images according to the label coordinates as the target sub-image sample set;

[0087] Randomly select some samples from the target sub-image sample set to construct the training set of the target segmentation network, and train the target segmentation network to obtain a trained target segmentation network;

[0088] Input the target sub-image set for prediction, and based on the target contour mask map, obtain an image set that only contains the target contour;

[0089] According to the distribution statistical chart, define the area to be expanded for the background sample set;

[0090] The overlap ratio of the target bounding boxes in the image set containing only the target outline and the background sample set is obtained to obtain the fused sample set.

[0091] For example, the method of this embodiment may further include the following based on step S1 described above:

[0092] Step S2: Based on statistical patterns, the target contour is precisely segmented and then fused with the background sample set B to form a new sample. The sample fusion flowchart is shown below. Figure 3 As shown;

[0093] Step S21: Read the sample set Y and the corresponding label set P, and read the images sequentially, where the j-th image y j Corresponding to the j-th label p j The image is cropped according to the label coordinates and saved as a target sub-image sample set A, A = {a1, a2, ..., a...} t}(t <N);

[0094] Step S22: Return a t area d t Where H represents a t The height, W represents the height of a t The width, where || is the modulo operator, is calculated as follows:

[0095] H=||tl-br||(3)

[0096] W = ||bl-tr|| 4)

[0097] d t =|H·W| (5)

[0098] In equation (5), "·" represents multiplication.

[0099] Step S23: Randomly select a portion of the samples in sample set A to construct the training set of the target segmentation network, train the target segmentation network, and follow the initial settings of the original network. After repeating the training for a preset number of rounds, the trained target segmentation network is obtained. The target segmentation network is a commonly used image segmentation network, but not limited to these networks.

[0100] Step S24: Input the target sub-image set A for prediction to obtain the target contour mask image. Further perform a mask operation to extract the target contour from A to obtain an image set A′={a′1,a′2,…,a′} containing only the target contour. t};

[0101] Step S25: Based on the statistical chart T obtained in step S13, delineate the region to be expanded for the background sample set B, represented by a rectangle, with label P=0, and save it as a label file with the same name; the number of times the fusion region participates in the fusion is controlled by the parameter α;

[0102] Step S26: Calculate a′ t and b k Target bounding box overlap ratio: q = Sa′ i ∪Sb j , when q=Sa′ i , for b k Perform a random rotation, the rotation angle of which is determined by β. If q > Sa′ i , for b k After scaling, the data is fed back into the network, and the scaling factor is determined by γ.

[0103] The above operations yield the fused sample set Y′={y′1,y′2,···y′}. m},y′ m The following calculations were performed:

[0104] y′ m =F[αb K +βa′ t (6)

[0105] Where F[·] represents the fusion method, and · refers to any quantity to explain the meaning of F. In this embodiment, the maximum value fusion method, minimum value fusion method, weighted average value fusion method, etc. can be selected.

[0106] In some implementations, the method of this disclosure embodiment may be:

[0107] Detection of specific target objects based on object detection networks, including:

[0108] Construct parallel convolutional branches to cover targets of large, medium and small sizes, forming convolutional blocks;

[0109] Construct three convolutional branches with identical structures, whose convolutional kernels are consistent with those in the original residual block;

[0110] Based on replacing the convolutional blocks, weight sharing of convolutional layers is set up so that convolutional branches can share weights for joint training, and convolutional branches can be automatically matched based on the target size.

[0111] For example, the method of this embodiment may further include the following steps based on the aforementioned step S2:

[0112] Step S3: Construct a target detection network for contraband detection. In this embodiment, Faster R-CNN is used as the base network for concealed item detection. The backbone network is changed to ResNeXt101, and the ResNeXt101 convolutional modules are improved to enhance feature detection of targets at multiple scales. The network structure is as follows: Figure 4 As shown, the specific improvement steps are as follows:

[0113] Step S31: Construct three parallel convolutional branches to cover targets of large, medium, and small sizes, forming a convolutional block. The structure diagram of the convolutional module is shown below. Figure 5 As shown;

[0114] Step S311: Construct three convolutional branches conv1, conv2, and conv3 with the same structure. Their convolutional kernels are consistent with the convolutional kernels in the original residual block. The dilation coefficients d are set to d1 = 1, d2 = 2, and d3 = 3, respectively.

[0115] Step S312: Construct the three branches in S311 into a convolutional block, and replace the last convolutional block of the original ResNeXt 101 with this convolutional block;

[0116] Step S32: For a in step S22 t Further analysis led to the development of a training strategy based on scale recommendations.

[0117] Step S321: Calculate the average size of the target bounding box. Indicates a t The average value is calculated as follows:

[0118]

[0119] Step S322: Calculate the image matching threshold of the convolution branch: This indicates the size of the region suggestion box, where w is the width of the region suggestion box and h is the width of the region suggestion box.

[0120] when The image is input to conv1,

[0121] when The image is input to conv2,

[0122] when The image is input to conv3,

[0123] By setting up weight sharing for convolutional layers, the three branches can share weights for joint training, and the convolutional branches can be automatically matched based on the target size.

[0124] In some implementations, the method of this disclosure embodiment may be:

[0125] Based on the training of an improved target detection model, the detection of cloaked targets in millimeter-wave images is performed, including:

[0126] Construct a training sample set, which is taken from the sample set and the fused sample set and the corresponding label set, and divide it into training set, test set and validation set according to the proportion;

[0127] The detection of millimeter-wave cloaked targets is based at least on the configuration and adjustment of training parameters;

[0128] Untrained millimeter-wave images were randomly selected as test data, and a trained target detection model was loaded to detect millimeter-wave cloaked targets.

[0129] In some implementations, the method of this disclosure may include: detecting cloaked targets in millimeter-wave images based on training an improved target detection model, and further includes:

[0130] The model output results are used to mark the location of hidden objects on the image for visualization and are generated into a file. The file content includes information on the type of hidden object and the coordinates of its carrier location.

[0131] In some implementations, the method of this disclosure can be: detecting millimeter-wave cloaked targets based at least on the configuration and adjustment of training parameters, including:

[0132] Set the number of training target categories and the names of the target categories;

[0133] Feature extraction;

[0134] Input images sequentially into the backbone network and modify the network;

[0135] Candidate bounding boxes are generated, and the classification branch is used to filter the bounding boxes, while the regression branch is used to optimize them.

[0136] Define a loss function to monitor the network's learning process;

[0137] The candidate bounding boxes are accurately classified and located. The output candidate bounding box regions are used as input and processed by the region of interest pooling layer to obtain a fixed-size feature map. The bounding box category is predicted by the classification branch network and the bounding box localization prediction is achieved by the regression branch network.

[0138] The non-maximum suppression algorithm is used to filter out redundant anchor points generated by a target, while retaining the target bounding box.

[0139] Modify the training hyperparameters and start training until the preset number of rounds are repeated, and you will get the trained object detection network.

[0140] Detection of millimeter-wave concealed targets.

[0141] For example, the method of this embodiment may further include the following based on the aforementioned step S3:

[0142] Step S4: Train the improved target detection model to detect occult targets in millimeter-wave images. The target detection flowchart is as follows: Figure 6 As shown;

[0143] Step S41: Construct a training sample set U = {u1, u2, ..., u} i}, and its corresponding label set is P = {p1, p2, ..., p} i Specifically, U and P are taken from Y and Y′ and their corresponding label sets, and are divided into training set, test set and validation set according to proportion;

[0144] Step S42: Set the number of training target classes (nc) and the target class names (classes);

[0145] Step S43: Feature Extraction:

[0146] The feature extraction structures of the various embodiments of this disclosure are improved based on ResNeXt101 with parallel branch convolutional structures to generate feature maps of different scales. The target feature maps are input into the RPN network to filter and optimize candidate bounding boxes to obtain a set of candidate bounding boxes containing the target. Finally, the filtered set of candidate bounding boxes is used for target classification and bounding box regression. The Faster Rcnn network structure is used to output the target category and accurate bounding box coordinates.

[0147] Step S431: Input the images into the backbone network sequentially, and modify the network to the improved ResNeXt101 network of the embodiments of this disclosure;

[0148] The improved ResNeXt101 network first distributes the input to C channels, performing transformations in each channel. Each channel consists of 1×1, 3×3, and 1×1 convolutional kernels, respectively. The last convolutional block is changed to a three-branch structure. Finally, the results from all branches are fused together, and a residual connection is established. The feature output is:

[0149]

[0150] Where y is the output of this module, u is the input image, and T is the input image. m (u) represents the output of each path;

[0151] Step S432: Input the RPN network to generate candidate bounding boxes. The scale factor and aspect ratio are set to the default values ​​{1, 0.5, 2} and {1:1, 2:1, 1:2}, respectively. This will generate 9 default candidate bounding boxes at any point. The classification branch performs bounding box selection, and the regression branch performs bounding box optimization. b and Let (x, y, h, w) represent the predicted bounding box and the regression target corresponding to the ground truth bounding box, respectively. Let (x, y, h, w) be the predicted bounding box. a y a h a w a ) is denoted as the candidate border, (xg y g h g w g If ) is denoted as the actual border, then

[0152]

[0153]

[0154] Step S433: Define the loss function, supervise the learning process of the network, calculate the model loss by comparing the difference between the prediction results of the neural network and the labels, so that the model can better fit the data;

[0155] The classification loss function is defined as:

[0156]

[0157] Where p b c represents the bounding box classification probability. b Indicates border category label This represents the predicted probability value corresponding to the true category label (target or background).

[0158] The bounding box regression loss function is defined as:

[0159]

[0160] in,

[0161]

[0162] The network training loss function can be obtained as follows:

[0163]

[0164] Where λ is a hyperparameter, [c b ≥1] means when c b The value is 1 if it is greater than or equal to 1, otherwise it is 0.

[0165] Step S44: Accurately classify and locate the candidate bounding boxes. The candidate bounding box region output by the RPN network is used as input and processed by the region of interest pooling layer to obtain a fixed-size feature map. The network predicts the bounding box category and bounding box coordinate prediction. The classification branch network realizes the bounding box category prediction, and the regression branch network realizes the bounding box location prediction.

[0166] Step S45: Use the Non-Maximum Suppression (NMS) algorithm to filter out redundant anchor boxes generated by a target, leaving only one target box. The NMS process is as follows:

[0167] Step S451: Obtain s candidate boxes b through the classifier. s(s≥1), the probability value of belonging to this category is p. s ;

[0168] Step S452: Press p on the s candidate boxes s Sort from largest to smallest, and select p. s Candidate box b for the maximum value max Iterate through the remaining candidate boxes b in sequence s ;

[0169] Step S453: Set the IOU threshold to ε, and calculate b max With b s IOU:

[0170]

[0171] Among them, b max ∩b s b max and b s The intersection of the areas of the target boxes, b max ∩b s b max and b s The union of the areas of the target boxes;

[0172] Step S454: If IOU(b) max ,b s )>ε, b s The corresponding p s Set to 0;

[0173] Step S455: Repeat steps S452, S453 and S454 until only one candidate box remains;

[0174] Step S46: Modify the training hyperparameters, set learning_rate, batch_size, and max_iters, keep other parameters at the network default values, start training, repeat for a preset number of rounds, and obtain the trained object detection network;

[0175] Step S47: Detect millimeter-wave cloaked targets:

[0176] Step S471: Randomly select untrained millimeter-wave images as test data, and load the trained target detection model to detect millimeter-wave cloaked targets;

[0177] Step S472: Mark the location of the hidden object on the image using the model output results for visualization and generate a file containing information such as the type of hidden object and the coordinates of the human body where it is located.

[0178] The improved algorithms in the embodiments of this disclosure provide visualization results for concealment detection in millimeter-wave images, as shown below. Figures 7(a) to 7(c) As shown, it can be observed that the detection performance is good for targets of all sizes. For example, Figure 7(a) shows a large-scale target, category: p5, precision: 0.96; Figure 7(b) shows a medium-scale target, category: p5, precision: 0.86; Figure 7(c) shows a small-scale target, category: p1, precision: 0.85.

[0179] As one of the solutions, such as Figure 8 As shown, embodiments of this disclosure provide a deep learning-based millimeter-wave image cloaked target detection device, comprising:

[0180] The preprocessing module is configured to preprocess the acquired images and statistically analyze the distribution patterns of the targets.

[0181] The fusion module is configured to segment the target contour according to the target distribution pattern and then fuse it with the background sample set;

[0182] The first detection module is configured to detect specific target objects based on a target detection network.

[0183] The second detection module is configured to detect hidden targets in millimeter-wave images based on the training of an improved target detection model.

[0184] As one implementation method, the deep learning-based millimeter-wave image concealed target detection device disclosed herein can be further configured, in conjunction with the steps described above, that the preprocessing module is:

[0185] The acquired images are preprocessed to statistically analyze the distribution patterns of the targets, including:

[0186] Construct a single-target sample set, while the total sample set contains multiple specific target objects;

[0187] Determine the location and type of the concealed target;

[0188] Assign different location category information to different categories;

[0189] Statistical analysis was performed on the labeled images to identify locations where hidden targets frequently appeared, and a distribution map was constructed.

[0190] As one implementation method, the deep learning-based millimeter-wave image concealed target detection device disclosed herein can be further configured, in conjunction with the steps described above, that the fusion module is:

[0191] Based on the distribution pattern of the target, the target contour is segmented and then fused with the background sample set, including:

[0192] Read the sample set and the corresponding label set, read the images sequentially, and crop and save the images as the target sub-image sample set according to the label coordinates;

[0193] A portion of the samples in the target sub-image sample set is randomly selected to construct the training set of the target segmentation network, and the target segmentation network is trained to obtain the trained target segmentation network.

[0194] The target sub-image set is input for prediction, and based on the target contour mask image, an image set containing only the target contour is obtained;

[0195] Based on the distribution statistics chart, the area to be expanded is delineated for the background sample set;

[0196] The overlap ratio of the target bounding boxes in the image set containing only the target outline and the background sample set is obtained to obtain the fused sample set.

[0197] As one implementation method, the deep learning-based millimeter-wave image concealed target detection device disclosed herein can be further configured, in conjunction with the steps described above, with the first detection module being:

[0198] Detection of specific target objects based on object detection networks, including:

[0199] Construct parallel convolutional branches to cover targets of large, medium and small sizes, forming convolutional blocks;

[0200] Construct three convolutional branches with identical structures, whose convolutional kernels are consistent with those in the original residual block;

[0201] Based on replacing the convolutional blocks, weight sharing of convolutional layers is set up so that convolutional branches can share weights for joint training, and convolutional branches can be automatically matched based on the target size.

[0202] As one implementation method, the deep learning-based millimeter-wave image concealed target detection device disclosed herein can be further configured, in conjunction with the steps described above, as follows:

[0203] Construct a training sample set, which is taken from the sample set and the fused sample set and the corresponding label set, and divide it into training set, test set and validation set according to the proportion;

[0204] The detection of millimeter-wave cloaked targets is based at least on the configuration and adjustment of training parameters;

[0205] Untrained millimeter-wave images were randomly selected as test data, and a trained target detection model was loaded to detect millimeter-wave cloaked targets.

[0206] Specifically, one of the inventive concepts disclosed herein aims to, through the aforementioned deep learning-based millimeter-wave image occult target detection method, deep learning-based millimeter-wave image occult target detection device, and computer-readable storage medium, at least preprocess the acquired images to statistically analyze target distribution patterns; based on these distribution patterns, segment the target contours and fuse them with a background sample set; detect specific target objects based on a target detection network; and detect occult targets in millimeter-wave images based on the training of an improved target detection model. This aims to provide a novel convolutional module to extract features from targets of different sizes, guiding the matching of convolutional branches with different receptive fields for different images based on target size distribution patterns, thereby improving model training efficiency. Simultaneously, image fusion is performed using distribution characteristics as prior information to generate new samples, enriching the number of samples and the diversity of targets, effectively improving the model's generalization ability and detection capability for targets of different scales, and meeting practical application needs. Taking full account of the features of millimeter-wave images and the distribution of hidden targets, the distribution characteristics of targets are first statistically analyzed. Based on the distribution characteristic map, image fusion is guided to accurately expand the diversity of targets. The convolutional module of the target detection network is redesigned, and the original convolutional block is replaced with three parallel convolutional branches, so that the features of large, medium and small targets can be fully learned, thereby improving the detection accuracy of hidden targets in millimeter-wave images.

[0207] The beneficial effects of the embodiments disclosed herein are at least reflected in:

[0208] 1) The convolutional structure of the backbone network of the target detection network was improved. Based on the target scale, three parallel convolutional branches with different dilation rates were designed to extract target features, ensuring that convolutional branches with different receptive fields are matched for large, medium and small targets, which effectively improves the detection accuracy of the model for targets of different sizes.

[0209] 2) Considering the requirement of massive data for model training, this disclosure presents an image fusion method based on target distribution statistics, which effectively expands the number of sample sets and the diversity of targets, enabling the model to better learn the different features of targets and enhance the model's generalization ability and robustness.

[0210] This disclosure also provides a computer-readable storage medium storing computer-executable instructions thereon, which, when executed by a processor, mainly implement the above-described deep learning-based millimeter-wave image concealed target detection method, including at least:

[0211] The acquired images are preprocessed, and the distribution patterns of the targets are statistically analyzed.

[0212] Based on the distribution pattern of the target, the target outline is segmented and then fused with the background sample set;

[0213] Detection of specific target objects based on object detection networks;

[0214] Based on the training of an improved target detection model, hidden targets in millimeter-wave images are detected.

[0215] The above embodiments are merely exemplary embodiments of this disclosure and are not intended to limit this disclosure. The scope of protection of this disclosure is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this disclosure within its substance and scope, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this disclosure.

Claims

1. A deep learning-based method for cloaked target detection in millimeter-wave images, including: The acquired images are preprocessed to statistically analyze the distribution patterns of targets, including: constructing a single-target sample set, with the total sample set containing multiple types of specific target objects; determining the location and category of concealed targets; assigning different location category information to different categories; statistically analyzing the labeled images to obtain the locations where concealed targets appear frequently, and constructing a distribution chart; wherein: specific target objects include concealed targets; Based on the target distribution pattern, the target contour is segmented and then fused with the background sample set. This includes: reading the sample set and the corresponding label set; sequentially reading the images; cropping the images according to the label coordinates and saving them as a target sub-image sample set; randomly selecting a portion of the samples in the target sub-image sample set to construct a training set for the target segmentation network; training the target segmentation network to obtain the trained target segmentation network; inputting the target sub-image set for prediction; and obtaining an image set containing only the target contour based on the target contour mask image; delineating the region to be expanded for the background sample set according to the distribution statistics; and obtaining the overlap ratio of the target bounding boxes between the image set containing only the target contour and the background sample set to obtain the fused sample set. Detection of specific target objects based on object detection networks; Based on the training of an improved target detection model, hidden targets in millimeter-wave images are detected. Specifically: Faster R-CNN was used as the baseline network for hidden object detection, and the backbone network was changed to ResNeXt 101; three parallel convolutional branches were constructed to cover large, medium and small targets, forming a convolutional block; the three branches were combined into a convolutional block, and this convolutional block replaced the last convolutional block of the original ResNeXt 101.

2. The method according to claim 1, wherein, Detection of specific target objects based on object detection networks, including: Based on replacing the convolutional blocks, weight sharing of convolutional layers is set up so that convolutional branches can share weights for joint training, and convolutional branches can be automatically matched based on the target size.

3. The method according to claim 2, wherein, Based on the training of an improved target detection model, the detection of hidden targets in millimeter-wave images includes: constructing a training sample set, which is taken from the sample set, the fused sample set, and the corresponding label set, and dividing them into a training set, a test set, and a validation set according to a certain ratio; detecting hidden targets in millimeter-wave images based at least on the configuration and adjustment of training parameters; randomly selecting untrained millimeter-wave images as test data, and loading the trained target detection model to detect hidden targets in millimeter-wave images.

4. The method according to claim 3, wherein, Based on the training of an improved target detection model, the detection of occult targets in millimeter-wave images also includes: The model output results are used to mark the location of hidden objects on the image for visualization and are generated into a file. The file content includes information on the type of hidden object and the coordinates of its carrier location.

5. The method according to claim 4, wherein, Detection of millimeter-wave cloaked targets, at least based on the configuration and adjustment of training parameters, including: Set the number of training target categories and the names of the target categories; Feature extraction; The images are sequentially input into the backbone network; Candidate bounding boxes are generated, and the classification branch is used to filter the bounding boxes, while the regression branch is used to optimize them. Define a loss function to monitor the network's learning process; The candidate bounding boxes are accurately classified and located. The output candidate bounding box regions are used as input and processed by the region of interest pooling layer to obtain a fixed-size feature map. The bounding box category is predicted by the classification branch network and the bounding box localization prediction is achieved by the regression branch network. The non-maximum suppression algorithm is used to filter out redundant anchor points generated by a target, while retaining the target bounding box. Modify the training hyperparameters and start training until the preset number of rounds are repeated, and you will get the trained object detection network. Detection of millimeter-wave concealed targets.

6. A deep learning-based millimeter-wave image concealed target detection device, employing the method described in claim 1; the device comprises: The preprocessing module is configured to preprocess the acquired images and statistically analyze the distribution patterns of the targets. The fusion module is configured to segment the target contour according to the target distribution pattern and then fuse it with the background sample set; The first detection module is configured to detect specific target objects based on a target detection network. The second detection module is configured to detect hidden targets in millimeter-wave images based on the training of an improved target detection model.

7. A computer-readable storage medium having stored thereon computer-executable instructions, which, when executed by a processor, implement: The method for detecting hidden targets in millimeter-wave images based on deep learning according to any one of claims 1 to 5.

Images