A millimeter wave image target detection method and system

Abstract

The application discloses a kind of millimeter wave image target detection method and system, belong to active millimeter wave image processing technical field, comprising the following steps: S1: label active millimeter wave image;S2: train semantic segmentation model;S3: train target detection model;S4: real-time picture is detected and enhanced display to target.Semantic segmentation model based on deep learning is used to the original image is segmented, and the image after segmentation is sent into target detection model based on deep learning simultaneously with original image to train, to improve the detection rate of target detection, reduce false alarm probability, worthy of being popularized and used.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of active millimeter wave image processing technology, and particularly relates to a millimeter wave image target detection method and system. BACKGROUND

[0002] The application of human security inspection equipment based on millimeter wave imaging technology in the field of public security is in a period of vigorous development. In order to develop in a more civilized, efficient, intelligent and safe direction, the human security inspection equipment needs to have the function of target detection.

[0003] Security inspection data is often detected from different spatial environments, and the data often has complex noise, and the signal-to-noise ratio of security inspection image data is low. In addition, the active millimeter wave security inspection instrument will produce inconsistent characteristics for different targets due to the differences in gender, body shape, height, and weight of the person. The electromagnetic characteristics of different clothes, the texture of clothes, wrinkles and other factors will greatly affect the imaging quality of the millimeter wave human security inspection instrument. Clothes, contraband and human body surface are closely attached and often cannot be effectively distinguished in distance. Clothes are attached to the human body and constantly change with the posture of the human body. The dielectric properties of different clothing materials differ. In actual processing of clothing, it is often treated as interference and detection clutter. In addition, the frequency source of the active security inspection instrument and the system calibration will have system errors with the long-time operation of the equipment. At the same time, the imaging quality of the security inspection instrument is greatly related to the antenna cover and the surrounding environmental interference. The above factors greatly increase the difficulty of target detection of the security inspection image of the active millimeter wave human security inspection instrument.

[0004] In order to improve the customer experience of human security inspection products, it is necessary to continuously improve the detection rate of target detection and reduce the false alarm probability. Therefore, a millimeter wave image target detection method is proposed. SUMMARY

[0005] The technical problem to be solved by the present application is how to improve the target detection rate of active millimeter wave images and reduce the false alarm probability, and a millimeter wave image target detection method is provided.

[0006] The present application solves the above technical problems by the following technical solutions. The present application comprises the following steps:

[0007] S1: Labeling active millimeter wave images

[0008] The active millimeter-wave images are labeled in two ways: semantic segmentation labeling and object detection labeling. Semantic segmentation labels the foreground region, background region, and target region to obtain a classification label image. The foreground region includes the human body region but does not include the target region. The background region includes the region other than the human body and the clutter around the human body. Object detection labeling uses rectangular boxes to mark the target rectangle and the human body rectangle to obtain a label image used for object detection.

[0009] S2: Training the semantic segmentation model

[0010] Pixel-level classification labels are obtained using semantic segmentation annotation. A semantic segmentation deep learning model is trained, and the loss on the training set and validation set is observed. After the training stopping condition is met, the training is completed, and the semantic segmentation model is obtained.

[0011] S3: Training the object detection model

[0012] Input the original image and the pixel-level classification label image after semantic segmentation, train the object detection deep learning model, observe the loss on the training set and validation set, and complete the training after the training stopping condition is met to obtain the object detection model.

[0013] S4: Perform object detection and enhance the display of real-time images.

[0014] The active millimeter-wave real-time image is first segmented using a semantic segmentation model. Both the segmented image and the original image are then input into a target detection model to obtain the final pixel prediction result. The bounding boxes of the prediction result are then superimposed onto the active millimeter-wave image to achieve target detection in the active millimeter-wave image.

[0015] Furthermore, the active millimeter-wave image is a grayscale image.

[0016] Furthermore, in step S1, the pixel-level classification labels obtained by semantic segmentation annotation have three values: 0, 1, and 2, where 0 represents background, 1 represents human body, and 2 represents target.

[0017] Furthermore, in step S1, the labels obtained from target detection annotation have three types of values: 0, 1, and 2, where 0 represents the human body, 1 represents the target box, and 2 represents other objects.

[0018] Furthermore, in step S2, the semantic segmentation deep learning model adopts a semantic segmentation model based on a U-shaped network.

[0019] Furthermore, in step S3, the object detection deep learning model adopts an object detection model based on the improved YOLO series.

[0020] Further, in the step S3, the original picture is instructed by the classification label image after semantic segmentation, and the distinction of the human body, the background and the target is completed.

[0021] Further, in the step S4, the classification probability value output by the semantic segmentation model is multiplied by the pixel value of the original image to obtain the pixel value of the display image.

[0022] The application also provides a millimeter wave image target detection system, which adopts the above target detection method to perform target detection on an active millimeter wave image, and comprises:

[0023] A labeling module is configured to label the active millimeter wave image, and the labeling is divided into two types, namely semantic segmentation labeling and target detection labeling, wherein the semantic segmentation labeling outputs a foreground region, a background region and a target region to obtain a classification label image, the foreground region includes a human body region and does not include a target region part, and the background region includes a region other than the human body and a clutter part around the human body; the target detection labeling labels a target rectangular frame and a human body rectangular frame in the form of a rectangular frame to obtain a label image.

[0024] A semantic segmentation model training module is configured to obtain a pixel-level classification label by using the semantic segmentation labeling, train a semantic segmentation deep learning model, observe the loss of a training set and a validation set, complete the training when a training stop condition is reached, and obtain a semantic segmentation model.

[0025] A target detection model training module is configured to input an original image and a pixel-level classification label image after semantic segmentation, train a target detection deep learning model, observe the loss of a training set and a validation set, complete the training when a training stop condition is reached, and obtain a target detection model.

[0026] A target detection display module is configured to first segment an active millimeter wave real-time image by using a semantic segmentation model, input the segmented image and an original image into a target detection model to obtain a final pixel prediction result, superimpose a rectangular frame of the prediction result on the active millimeter wave image, and realize target detection on the active millimeter wave image.

[0027] A central processing module is configured to issue instructions to each module and complete relevant actions.

[0028] The labeling module, the semantic segmentation model training module, the target detection model training module and the target detection display module are electrically connected with the central processing module.

[0029] Compared with the prior art, the millimeter wave image target detection method has the following advantages: the millimeter wave image target detection method utilizes a semantic segmentation model based on deep learning to segment an original image, and simultaneously inputs the segmented image and the original image into a target detection model based on deep learning for training, thereby improving the detection rate of target detection and reducing the false alarm probability, and is worthy of promotion and use. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of the millimeter wave image target detection method in the embodiment one of the present application;

[0031] Figure 2 is a working flowchart of the Faster R-CNN semantic segmentation model network structure in the embodiment one of the present application;

[0032] Figure 3 a is a network algorithm flowchart based on the improved yolo V3 target detection in the embodiment one of the present application;

[0033] Figure 3 b is Figure 3 a schematic diagram of the spp module structure in a;

[0034] Figure 4 is a cylindrical surface millimeter wave product semantic segmentation result graph in the embodiment two of the present application;

[0035] Figure 5 is a cylindrical surface millimeter wave product segmentation clutter target detection comprehensive result graph in the embodiment one of the present application. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be described in detail below, and the embodiments are implemented on the premise of the technical scheme of the present application, and detailed implementation modes and specific operation processes are given, but the protection scope of the present application is not limited to the following embodiments.

[0037] Embodiment one

[0038] The embodiment provides a technical scheme: a millimeter wave image target detection method, comprising the following steps:

[0039] S1: labeling active millimeter wave images

[0040] The active millimeter wave images are labeled, and two models of semantic segmentation and target detection are included in the method, and two kinds of labeling of the active millimeter wave images need to be completed. Among them, the semantic segmentation labels the foreground region, the background region and the target region to obtain a classification label image, the foreground region includes the human body region but does not include the target region part, and the background region includes the region outside the human body and the clutter part around the human body; the target detection labels the target rectangular frame and the human body rectangular frame in the form of a rectangular frame;

[0041] S2: training of a semantic segmentation model and obtaining the model

[0042] The semantic segmentation deep learning model is trained using the semantic segmentation pixel-level classification label, the training set and the validation set loss are observed, and after the training stop condition is reached, the training is completed to obtain the semantic segmentation model;

[0043] S3: training of a target detection model and obtaining the model

[0044] The training input of the target detection model includes the original image and the classification label image after semantic segmentation, and the target detection deep learning model is trained, the training set and the validation set loss are observed, and after the training stop condition is reached, the training is completed to obtain the target detection model;

[0045] S4: target detection and enhanced display of real-time images

[0046] The active millimeter wave real-time image is subjected to target detection according to the steps of S2 and S3, the image after image segmentation and the original image are input into the target detection model to obtain the final pixel prediction result, and then the rectangular frame of the prediction result is superimposed on the active millimeter wave image to realize target detection of the active millimeter wave image.

[0047] The active millimeter wave image is a gray image.

[0048] In the step S1, the pixel-level classification label obtained by semantic segmentation annotation has three values of 0, 1 and 2, wherein 0 represents background, 1 represents human body, and 2 represents target.

[0049] In the step S1, the label image obtained by target detection annotation has three values of 0, 1 and 2, wherein 0 represents human body, 1 represents a target frame, and 2 represents other objects.

[0050] In the step S2, the semantic segmentation model uses a semantic segmentation model based on a U-shaped network, as shown in Figure 2 The working flow chart of the network structure of the Faster R-CNN semantic segmentation model in this embodiment is shown.

[0051] A mask prediction branch parallel to the target fine classification and regression network is introduced on the Faster R-CNN network structure. The mask branch is a full convolutional neural network for each region of interest, which predicts the pixel semantic information of the region of interest in a pixel-to-pixel manner. First, the feature extraction network extracts multi-scale features of the image. Then, the region proposal network extracts candidate regions that may contain contraband on different convolution layers at the same time. Finally, the fine categories of these candidate regions are predicted, the positions of the contraband are regressed, and the pixel categories of the candidate regions are predicted.

[0052] In the step S3, the picture input by the target detection model includes the original picture and the classification label image after the semantic segmentation, and the classification label image after the semantic segmentation is used to train the original picture to distinguish the human body, the background and the target.

[0053] In the step S3, the target detection model adopts the target detection model based on the improved yolo series, such as Figure 3 a, which is a network algorithm flowchart based on the improved yolo V3 target detection in the embodiment.

[0054] The original picture and the picture after the segmentation are simultaneously input into the input end of the detection network of the deep learning. The effective sample label is used to make the gaussian attention mask label, which uses the parameterized one-dimensional gaussian filter to create the soft attention mask of the image size. By adding the attention supervision branch, the attention weight with the same size as the extracted feature is obtained, and then the attention weight is multiplied with the extracted feature to emphasize the spatial position information and improve the network model performance. In the training process, the smooth L1 function can be selected for the loss of the attention module to accelerate the convergence process, enhance the robustness of the network and avoid gradient explosion.

[0055] In the step S4, the classification probability value output by the semantic segmentation model is multiplied by the pixel value of the original image to obtain the display image pixel value. The classification probability value of the semantic segmentation model is multiplied by the pixel value of the original image, S=I*P (on each pixel unit, I represents the pixel value of the original image (0-255), P is the corresponding probability value of the image classification (0-1), and S represents the final display pixel value, which can be rounded).

[0056] In the step S4, the pixel value of the rectangular frame output by the target detection model is directly superimposed on the original image for display, and finally the picture of the target detection result display is obtained.

[0057] The embodiment also provides a millimeter wave image target detection system, which comprises:

[0058] The labeling module is divided into a semantic segmentation labeling module and a target detection labeling module. The semantic segmentation labeling module is used for labeling the foreground region, the background region and the target region in the active millimeter wave image to obtain a classification label image. The target detection labeling module is used for labeling the human body, the target and other objects in the active millimeter wave image to obtain a label image.

[0059] The semantic segmentation model training module is used for training the model by using the pixel-level label, observing the loss of the training set and the validation set, completing the training after the training stop condition is reached, and saving and obtaining the semantic segmentation model.

[0060] The target detection model training module inputs the semantic segmentation label and the original image into a target detection model for training, uses the label generated by the target detection labeling module, combines a loss function, trains a preselected pixel prediction deep learning model, observes the loss of the training set and the validation set, and completes training after reaching a training stop condition, saves and obtains a target detection model;

[0061] The target detection display module inputs the active millimeter wave image into the target detection model to obtain a pixel prediction result, then superimposes the input active millimeter wave image, and realizes target display on the active millimeter wave image.

[0062] The control processing module is configured to issue instructions to each module to complete relevant actions.

[0063] The labeling module, the semantic segmentation model training module, the target detection model training module, and the target detection display module are connected to the control processing module.

[0064] Embodiment Two

[0065] The embodiment provides a technical solution for millimeter wave real-time image target detection in an active cylindrical millimeter wave product, including the following steps:

[0066] The labeling module includes a semantic segmentation labeling module and a target detection labeling module. The semantic segmentation labeling module is configured to label foreground regions, background regions, and target regions in the active millimeter wave image to obtain a classification label image. The target detection labeling module is configured to label human bodies, targets, and other objects in the active millimeter wave image to obtain a label image.

[0067] The semantic segmentation model training module is configured to train a model using pixel-level labels, observe the loss of the training set and the validation set, complete training after reaching a training stop condition, save and obtain a semantic segmentation model. Figure 4 An effect diagram after semantic segmentation model training is displayed.

[0068] The target detection model training module inputs the semantic segmentation label and the original image into a target detection model for training, uses the label generated by the target detection labeling module, combines a loss function, trains a preselected pixel prediction deep learning model, observes the loss of the training set and the validation set, and completes training after reaching a training stop condition, saves and obtains a target detection model.

[0069] The target detection display module inputs the active millimeter wave image into the target detection model to obtain a pixel prediction result, then superimposes the input active millimeter wave image, and realizes target display on the active millimeter wave image. Figure 5 An active millimeter wave final frame selection result diagram is displayed.

[0070] The control processing module is used for issuing instructions to each module to complete relevant actions.

[0071] The labeling module, the semantic segmentation model training module, the target detection model training module, and the target detection display module are connected with the control processing module.

[0072] In summary, the millimeter wave image target detection method system of the above embodiments uses a deep learning semantic segmentation model to segment the original image, and simultaneously sends the segmented image and the original image into a deep learning target detection model for training, thereby improving the detection rate of target detection and reducing the false alarm probability, and is worthy of popularization and use.

[0073] Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A millimeter-wave image target detection method, characterized in that, The method comprises the following steps: S1: labeling the active millimeter wave image Labeling in the active millimeter wave image, labeling is divided into two kinds, which are semantic segmentation labeling and target detection labeling, wherein the semantic segmentation labeling labels foreground region, background region and target region to obtain a classification label image, the foreground region includes human body region and does not include target region part, the background region includes regions other than human body and clutter part around human body; the target detection labeling labels target rectangular frame and human body rectangular frame in the form of rectangular frame to obtain a label image for target detection; S2: training a semantic segmentation model Using the pixel-level classification label obtained by semantic segmentation labeling, a semantic segmentation deep learning model is trained, and the training set and validation set losses are observed until the training stopping condition is reached, and the training is completed to obtain the semantic segmentation model; S3: training a target detection model The original image and the pixel-level classification label image obtained by semantic segmentation are input into the target detection deep learning model for training, and the training set and validation set losses are observed until the training stopping condition is reached, and the training is completed to obtain the target detection model; S4: target detection and enhanced display of real-time pictures The active millimeter wave real-time image is first segmented by the semantic segmentation model, and the segmented image and the original image are input into the target detection model to obtain the final pixel prediction result, and the rectangular frame of the prediction result is superimposed on the active millimeter wave image to realize target detection of the active millimeter wave image; In the step S1, the pixel-level classification label obtained by semantic segmentation labeling has three values of 0, 1 and 2, wherein 0 represents background, 1 represents human body, and 2 represents target; In the step S1, the label image obtained by target detection labeling has three values of 0, 1 and 2, wherein 0 represents human body, 1 represents target frame, and 2 represents other objects; In the step S2, the semantic segmentation deep learning model adopts a semantic segmentation model based on a U-shaped network; In the step S3, the target detection deep learning model adopts a target detection model based on an improved yolo series. 2.The millimeter wave image target detection method of claim 1, wherein: The active millimeter wave image is a gray image. 3.The method of claim 1, wherein: In the step S3, the classification label image after semantic segmentation is used to train the original image to distinguish human body, background and target.

4. The millimeter wave image target detection method of claim 3, wherein: In the step S4, the classification probability value output by the semantic segmentation model is multiplied by the pixel value of the original image to obtain the pixel value of the display image.

5. A millimeter wave image target detection system, characterized by, The target detection method according to any one of claims 1-4 is used for target detection of the active millimeter wave image, comprising: a labeling module configured to label in the active millimeter wave image, labeling is divided into two kinds, which are semantic segmentation labeling and target detection labeling, wherein the semantic segmentation labeling labels foreground region, background region and target region to obtain a classification label image, the foreground region includes human body region and does not include target region part, the background region includes regions other than human body and clutter part around human body; the target detection labeling labels target rectangular frame and human body rectangular frame in the form of rectangular frame to obtain a label image for target detection; The semantic segmentation model training module is configured to obtain pixel-level classification labels by using semantic segmentation labeling, train a semantic segmentation deep learning model, observe losses of a training set and a validation set, complete training after a training stop condition is reached, and obtain a semantic segmentation model; The target detection model training module is configured to input original images and pixel-level classification label images obtained by semantic segmentation, train a target detection deep learning model, observe losses of a training set and a validation set, complete training after a training stop condition is reached, and obtain a target detection model; The target detection display module is configured to first segment active millimeter wave real-time images by using the semantic segmentation model, input the segmented images and the original images into the target detection model, obtain final pixel prediction results, and superimpose the prediction results on the active millimeter wave images to achieve target detection of the active millimeter wave images. The central processing module is configured to issue instructions to each module and complete related actions. The labeling module, the semantic segmentation model training module, the target detection model training module, and the target detection display module are electrically connected to the central processing module.

Images