Millimeter wave radar security inspection instrument equipment imaging and model training method, device, equipment and medium

Abstract

The embodiment of the invention provides millimeter wave radar security inspection instrument equipment imaging and model training methods, devices, equipment and a medium. An AI imaging network is trained based on sample millimeter wave original data obtained by scanning a first sample detected object by a millimeter wave radar and sample radar three-dimensional voxel data corresponding to the sample millimeter wave original data; training the AI enhancement network based on the sample prediction radar three-dimensional voxel data output by the AI imaging network and the sample foreign matter labeling enhancement image of the first sample detected object; and subsequently, the AI imaging network and the AI enhancement network are trained by using sample millimeter wave original data obtained by scanning a second sample detected object through the millimeter wave radar and a sample foreign matter labeling enhancement image of the second sample detected object, and a trained millimeter wave radar security inspection instrument equipment imaging model is obtained. By adopting a training method of grading pre-training and integral fine tuning, the network training difficulty is reduced, the network convergence is accelerated, and the training resource consumption is reduced.

Description

Technical Field

[0001] The present application relates to the field of millimeter-wave radar security inspection equipment imaging technology, and in particular to millimeter-wave radar security inspection equipment imaging and model training methods, devices, equipment and media. Background Art

[0002] With the rapid development of internet technology, detection equipment is becoming increasingly intelligent. Millimeter-wave detection equipment is currently a common method of detecting objects, leveraging the penetrating properties of millimeter waves. However, the raw millimeter-wave data collected by these devices cannot be directly used for object detection. Instead, the raw millimeter-wave data must be processed using Fourier transform to generate millimeter-wave images, which can then be used for object detection. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method, device, equipment and medium for imaging and model training of millimeter-wave radar security inspection equipment, so as to obtain intelligent enhanced images based on millimeter-wave raw data, thereby facilitating subsequent object detection.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for training an imaging model of a millimeter-wave radar security inspection device, the method comprising:

[0006] Obtaining first sample millimeter-wave raw data obtained by scanning a first sample inspected object by a millimeter-wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter-wave raw data, and a millimeter-wave radar security inspection device imaging model to be trained, wherein the first sample radar three-dimensional voxel data is used to represent a millimeter-wave imaging three-dimensional contour of the first sample inspected object, and the millimeter-wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0007] Training the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as labels;

[0008] Acquire a first sample foreign body annotated enhanced image of the first sample inspected object and first sample predicted radar three-dimensional voxel data output by the AI ​​imaging network;

[0009] Training the AI ​​enhanced network using the first sample predicted radar three-dimensional voxel data as input and the first sample foreign object annotated enhanced image as a label;

[0010] Acquire second sample millimeter wave raw data obtained by scanning a second sample inspected object by a millimeter wave radar and a second sample foreign object annotated enhanced image corresponding to the second sample inspected object;

[0011] The second sample millimeter wave original data is used as input and the second sample foreign object annotated enhanced image is used as a label, and the AI ​​imaging network and the AI ​​enhanced network are trained to obtain a trained millimeter wave radar security inspection equipment imaging model.

[0012] In a possible embodiment, the training of the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as a label includes:

[0013] Inputting the first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data;

[0014] determining a loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data;

[0015] Parameters of the AI ​​imaging network are adjusted according to the loss of the AI ​​imaging network.

[0016] In a possible embodiment, the AI ​​imaging network is a fully convolutional network including k convolutional layers. Inputting the first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data includes:

[0017] Performing a convolution operation on the first sample millimeter wave raw data using a first convolution layer;

[0018] The t-th convolutional layer is used to perform a convolution operation on the features output by the t-1-th convolutional layer to obtain the first sample predicted radar 3D voxel data, where t∈[2,K].

[0019] In a possible embodiment, determining the loss of the AI ​​imaging network according to the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data includes:

[0020] The loss of the AI ​​imaging network is determined based on the first sample predicted radar 3D voxel data and the first sample radar 3D voxel data using the following formula:

[0021]

[0022] Among them, L 2_loss is the loss of the AI ​​imaging network, y iis the first sample radar 3D voxel data of the i-th, x i is the first sample predicted radar 3D voxel data corresponding to the i-th first sample millimeter wave original data, and M is the number of the first sample predicted radar 3D voxel data.

[0023] In a possible embodiment, the training of the AI ​​enhanced network using the first sample predicted radar three-dimensional voxel data as input and the first sample foreign object annotated enhanced image as a label includes:

[0024] Inputting the first sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a first sample predicted foreign object annotated enhanced image, wherein the first sample foreign object annotated enhanced image is millimeter wave imaging data of the first sample inspected object annotated with a foreign object, and the foreign object is annotated with a color that is different from the first sample inspected object;

[0025] Determining the loss of the AI ​​enhanced network based on the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image;

[0026] Adjusting parameters of the AI ​​enhanced network according to the loss of the AI ​​enhanced network.

[0027] In a possible embodiment, determining the loss of the AI ​​enhanced network according to the first sample predicted foreign object annotated enhanced image and the first sample foreign object annotated enhanced image includes:

[0028] Determining the L1 loss and the structural similarity SSIM loss of the AI ​​enhanced network according to the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image;

[0029] Determine a loss of the AI ​​enhanced network based on the L1 loss and the SSIM loss of the AI ​​enhanced network.

[0030] In a possible embodiment, the method of using the second sample millimeter wave raw data as input and the second sample foreign object annotated enhanced image as a label to train the AI ​​imaging network and the AI ​​enhanced network to obtain a trained millimeter wave radar security inspection device imaging model includes:

[0031] Inputting the second sample millimeter wave raw data into the AI ​​imaging network to obtain second sample predicted radar three-dimensional voxel data;

[0032] Inputting the second sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a second sample predicted foreign object annotated enhanced image, where the second sample foreign object annotated enhanced image is millimeter wave imaging data of the second sample inspected object annotated with a foreign object, where the foreign object is annotated with a color that is different from the second sample inspected object;

[0033] Determine the loss of the imaging model of the millimeter-wave radar security inspection device according to the second sample predicted foreign object annotated enhanced image and the second sample foreign object annotated enhanced image;

[0034] Adjust the parameters of at least one of the AI ​​imaging network and the AI ​​enhancement network according to the loss of the imaging model of the millimeter wave radar security inspection device.

[0035] In a second aspect, an embodiment of the present application provides a millimeter wave radar security inspection device imaging method, the method comprising:

[0036] Obtaining unprocessed millimeter wave raw data obtained by scanning the target object by the millimeter wave radar, and a pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0037] Inputting the to-be-processed millimeter-wave raw data into an AI imaging network to obtain predicted radar three-dimensional voxel data, where the predicted radar three-dimensional voxel data is used to represent the millimeter-wave imaging three-dimensional contour of the target object;

[0038] The predicted radar three-dimensional voxel data is input into the AI ​​enhancement network to obtain an intelligent enhanced image.

[0039] In one possible embodiment, the AI ​​enhanced network includes an image segmentation subnetwork and an attention mechanism subnetwork;

[0040] Inputting the predicted radar three-dimensional voxel data into an AI enhancement network to obtain an intelligent enhanced image includes:

[0041] Inputting the predicted radar 3D voxel data into an image segmentation subnetwork to obtain a first segmentation feature map;

[0042] Inputting the first segmentation feature map into the attention mechanism sub-network to obtain a second segmentation feature map;

[0043] The predicted radar three-dimensional voxel data is labeled using the second segmentation feature map to obtain an intelligent enhanced image.

[0044] In a possible embodiment, the AI ​​imaging network is a fully convolutional network including multiple convolutional layers, the AI ​​enhanced network includes a U-Net subnetwork and a convolutional attention mechanism subnetwork CBAM, the U-Net subnetwork includes n downsampling layers and n upsampling layers corresponding to the downsampling layers, wherein the i-th downsampling layer is jump-connected to the n-i+1-th upsampling layer.

[0045] In a possible embodiment, the attention mechanism sub-network includes a channel attention module and a spatial attention module, and inputting the first segmentation feature map into the attention mechanism sub-network to obtain the second segmentation feature map includes:

[0046] Inputting the first segmentation feature map into the channel attention module, and bitwise multiplying the first segmentation feature map by the channel attention parameter to obtain a first segmentation feature map after channel attention correction;

[0047] The first segmentation feature map after the channel attention correction is input into the spatial attention module, and the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map.

[0048] In a possible embodiment, the channel attention module includes a first pooling submodule and a fully connected neural network submodule, the fully connected neural network submodule includes a fully connected neural network, the first segmentation feature map is input into the channel attention module, the first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction, including:

[0049] Inputting the first segmentation feature map into the first pooling submodule, performing a maximum pooling operation and an average pooling operation on the first segmentation feature map to obtain a first maximum pooling feature and a first average pooling feature;

[0050] Inputting the first maximum pooling feature and the first average pooling feature into a fully connected network to obtain a channel attention parameter;

[0051] The first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction.

[0052] In a possible embodiment, the spatial attention module includes a second pooling submodule and a convolutional neural network submodule, the convolutional neural network submodule includes a convolutional neural network, the first segmentation feature map after the channel attention correction is input into the spatial attention module, the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map, including:

[0053] Inputting the first segmentation feature map after the channel attention correction into the second pooling submodule, performing a maximum pooling operation and an average pooling operation on the first segmentation feature map after the channel attention correction to obtain a second maximum pooling feature and a second average pooling feature;

[0054] Inputting the second maximum pooling feature and the second average pooling feature into a convolutional neural network to obtain a spatial attention parameter;

[0055] The spatial attention parameter is bitwise multiplied by the first segmentation feature map corrected by the channel attention to obtain a second segmentation feature map.

[0056] In a third aspect, an embodiment of the present application provides a millimeter-wave radar security inspection device imaging model training device, the device comprising:

[0057] A second acquisition module is configured to acquire first sample millimeter-wave raw data obtained by scanning a first sample inspected object by a millimeter-wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter-wave raw data, and a millimeter-wave radar security inspection device imaging model to be trained, wherein the first sample radar three-dimensional voxel data is used to represent the millimeter-wave imaging three-dimensional contour of the first sample inspected object, and the millimeter-wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0058] a first training module, configured to train the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as labels;

[0059] a third acquisition module, configured to acquire a first sample foreign body annotated enhanced image of the first sample inspected object and first sample predicted radar 3D voxel data output by the AI ​​imaging network;

[0060] a second training module, configured to train the AI ​​enhanced network using the first sample predicted radar 3D voxel data as input and the first sample foreign object annotated enhanced image as a label;

[0061] A fourth acquisition module is configured to acquire second sample millimeter wave raw data obtained by scanning a second sample inspected object by a millimeter wave radar and a second sample foreign body annotated enhanced image corresponding to the second sample inspected object;

[0062] The third training module is used to take the second sample millimeter wave original data as input and the second sample foreign object annotated enhanced image as a label, train the AI ​​imaging network and the AI ​​enhanced network, and obtain the trained millimeter wave radar security inspection equipment imaging model.

[0063] In a possible embodiment, the first training module is configured to input the first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data;

[0064] determining a loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data;

[0065] Parameters of the AI ​​imaging network are adjusted according to the loss of the AI ​​imaging network.

[0066] In a possible embodiment, the AI ​​imaging network is a fully convolutional network, which includes k convolutional layers, and the first training module is configured to perform a convolution operation on the first sample millimeter wave raw data using the first convolutional layer;

[0067] The t-th convolutional layer is used to perform a convolution operation on the features output by the t-1-th convolutional layer to obtain the first sample predicted radar 3D voxel data, where t∈[2,K].

[0068] In a possible embodiment, the first training module is configured to determine the loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data using the following formula:

[0069]

[0070] Among them, L 2_loss is the loss of the AI ​​imaging network, y i is the first sample radar 3D voxel data of the i-th, x i is the first sample predicted radar 3D voxel data corresponding to the i-th first sample millimeter wave original data, and M is the number of the first sample predicted radar 3D voxel data.

[0071] In a possible embodiment, the second training module is configured to input the first sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a first sample predicted foreign object annotated enhanced image, wherein the first sample foreign object annotated enhanced image is millimeter wave imaging data of the first sample inspected object annotated with a foreign object, and the foreign object is annotated with a color that is different from the first sample inspected object;

[0072] Determining the loss of the AI ​​enhanced network based on the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image;

[0073] Adjusting parameters of the AI ​​enhanced network according to the loss of the AI ​​enhanced network.

[0074] In a possible embodiment, the second training module is configured to determine the L1 loss and the structural similarity SSIM loss of the AI ​​enhanced network based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image;

[0075] Determine a loss of the AI ​​enhanced network based on the L1 loss and the SSIM loss of the AI ​​enhanced network.

[0076] In a possible embodiment, the third training module is configured to input the second sample millimeter wave raw data into the AI ​​imaging network to obtain second sample predicted radar three-dimensional voxel data;

[0077] Inputting the second sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a second sample predicted foreign object annotated enhanced image, where the second sample foreign object annotated enhanced image is millimeter wave imaging data of the second sample inspected object annotated with a foreign object, where the foreign object is annotated with a color that is different from the second sample inspected object;

[0078] Determine the loss of the imaging model of the millimeter-wave radar security inspection device according to the second sample predicted foreign object annotated enhanced image and the second sample foreign object annotated enhanced image;

[0079] Adjust the parameters of at least one of the AI ​​imaging network and the AI ​​enhancement network according to the loss of the imaging model of the millimeter wave radar security inspection device.

[0080] In a fourth aspect, an embodiment of the present application provides an imaging device for a millimeter-wave radar security inspection device, the device comprising:

[0081] A first acquisition module is used to obtain unprocessed millimeter wave raw data obtained by the millimeter wave radar scanning the target object, and a pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0082] A first determination module is configured to input the to-be-processed millimeter-wave raw data into an AI imaging network to obtain predicted radar three-dimensional voxel data, wherein the predicted radar three-dimensional voxel data is used to represent a millimeter-wave imaging three-dimensional contour of the target object;

[0083] The second determination module is used to input the predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain an intelligent enhanced image.

[0084] In one possible embodiment, the AI ​​enhanced network includes an image segmentation subnetwork and an attention mechanism subnetwork;

[0085] The second determining module is configured to input the predicted radar three-dimensional voxel data into an image segmentation subnetwork to obtain a first segmentation feature map;

[0086] Inputting the first segmentation feature map into the attention mechanism sub-network to obtain a second segmentation feature map;

[0087] The predicted radar three-dimensional voxel data is labeled using the second segmentation feature map to obtain an intelligent enhanced image.

[0088] In a possible embodiment, the AI ​​imaging network is a fully convolutional network including multiple convolutional layers, the AI ​​enhanced network includes a U-Net subnetwork and a convolutional attention mechanism subnetwork CBAM, the U-Net subnetwork includes n downsampling layers and n upsampling layers corresponding to the downsampling layers, wherein the i-th downsampling layer is jump-connected to the n-i+1-th upsampling layer.

[0089] In a possible embodiment, the attention mechanism subnetwork includes a channel attention module and a spatial attention module, and the second determination module is configured to input the first segmentation feature map into the channel attention module, and bitwise multiply the first segmentation feature map by the channel attention parameter to obtain the first segmentation feature map after channel attention correction;

[0090] The first segmentation feature map after the channel attention correction is input into the spatial attention module, and the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map.

[0091] In a possible embodiment, the channel attention module includes a first pooling submodule and a fully connected neural network submodule, the fully connected neural network submodule includes a fully connected neural network, and the second determination module is used to input the first segmentation feature map into the first pooling submodule, perform a maximum pooling operation and an average pooling operation on the first segmentation feature map, and obtain a first maximum pooling feature and a first average pooling feature;

[0092] Inputting the first maximum pooling feature and the first average pooling feature into a fully connected network to obtain a channel attention parameter;

[0093] The first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction.

[0094] In a possible embodiment, the spatial attention module includes a second pooling submodule and a convolutional neural network submodule, the convolutional neural network submodule includes a convolutional neural network, and the second determination module is used to input the first segmentation feature map after the channel attention correction into the second pooling submodule, and perform a maximum pooling operation and an average pooling operation on the first segmentation feature map after the channel attention correction to obtain a second maximum pooling feature and a second average pooling feature;

[0095] Inputting the second maximum pooling feature and the second average pooling feature into a convolutional neural network to obtain a spatial attention parameter;

[0096] The spatial attention parameter is bitwise multiplied by the first segmentation feature map corrected by the channel attention to obtain a second segmentation feature map.

[0097] In a fifth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;

[0098] The memory is used to store computer programs;

[0099] The processor is used to implement the millimeter wave radar security inspection device imaging method or millimeter wave radar security inspection device imaging model training method described in this application when executing the program stored in the memory.

[0100] In the sixth aspect, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the millimeter-wave radar security inspection device imaging method or the millimeter-wave radar security inspection device imaging model training method described in this application.

[0101] Beneficial effects of the embodiments of the present application:

[0102] The millimeter-wave radar security inspection equipment imaging and model training method, apparatus, equipment and medium provided in the embodiments of the present application use the first-sample millimeter-wave raw data obtained by the millimeter-wave radar scanning the first-sample inspected object as input and the first-sample radar three-dimensional voxel data corresponding to the first-sample millimeter-wave raw data as labels to train the AI ​​imaging network; use the first-sample predicted radar three-dimensional voxel data output by the AI ​​imaging network as input and the first-sample foreign object annotated enhanced image of the first-sample inspected object as labels to train the AI ​​enhanced network; use the second-sample millimeter-wave raw data obtained by the millimeter-wave radar scanning the second-sample inspected object as input and the second-sample foreign object annotated enhanced image corresponding to the second-sample inspected object as labels to train the AI ​​imaging network and the AI ​​enhanced network to obtain a trained millimeter-wave radar security inspection equipment imaging model. The embodiment of the present application adopts an end-to-end neural network training method based on millimeter-wave radar security inspection equipment. By adopting a hierarchical pre-training + overall fine-tuning training method, the network training difficulty is reduced, the network convergence is accelerated, and the training resource consumption is reduced; in addition, an AI imaging network is used to replace the traditional Fourier transform algorithm, and the millimeter-wave radar millimeter-wave raw data is used as input to give full play to the advantages of the neural network, which can improve the imaging performance and simplify the processing process; the AI ​​enhancement network is used to project and enhance the three-dimensional voxel data after imaging to obtain an enhanced image, which can improve the foreign object detection performance and improve the reliability of security inspection.

[0103] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0105] Figure 1 A schematic diagram of a millimeter-wave radar security inspection device imaging method provided in an embodiment of the present application;

[0106] Figure 2 A schematic diagram of inputting raw millimeter-wave data to be processed into an AI imaging network to obtain predicted radar three-dimensional voxel data, provided in an embodiment of the present application;

[0107] Figure 3 A schematic diagram of an intelligently enhanced image provided in an embodiment of the present application;

[0108] Figure 4A schematic diagram of inputting predicted radar 3D voxel data into an AI enhancement network to obtain an intelligently enhanced image, provided in an embodiment of the present application;

[0109] Figure 5 A schematic diagram of inputting a first segmentation feature map into an attention mechanism sub-network to obtain a second segmentation feature map provided in an embodiment of the present application;

[0110] Figure 6 A schematic diagram of inputting a first segmentation feature map into a channel attention module to obtain channel attention parameters provided in an embodiment of the present application;

[0111] Figure 7 A schematic diagram of inputting the first segmentation feature map after channel attention correction into the spatial attention module to obtain spatial attention parameters provided in an embodiment of the present application;

[0112] Figure 8 A flowchart of the millimeter wave radar security inspection equipment imaging model training method provided for the implementation of this application;

[0113] Figure 9 Another flowchart of the method for training an imaging model for a millimeter-wave radar security inspection device provided in an embodiment of the present application;

[0114] Figure 10 A schematic diagram of an imaging device of a millimeter-wave radar security inspection device provided in an embodiment of the present application;

[0115] Figure 11 A flow chart of the imaging model training device for millimeter-wave radar security inspection equipment provided in an embodiment of the present application;

[0116] Figure 12 This is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0117] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0118] The terms used in the embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "an", "the" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0119] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the associated objects.

[0120] Before explaining in detail the millimeter-wave radar security inspection device imaging and model training method provided in the embodiment of the present application, the application scenarios of the millimeter-wave radar security inspection device imaging and model training method provided in the embodiment of the present application are first introduced.

[0121] The millimeter-wave radar security inspection device imaging and model training method provided in the embodiments of the present application can be applied in security inspection, for example, in application scenarios where millimeter-wave radar security inspection device is used to detect the inspected object.

[0122] In the prior art, when obtaining the three-dimensional raw data corresponding to the inspected object obtained by the millimeter-wave radar security inspection device through scanning the inspected object, the obtained three-dimensional raw data is usually converted into three-dimensional point cloud data through Fourier transform, and then target recognition is performed from the three-dimensional point cloud data; or a target detection network is used to perform target recognition. However, when using three-dimensional point cloud data to complete target recognition, the imaging quality of the millimeter-wave image obtained by this method is poor. Moreover, since the processing process of the millimeter-wave raw data is not end-to-end, the advantages of the neural network cannot be fully utilized, resulting in a high system complexity. The imaging and model training method of the millimeter-wave radar security inspection device provided in the embodiment of the present application reduces the system complexity by using a convolutional neural network instead of the traditional imaging algorithm (Fourier transform algorithm). It is an end-to-end processing and fully utilizes the advantages of the neural network's learnability. In addition, the AI ​​enhancement network including the image segmentation subnetwork and the attention mechanism subnetwork is used to enhance the three-dimensional point cloud data for foreign objects, and obtains an enhanced imaging result (intelligent enhanced image) after foreign object enhancement, which improves the imaging quality and foreign object enhancement quality, thereby improving the reliability of security inspection.

[0123] Next, the implementation environment involved in the embodiments of the present application is introduced.

[0124] The implementation environment of the embodiments of the present application may be a millimeter-wave radar security inspection device, which may be composed of a transmitter, a receiver, and a processor. The transmitter is a device that generates millimeter waves. It emits a beam of high-frequency, low-power millimeter waves and scans the inspected object as it passes. The receiver receives and records the millimeter-wave signal reflected by the inspected object. The processor analyzes and processes the received millimeter-wave signal to perform foreign object detection on the raw millimeter-wave data obtained from the millimeter-wave radar security inspection device.

[0125] Those skilled in the art should understand that the above-mentioned millimeter-wave radar security inspection equipment is only an example. Other existing or future millimeter-wave radar security inspection equipment, if applicable to this application, should also be included in the scope of protection of this application and included here by reference.

[0126] After introducing the implementation environment provided by the embodiment of the present application, the imaging method of the millimeter wave radar security inspection device provided by the embodiment of the present application is explained in detail.

[0127] See also Figure 1 , Figure 1 This is a flowchart of a millimeter wave radar security inspection device imaging method provided in an embodiment of the present application. The method can be applied to the above-mentioned millimeter wave radar security inspection device. The method may include the following steps:

[0128] S101, obtaining the unprocessed millimeter wave raw data obtained by the millimeter wave radar scanning the target object, and a pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an AI (Artificial Intelligence) imaging network and an AI enhanced network.

[0129] It should be noted that the millimeter-wave raw data to be processed is the millimeter-wave signal emitted by the millimeter-wave radar security inspection device to the target object to be inspected, which is reflected by the surface of the target object to be inspected and then returned to the receiver; the receiver converts the received signal into an electrical signal, and the image data is generated after processing by the signal processing system.

[0130] S102: Input the millimeter wave raw data to be processed into the AI ​​imaging network to obtain predicted radar three-dimensional voxel data.

[0131] It should be noted that the predicted radar 3D voxel data is used to represent the millimeter-wave imaging 3D contour of the target object. 3D voxel data is the smallest unit of digital data segmented in 3D space. It is a 3D concept and is commonly used in 3D imaging, scientific data, medical imaging and other fields.

[0132] In one possible embodiment, the AI ​​imaging network can be a fully convolutional network including multiple convolutional layers. For example, the fully convolutional network includes k convolutional layers. The process of using the fully convolutional network to process the millimeter wave raw data to be processed is as follows: using the first convolutional layer to perform a convolution operation on the millimeter wave raw data to be processed; using the tth convolutional layer to perform a convolution operation on the features output by the t-1th convolutional layer to obtain predicted radar three-dimensional voxel data, where t∈[2,K], K is a positive integer greater than or equal to 2. For an example, see Figure 2 , Figure 2This is a schematic diagram provided by an embodiment of the present application of inputting the millimeter wave raw data to be processed into the AI ​​imaging network to obtain predicted radar three-dimensional voxel data. The fully convolutional network includes 5 convolutional layers. The process of using the fully convolutional network to process the millimeter wave raw data to be processed is: using the first convolutional layer to perform a convolution operation on the millimeter wave raw data to be processed; using the tth convolutional layer to perform a convolution operation on the features output by the t-1th convolutional layer to obtain predicted radar three-dimensional voxel data, where t∈[2,5].

[0133] In the embodiment of the present application, by using a fully convolutional network to obtain the predicted radar three-dimensional voxel data corresponding to the millimeter-wave raw data to be processed, the advantage of the neural network's learnability is fully utilized. By optimizing the network loss function, it can be jointly trained with the AI ​​enhanced network to obtain better radar three-dimensional voxel data. The AI ​​imaging network is composed of a fully convolutional network with a small number of parameters and simple implementation. After hierarchical training, the parameters can be migrated to other networks without retraining. And because convolution calculation is used instead of traditional imaging algorithms (for example, fast Fourier transform algorithm FFT or inverse fast Fourier transform IFFT), the complexity of foreign object detection is reduced.

[0134] S103: Input the predicted radar 3D voxel data into the AI ​​enhancement network to obtain an intelligent enhanced image.

[0135] It should be noted that the intelligent enhanced image is the millimeter wave imaging data of the target object to be inspected that is marked with foreign matter, and the foreign matter is marked with a color that is different from the target object to be inspected. For example, see Figure 3 , Figure 3 This is a schematic diagram of an intelligent enhanced image provided in an embodiment of the present application, in which foreign matter is marked with a color that distinguishes it from the target object being inspected.

[0136] In one possible embodiment, the AI ​​enhanced network may include an image segmentation subnetwork and an attention mechanism subnetwork. For example, the AI ​​enhanced network includes a U-Net subnetwork and a convolutional attention mechanism subnetwork CBAM.

[0137] It should be noted that the U-Net network structure significantly improves image segmentation accuracy by introducing skip connections. The U-Net network mainly consists of n downsampling layers and n upsampling layers corresponding to the downsampling layers, where the i-th downsampling layer is skip-connected to the n-i+1-th upsampling layer. The image is downsampled for feature extraction and then upsampled to output an image of the corresponding size. The U-Net network structure is prior art and will not be described in detail in this application.

[0138] The Convolutional Block Attention Module (CBAM) is a lightweight, universal module that can be seamlessly integrated into any convolutional neural network architecture with negligible overhead and can be trained end-to-end alongside convolutional neural networks. Integrating CBAM into various models on various classification and detection datasets consistently improved model performance, demonstrating its broad applicability.

[0139] In one possible embodiment, see Figure 4 , Figure 4 A schematic diagram of inputting predicted radar 3D voxel data into an AI enhancement network to obtain an intelligently enhanced image is provided in an embodiment of the present application. Inputting predicted radar 3D voxel data into an AI enhancement network to obtain an intelligently enhanced image includes:

[0140] The predicted radar 3D voxel data is input into the image segmentation subnetwork to obtain the first segmentation feature map; the first segmentation feature map is input into the attention mechanism subnetwork to obtain the second segmentation feature map; the predicted radar 3D voxel data is labeled using the second segmentation feature map to obtain an intelligent enhanced image.

[0141] In one possible embodiment, see Figure 5 , Figure 5 The embodiment of the present application provides a schematic diagram of inputting a first segmentation feature map into an attention mechanism sub-network to obtain a second segmentation feature map, wherein the attention mechanism sub-network includes a channel attention module and a spatial attention module, and inputting the first segmentation feature map into the attention mechanism sub-network to obtain a second segmentation feature map, including:

[0142] The first segmentation feature map is input into the channel attention module, and the first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction; the first segmentation feature map after channel attention correction is input into the spatial attention module, and the first segmentation feature map after channel attention correction is bitwise multiplied by the spatial attention parameter to obtain the second segmentation feature map.

[0143] In a possible embodiment, the channel attention module includes a first pooling submodule and a fully connected neural network submodule, the fully connected neural network submodule includes a fully connected neural network, the first segmentation feature map is input into the channel attention module, the first segmentation feature map is bitwise multiplied by the channel attention parameter, and the first segmentation feature map after channel attention correction is obtained, including:

[0144] Input the first segmentation feature map into the first pooling submodule, perform the maximum pooling operation and the average pooling operation on the first segmentation feature map to obtain the first maximum pooling feature and the first average pooling feature; input the first maximum pooling feature and the first average pooling feature into the fully connected network to obtain the channel attention parameter; multiply the first segmentation feature map and the channel attention parameter bit by bit to obtain the first segmentation feature map after the channel attention correction. For an example, see Figure 6 , Figure 6 A schematic diagram of inputting the first segmentation feature map into the channel attention module to obtain channel attention parameters is provided in an embodiment of the present application.

[0145] For example, assuming that the dimension of the first segmentation feature map is C*H*W, C represents the channel, H represents the height, and W represents the width, the process of multiplying the first segmentation feature map by the channel attention parameter bit by bit to obtain the first segmentation feature map after the channel attention correction can be: performing the maximum pooling operation and the average pooling operation on each H*W first segmentation feature map in the first segmentation feature map F with the dimension of C*H*W, and obtaining the first maximum pooling feature map and the first average pooling feature The first maximum pooling feature F max And the first average pooling feature F avg Input into the fully connected network to obtain the channel attention parameter M c (F); Set the channel attention parameter M c (F) is bitwise multiplied with the first segmentation feature map F to obtain the first segmentation feature map F′ after channel attention correction.

[0146] As an example, the channel attention parameter M c (F) is bitwise multiplied with the first segmentation feature map F to obtain the first segmentation feature map F′ after channel attention correction, including:

[0147] Based on the channel attention parameter M c (F) and the first segmentation feature map F, use the following formula to obtain the first segmentation feature map F′ after channel attention correction:

[0148]

[0149] Among them, F′ is the first segmentation feature map after channel attention correction, M c (F) is the channel attention parameter, F is the first segmentation feature map, is bitwise multiplication.

[0150] In one possible embodiment, the first maximum pooling feature and the first average pooling feature are input into a fully connected network to obtain a channel attention parameter, including: maxAnd the first average pooling feature F avg Input into the fully connected network to get the first maximum pooling parameter and the first average pooling parameter The first maximum pooling parameter and the first average pooling parameter Add them together to get the channel attention parameter M c (F).

[0151] As an example, the first maximum pooling parameter and the first average pooling parameter are added to obtain the channel attention parameter, including:

[0152] Based on the first maximum pooling parameter and the first average pooling parameter, the channel attention parameter is obtained using the following formula:

[0153]

[0154] Among them, M c (F) is the channel attention parameter, is the first maximum pooling parameter, is the first average pooling parameter.

[0155] In a possible embodiment, the fully connected network is a fully connected network with a single hidden layer.

[0156] In a possible embodiment, the network parameter of the hidden layer in the fully connected network is W0, the activation function σ0 of the hidden layer in the fully connected network is the Relu activation function, the network parameter of the output layer in the fully connected network is W1, and the activation function σ1 of the output layer in the fully connected network is the Sigmoid activation function.

[0157] Accordingly, based on the first maximum pooling feature, determining the first maximum pooling parameter includes:

[0158] Based on the first maximum pooling feature, the first maximum pooling parameter is determined using the following formula:

[0159]

[0160] in, is the first maximum pooling parameter, σ1 is the Sigmoid activation function of the output layer in the fully connected network, W1 is the network parameter of the output layer in the fully connected network, σ0 is the Relu activation function of the hidden layer in the fully connected network, W0 is the network parameter of the hidden layer in the fully connected network, F max is the first maximum pooling eigenvalue.

[0161] Determining the first average pooling parameter based on the first average pooling feature value includes:

[0162] Based on the first average pooling feature value, the first average pooling parameter is determined using the following formula:

[0163]

[0164] in, is the first average pooling parameter, σ1 is the Sigmoid activation function of the output layer in the fully connected network, W1 is the network parameter of the output layer in the fully connected network, σ0 is the Relu activation function of the hidden layer in the fully connected network, W0 is the network parameter of the hidden layer in the fully connected network, F avg is the first average pooling eigenvalue.

[0165] In a possible embodiment, the input layer of the fully connected network has 512 neurons, the hidden layer of the fully connected network has 16 neurons, and the output layer of the fully connected network has 512 neurons.

[0166] In a possible embodiment, the spatial attention module includes a second pooling submodule and a convolutional neural network submodule, the convolutional neural network submodule includes a convolutional neural network, the first segmentation feature map after the channel attention correction is input into the spatial attention module, the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map, including:

[0167] The first segmentation feature map after channel attention correction is input into the second pooling submodule, and the first segmentation feature map after channel attention correction is subjected to maximum pooling and average pooling operations to obtain the second maximum pooling feature and the second average pooling feature; the second maximum pooling feature and the second average pooling feature are input into the convolutional neural network to obtain the spatial attention parameter; the spatial attention parameter is bitwise multiplied with the first segmentation feature map after channel attention correction to obtain the second segmentation feature map. For an example, see Figure 7 , Figure 7 The embodiment of the present application provides a schematic diagram of inputting the first segmentation feature map after channel attention correction into the spatial attention module to obtain spatial attention parameters.

[0168] For example, the process of multiplying the first segmentation feature map after channel attention correction with the spatial attention parameter bit by bit to obtain the second segmentation feature map can be: performing maximum pooling and average pooling operations along the C dimension on the first segmentation feature map F′ after channel attention correction with a dimension of C*H*W to obtain the second maximum pooling feature map and the second average pooling feature The second maximum pooling feature F′ max and the second average pooling feature F′ avg Splice along the C dimension and take the second largest pooling feature F′ after splicing maxand the second average pooling feature F′ avg Input into the convolutional neural network to obtain the spatial attention parameter M s (F′); the spatial attention parameter M s (F′) is bitwise multiplied with the first segmentation feature map F′ after channel attention correction to obtain the second segmentation feature map F″.

[0169] As an example, the spatial attention parameter M s (F′) is bitwise multiplied with the first segmentation feature map F′ after channel attention correction to obtain the second segmentation feature map F″, including:

[0170] Based on the spatial attention parameter M s (F′) and the first segmentation feature map F′ after channel attention correction, use the following formula to obtain the second segmentation feature map F″:

[0171]

[0172] Among them, F″ is the second segmentation feature map, M s (F′) is the spatial attention parameter, F′ is the first segmentation feature map after channel attention correction, is bitwise multiplication.

[0173] In a possible embodiment, the convolution layer in the convolutional neural network is a 7*7 convolution kernel, and the activation function σ of the output layer in the convolutional neural network is a Sigmoid activation function.

[0174] Accordingly, the second maximum pooling feature and the second average pooling feature are spliced ​​along the C dimension, and the spliced ​​second maximum pooling feature and the second average pooling feature are input into the convolutional neural network to obtain the spatial attention parameters, including:

[0175] Based on the second maximum pooling feature and the second average pooling feature, the spatial attention parameter is determined using the following formula:

[0176] M s (F′)=σ(f 7*7 ([F′ max ; F′ avg ]))

[0177] Among them, M s (F′) is the spatial attention parameter, σ is the Sigmoid activation function of the convolutional neural network, and f 7*7 For 7*7 convolution processing, F′ max is the second largest pooling feature, F′ avg is the second average pooling feature.

[0178] In a possible embodiment, the convolutional neural network is a single-layer convolutional neural network.

[0179] In the embodiments of the present application, an AI-enhanced network comprising an image segmentation subnetwork and an attention mechanism subnetwork is used to enhance foreign objects in three-dimensional point cloud data. This results in an enhanced imaging result (intelligently enhanced image) after foreign object enhancement, improving both imaging quality and foreign object enhancement quality. Furthermore, the intelligently enhanced image obtained by the imaging method for millimeter-wave radar security inspection equipment provided in the embodiments of the present application allows users to visually identify foreign objects, thereby improving the reliability of security inspections.

[0180] The technical solution provided in the embodiments of this application obtains unprocessed millimeter-wave raw data obtained by scanning a target object with a millimeter-wave radar, along with a pre-trained imaging model for the millimeter-wave radar security inspection device, which includes an AI imaging network and an AI enhancement network. The unprocessed millimeter-wave raw data is input into the AI ​​imaging network to obtain predicted three-dimensional radar voxel data; the predicted three-dimensional radar voxel data is then input into the AI ​​enhancement network to obtain an intelligently enhanced image. The AI ​​imaging network and the AI ​​enhancement network are used to perform foreign object enhancement on the unprocessed millimeter-wave raw data, resulting in an intelligently enhanced image, which improves both imaging quality and foreign object enhancement quality.

[0181] Figure 8 This is a flowchart of a method for training an imaging model of a millimeter wave radar security inspection device provided by an embodiment of the present application. This method can be applied to the above-mentioned millimeter wave radar security inspection device. Figure 8 , the method may include the following steps:

[0182] S801, obtaining first sample millimeter wave raw data obtained by scanning a first sample inspected object by a millimeter wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter wave raw data, and a millimeter wave radar security inspection device imaging model to be trained, wherein the millimeter wave radar security inspection device imaging model includes an AI imaging network and an AI enhancement network.

[0183] It should be noted that the first-sample millimeter-wave raw data is the millimeter-wave signal emitted by the millimeter-wave radar security scanner toward the first-sample inspected object, which is reflected off the surface of the first-sample inspected object and returned to the receiver. The receiver converts the received signal into an electrical signal, which is then processed by the signal processing system to generate image data. The first-sample radar 3D voxel data represents the 3D contour of the millimeter-wave imaging of the first-sample inspected object. 3D voxel data is the smallest unit of digital data segmented in 3D space and is a 3D concept commonly used in fields such as 3D imaging, scientific data, and medical imaging.

[0184] S802: Use the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as a label to train the AI ​​imaging network.

[0185] In one possible embodiment, the first sample millimeter wave raw data is used as input and the first sample radar three-dimensional voxel data is used as a label to train the AI ​​imaging network, including:

[0186] The first sample millimeter wave raw data is input into the AI ​​imaging network to obtain the first sample predicted radar three-dimensional voxel data; the loss of the AI ​​imaging network is determined based on the first sample predicted radar three-dimensional voxel data and the first sample radar three-dimensional voxel data; and the parameters of the AI ​​imaging network are adjusted according to the loss of the AI ​​imaging network.

[0187] In one possible embodiment, the AI ​​imaging network is a fully convolutional network including k convolutional layers. The first sample millimeter wave raw data is input into the AI ​​imaging network to obtain the first sample predicted radar three-dimensional voxel data, including:

[0188] Performing a convolution operation on the first sample millimeter wave raw data using the first convolution layer;

[0189] The t-th convolutional layer is used to perform a convolution operation on the features output by the t-1-th convolutional layer to obtain the first sample predicted radar 3D voxel data, where t∈[2,K].

[0190] The process of inputting the first sample millimeter wave raw data into the AI ​​imaging network to obtain the first sample predicted radar three-dimensional voxel data is similar to step S102 and will not be repeated here.

[0191] In one possible embodiment, determining the loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data includes:

[0192] Based on the first sample predicted radar 3D voxel data and the first sample radar 3D voxel data, the loss of the AI ​​imaging network is determined using the following formula:

[0193]

[0194] Among them, L 2_loss is the loss of the AI ​​imaging network, y i is the first sample radar 3D voxel data of the i-th, x i is the first sample predicted radar 3D voxel data corresponding to the i-th first sample millimeter wave original data, and M is the number of the first sample predicted radar 3D voxel data.

[0195] In a possible embodiment, after adjusting the parameters of the AI ​​imaging network according to the loss of the AI ​​imaging network, other sample millimeter wave raw data are selected to continue training until the preset end condition is met to obtain a trained intelligent imaging model.

[0196] S803 , obtaining a first sample foreign body annotated enhanced image of a first sample inspected object and first sample predicted radar three-dimensional voxel data output by an AI imaging network.

[0197] In a possible embodiment, the first sample foreign body annotated enhanced image is millimeter wave imaging data of the first sample inspected object annotated with foreign bodies, and the foreign bodies are annotated with a color that is different from the first sample inspected object. Figure 3 In this image, the foreign matter is marked with a color that is different from the first sample object.

[0198] S804: Using the first sample predicted radar three-dimensional voxel data as input and the first sample foreign object annotated enhanced image as a label, the AI ​​enhanced network is trained.

[0199] In a possible embodiment, the first sample predicted radar 3D voxel data is used as input and the first sample foreign object annotated enhanced image is used as a label to train the AI ​​enhanced network, including:

[0200] The first sample predicted radar three-dimensional voxel data is input into the AI ​​enhancement network to obtain the first sample predicted foreign object labeling enhanced image; the loss of the AI ​​enhancement network is determined according to the first sample predicted foreign object labeling enhanced image and the first sample foreign object labeling enhanced image; and the parameters of the AI ​​enhancement network are adjusted according to the loss of the AI ​​enhancement network.

[0201] The process of inputting the first sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain the first sample predicted foreign object annotation enhanced image is similar to step S103 and will not be repeated here.

[0202] It should be noted that the first sample predicted foreign body annotated enhanced image is the millimeter wave imaging data of the first sample inspected object predicted to be annotated with foreign bodies, and the foreign bodies are annotated with a color that is different from the first sample inspected object. For example, see Figure 3 In this image, the foreign matter is marked with a color that is different from the first sample object.

[0203] Considering that the SSIM loss can retain high-frequency information, such as the edges and details of the image, the L1 loss function can better keep the brightness and color unchanged. In a possible embodiment, the loss of the AI ​​enhancement network is determined based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, including:

[0204] Determine the L1 loss and structural similarity SSIM loss of the AI ​​enhanced network based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image;

[0205] Determine the loss of the AI-enhanced network based on its L1 loss and SSIM loss.

[0206] In a possible embodiment, determining the L1 loss of the AI ​​enhancement network according to the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image includes:

[0207] Based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, the L1 loss of the AI ​​enhanced network is determined using the following formula:

[0208]

[0209] in, L1 loss for AI enhanced network, y i For the first sample i foreign body labeling enhanced image, x i A foreign body labeling enhanced image is predicted for the first sample corresponding to the i-th first inspected object, and N is the number of foreign body labeling enhanced images predicted for the first sample.

[0210] In a possible embodiment, the SSIM loss of the AI ​​enhanced network is determined using the following formula based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image:

[0211]

[0212] in, is the SSIM loss, and SSIM(x, y) is the SSIM similarity formula between the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image.

[0213] In a possible embodiment, the specific expression of the SSIM similarity formula is:

[0214] SSIM(x,y)=l(x,y)*c(x,y)*s(x,y)

[0215] Among them, SSIM(x, y) is the SSIM similarity formula between the first sample predicted foreign body labeling enhanced image and the first sample foreign body labeling enhanced image, l(x, y) is the brightness similarity function between the first sample predicted foreign body labeling enhanced image and the first sample foreign body labeling enhanced image, c(x, y) is the contrast similarity function between the first sample predicted foreign body labeling enhanced image and the first sample foreign body labeling enhanced image, and s9x, y) is the structural similarity function between the first sample predicted foreign body labeling enhanced image and the first sample foreign body labeling enhanced image.

[0216] In a possible embodiment, the specific expression of l(x, y) is:

[0217]

[0218] Among them, l(x,y) is the brightness similarity function between the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, μ x The average brightness of the first sample foreign body annotation enhanced image, μ y is the average brightness of the first sample foreign body annotated enhanced image, and C1 is a constant to prevent the denominator from being 0;

[0219]

[0220] Among them, μ x is the average brightness of the first sample predicted foreign body annotation enhanced image, n is the number of pixels of the first sample predicted foreign body annotation enhanced image, x i Predict the pixel value of the i-th pixel in the foreign body annotation enhanced image for the first sample;

[0221]

[0222] Among them, μ y is the average brightness of the first sample foreign body annotated enhanced image, n is the number of pixels in the first sample foreign body annotated enhanced image, y i The pixel value of the i-th pixel in the first sample foreign body label enhancement image is given.

[0223] In a possible embodiment, the specific expression of c(x, y) is:

[0224]

[0225] Where c(x, y) is the contrast similarity function between the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, σ x The standard deviation brightness of the first sample foreign body annotation enhanced image is predicted, σ y is the standard deviation brightness of the first sample foreign body annotation enhanced image, and C2 is a constant to prevent the denominator from being 0;

[0226]

[0227] Among them, σ x is the standard deviation brightness of the first sample predicted foreign body annotation enhanced image, n is the number of pixels of the first sample predicted foreign body annotation enhanced image, x i Predict the pixel value of the i-th pixel in the foreign body annotation enhanced image for the first sample, μ x Predict the average brightness of the foreign body annotation enhanced image for the first sample;

[0228]

[0229] Among them, σ y is the standard deviation brightness of the first sample foreign body annotation enhanced image, n is the number of pixels of the first sample foreign body annotation enhanced image, y i is the pixel value of the i-th pixel in the first sample foreign body annotation enhanced image, μ y The average brightness of the object annotations in the sample 3D images.

[0230] In a possible embodiment, the specific expression of s(x, y) is:

[0231]

[0232] Among them, s(x, y) is the structural similarity function between the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, σ xy The covariance brightness of the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, σ x The standard deviation brightness of the first sample foreign body annotation enhanced image is predicted, σ y is the standard deviation brightness of the first sample foreign body annotation enhanced image, and C3 is a constant to prevent the denominator from being 0;

[0233]

[0234] Among them, σ xy is the covariance brightness of the first sample predicted foreign body label enhanced image and the first sample foreign body label enhanced image, n is the number of pixels of the first sample predicted foreign body label enhanced image or the first sample foreign body label enhanced image, x i Predict the pixel value of the i-th pixel in the foreign body annotation enhanced image for the first sample, μ x The average brightness of the first sample predicted foreign body annotation enhanced image, y i is the pixel value of the i-th pixel in the first sample foreign body annotation enhanced image, μ y The average brightness of the enhanced image for the first sample foreign body labeling.

[0235] In one possible embodiment, if The specific expression of the SSIM similarity formula is:

[0236]

[0237] Among them, SSIM(x, y) is the similarity formula between the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image, μ x The average brightness of the first sample foreign body annotation enhanced image, μ y is the average brightness of the first sample foreign body annotation enhanced image, σ x The standard deviation brightness of the first sample foreign body annotation enhanced image is predicted, σ y is the standard deviation brightness of the first sample foreign body annotation enhanced image, σ xy is the covariance brightness between the first sample predicted foreign body label enhanced image and the first sample foreign body label enhanced image, and C2 is a constant to prevent the denominator from being 0.

[0238] In one possible embodiment, considering that the pixel mean and variance vary dramatically across the entire image, which can easily affect the various indicators of SSIM similarity, the SSIM similarity formula cannot be directly applied to the entire image. Instead, a sliding window with a preset window size and a preset step size is used to divide the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image into m sub-images, calculate the SSIM similarity between the corresponding sub-images of the two images, and then take the average as the overall SSIM similarity. Therefore, the specific expression of the SSIM loss of the AI ​​enhanced network is:

[0239]

[0240] in, is the SSIM loss, m is the number of subgraphs divided, SSIM(x p ,y p ) is the SSIM similarity formula corresponding to the p-th subgraph.

[0241] In one possible embodiment, determining the loss of the AI-enhanced network based on the L1 loss and SSIM loss of the AI-enhanced network includes:

[0242] The loss of the AI ​​enhanced network is obtained by adding the product of the L1 loss of the AI ​​enhanced network and the first weight and the product of the SSIM loss and the second weight, and the sum of the first weight and the second weight is 1.

[0243] As an example, based on the L1 loss and SSIM loss of the AI-enhanced network, the loss of the AI-enhanced network is determined using the following formula:

[0244]

[0245] in, is the loss of the AI ​​enhanced network, 1-α is the first weight, α is the second weight, L1 loss for AI-enhanced networks, is the SSIM loss.

[0246] In a possible embodiment, α=0.84, where 0.84 is an empirical value.

[0247] In a possible embodiment, after adjusting the parameters of the AI-enhanced network according to the loss of the AI-enhanced network, other samples are selected to predict the radar three-dimensional voxel data and continue training until a preset end condition is met to obtain a trained AI-enhanced network.

[0248] S805 , obtaining second sample millimeter wave raw data obtained by scanning the second sample inspected object with a millimeter wave radar and a second sample foreign object annotated enhanced image corresponding to the second sample inspected object.

[0249] It should be noted that the millimeter wave raw data of the second sample is transmitted by the millimeter wave radar security inspection device to the second sample inspected object. The millimeter wave signal is reflected by the surface of the second sample inspected object and then returned to the receiver. The receiver converts the received signal into an electrical signal, and generates image data after processing by the signal processing system. The second sample foreign body annotation enhanced image is the millimeter wave imaging data of the second sample inspected object marked with foreign bodies. The foreign bodies are marked with a color that is different from the second sample inspected object. For example, see Figure 3 In this image, the foreign matter is marked with a color that distinguishes it from the second sample object.

[0250] S806: Use the second sample millimeter wave original data as input and the second sample foreign object annotated enhanced image as a label to train the AI ​​imaging network and the AI ​​enhanced network to obtain a trained millimeter wave radar security inspection device imaging model.

[0251] In one possible embodiment, the second sample millimeter wave raw data is used as input and the second sample foreign object annotated enhanced image is used as a label to train the AI ​​imaging network and the AI ​​enhancement network to obtain a trained millimeter wave radar security inspection device imaging model, including:

[0252] Input the second sample millimeter wave original data into the AI ​​imaging network to obtain the second sample predicted radar three-dimensional voxel data; input the second sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain the second sample predicted foreign object labeling enhanced image; determine the loss of the millimeter wave radar security inspection equipment imaging model based on the second sample predicted foreign object labeling enhanced image and the second sample foreign object labeling enhanced image; adjust the parameters of at least one network in the AI ​​imaging network and the AI ​​enhancement network based on the loss of the millimeter wave radar security inspection equipment imaging model.

[0253] In a possible embodiment, the process of inputting the second sample millimeter wave raw data into the AI ​​imaging network to obtain the second sample predicted radar three-dimensional voxel data is similar to steps S801-S802 and is not repeated here.

[0254] In a possible embodiment, the process of inputting the second sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain the second sample predicted foreign object annotated enhanced image is similar to steps S803-S804 and will not be repeated here.

[0255] Through the technical solution provided in the embodiment of the present application, the first sample millimeter wave raw data obtained by the millimeter wave radar scanning the first sample inspected object is used as input, and the first sample radar three-dimensional voxel data corresponding to the first sample millimeter wave raw data is used as a label to train the AI ​​imaging network; the first sample predicted radar three-dimensional voxel data output by the AI ​​imaging network is used as input, and the first sample foreign object annotated enhanced image of the first sample inspected object is used as a label to train the AI ​​enhanced network; the second sample millimeter wave raw data obtained by the millimeter wave radar scanning the second sample inspected object is used as input, and the second sample foreign object annotated enhanced image corresponding to the second sample inspected object is used as a label to train the AI ​​imaging network and the AI ​​enhanced network to obtain a trained millimeter wave radar security inspection equipment imaging model. The embodiment of the present application adopts an end-to-end neural network training method based on millimeter-wave radar security inspection equipment. By adopting a hierarchical pre-training + overall fine-tuning training method, the network training difficulty is reduced, the network convergence is accelerated, and the training resource consumption is reduced; and an AI imaging network is used to replace the traditional imaging algorithm to give full play to the advantages of the neural network, improve the imaging performance, and simplify the system complexity; the AI ​​enhancement network is used to project and enhance the three-dimensional voxel data after imaging to obtain an enhanced image, thereby improving the foreign object detection performance and greatly improving the reliability of security inspection.

[0256] See also Figure 9 , Figure 9 This is a flowchart of a method for training an imaging model for a millimeter-wave radar security inspection device provided in an embodiment of the present application. The method may include the following steps:

[0257] The millimeter-wave radar security inspection device imaging model in the embodiments of this application primarily consists of an AI imaging network and an AI enhancement network. This model is trained using a hierarchical pre-training and overall fine-tuning approach to obtain the trained model. Hierarchical pre-training includes training the AI ​​imaging network and the AI ​​enhancement network, while overall fine-tuning includes overall network training. The AI ​​imaging network is first trained using traditionally generated radar 3D voxel data as intermediate labels and sample millimeter-wave raw data as input. The AI ​​enhancement network is then trained using manually annotated enhanced images as intermediate labels and the output of the AI ​​imaging network as input. Finally, the millimeter-wave radar security inspection device imaging model is fine-tuned using sample millimeter-wave raw data as input and the manually annotated enhanced images as intermediate labels. This overall end-to-end processing fully leverages the advantages of neural networks and improves image enhancement performance. Foreign object detection is achieved using only two neural networks, with no additional processing required, simplifying the complexity of foreign object detection. This hierarchical pre-training and overall fine-tuning training approach reduces network training difficulty, accelerates network convergence, and reduces training resource consumption.

[0258] The present application also provides a millimeter wave radar security inspection device imaging device, see Figure 10 , the device comprises:

[0259] The first acquisition module 1001 is used to obtain the millimeter wave raw data to be processed obtained by the millimeter wave radar scanning the target object, and the pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0260] A first determining module 1002 is configured to input the millimeter wave raw data to be processed into an AI imaging network to obtain predicted radar 3D voxel data, where the predicted radar 3D voxel data is used to represent the millimeter wave imaging 3D contour of the target object;

[0261] The second determination module 1003 is used to input the predicted radar 3D voxel data into the AI ​​enhancement network to obtain an intelligent enhanced image.

[0262] In one possible embodiment, the AI ​​enhanced network includes an image segmentation subnetwork and an attention mechanism subnetwork;

[0263] A second determination module is used to input the predicted radar 3D voxel data into the image segmentation subnetwork to obtain a first segmentation feature map;

[0264] Input the first segmentation feature map into the attention mechanism sub-network to obtain the second segmentation feature map;

[0265] The predicted radar 3D voxel data is labeled using the second segmentation feature map to obtain an intelligent enhanced image.

[0266] In one possible embodiment, the AI ​​imaging network is a fully convolutional network including multiple convolutional layers, the AI ​​enhanced network includes a U-Net subnetwork and a convolutional attention mechanism subnetwork CBAM, the U-Net subnetwork includes n downsampling layers and n upsampling layers corresponding to the downsampling layers, wherein the i-th downsampling layer is jump-connected to the n-i+1-th upsampling layer.

[0267] In one possible embodiment, the attention mechanism subnetwork includes a channel attention module and a spatial attention module, and a second determination module is used to input the first segmentation feature map into the channel attention module, multiply the first segmentation feature map by the channel attention parameter bitwise, and obtain the first segmentation feature map after channel attention correction;

[0268] The first segmentation feature map after channel attention correction is input into the spatial attention module, and the first segmentation feature map after channel attention correction is bitwise multiplied by the spatial attention parameter to obtain the second segmentation feature map.

[0269] In a possible embodiment, the channel attention module includes a first pooling submodule and a fully connected neural network submodule, the fully connected neural network submodule includes a fully connected neural network, and a second determination module is used to input the first segmentation feature map into the first pooling submodule, perform a maximum pooling operation and an average pooling operation on the first segmentation feature map, and obtain a first maximum pooling feature and a first average pooling feature;

[0270] Input the first maximum pooling feature and the first average pooling feature into the fully connected network to obtain the channel attention parameter;

[0271] The first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction.

[0272] In a possible embodiment, the spatial attention module includes a second pooling submodule and a convolutional neural network submodule, the convolutional neural network submodule includes a convolutional neural network, and a second determination module, which is used to input the first segmentation feature map after channel attention correction into the second pooling submodule, and perform a maximum pooling operation and an average pooling operation on the first segmentation feature map after channel attention correction to obtain a second maximum pooling feature and a second average pooling feature;

[0273] Input the second maximum pooling feature and the second average pooling feature into the convolutional neural network to obtain the spatial attention parameters;

[0274] The spatial attention parameter is bitwise multiplied with the first segmentation feature map corrected by the channel attention to obtain the second segmentation feature map.

[0275] The present application also provides a millimeter wave radar security inspection device imaging model training device, see Figure 11 , the device comprises:

[0276] A second acquisition module 1101 is configured to acquire first sample millimeter-wave raw data obtained by scanning a first sample inspected object by a millimeter-wave radar, first sample radar 3D voxel data corresponding to the first sample millimeter-wave raw data, and a millimeter-wave radar security inspection device imaging model to be trained, wherein the first sample radar 3D voxel data is used to represent a millimeter-wave imaging 3D contour of the first sample inspected object, and the millimeter-wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0277] A first training module 1102 is configured to train an AI imaging network using the first sample millimeter wave raw data as input and the first sample radar 3D voxel data as labels;

[0278] The third acquisition module 1103 is configured to acquire a first sample foreign body annotated enhanced image of a first sample inspected object and first sample predicted radar 3D voxel data output by the AI ​​imaging network;

[0279] The second training module 1104 is configured to train the AI ​​enhancement network using the first sample predicted radar 3D voxel data as input and the first sample foreign object annotated enhanced image as a label;

[0280] The fourth acquisition module 1105 is configured to acquire second sample millimeter wave raw data obtained by scanning the second sample inspected object by the millimeter wave radar and a second sample foreign body annotated enhanced image corresponding to the second sample inspected object;

[0281] The third training module 1106 is used to train the AI ​​imaging network and the AI ​​enhancement network using the second sample millimeter wave original data as input and the second sample foreign object annotated enhanced image as a label to obtain a trained millimeter wave radar security inspection device imaging model.

[0282] In a possible embodiment, the first training module is configured to input first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data;

[0283] determining a loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data;

[0284] Adjust the parameters of the AI ​​imaging network according to its loss.

[0285] In one possible embodiment, the AI ​​imaging network is a fully convolutional network, which includes k convolutional layers, and a first training module for performing a convolution operation on the first sample millimeter wave raw data using the first convolutional layer;

[0286] The t-th convolutional layer is used to perform a convolution operation on the features output by the t-1-th convolutional layer to obtain the first sample predicted radar 3D voxel data, where t∈[2,K].

[0287] In one possible embodiment, the first training module is configured to determine the loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data using the following formula:

[0288]

[0289] Among them, L 2_loss is the loss of the AI ​​imaging network, y i is the first sample radar 3D voxel data of the i-th, x i is the first sample predicted radar 3D voxel data corresponding to the i-th first sample millimeter wave original data, and M is the number of the first sample predicted radar 3D voxel data.

[0290] In a possible embodiment, the second training module is configured to input the first sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain the first sample predicted foreign object annotated enhanced image;

[0291] Determine the loss of the AI ​​enhancement network based on the first sample predicted foreign body annotation enhanced image and the first sample foreign body annotation enhanced image;

[0292] Adjust the parameters of the AI-enhanced network according to its loss.

[0293] In a possible embodiment, the second training module is configured to determine the L1 loss and the structural similarity SSIM loss of the AI ​​enhancement network based on the foreign object annotation enhanced image predicted by the first sample and the foreign object annotation enhanced image of the first sample;

[0294] Determine the loss of the AI-enhanced network based on its L1 loss and SSIM loss.

[0295] The present application also provides an electronic device, such as Figure 12 Shown, including:

[0296] Memory 1201, used for storing computer programs;

[0297] The processor 1202 is configured to execute the program stored in the memory 1201 and implement the following steps:

[0298] Obtaining first sample millimeter wave raw data obtained by scanning a first sample inspected object by a millimeter wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter wave raw data, and a millimeter wave radar security inspection device imaging model to be trained, wherein the first sample radar three-dimensional voxel data is used to represent a millimeter wave imaging three-dimensional contour of the first sample inspected object, and the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network;

[0299] The AI ​​imaging network is trained using the first sample millimeter wave raw data as input and the first sample radar 3D voxel data as labels;

[0300] Obtaining a first sample foreign body annotated enhanced image of a first sample inspected object and first sample predicted radar three-dimensional voxel data output by an AI imaging network;

[0301] The AI ​​enhancement network is trained using the first sample predicted radar 3D voxel data as input and the first sample foreign object annotated enhanced image as a label;

[0302] Acquire second sample millimeter wave raw data obtained by scanning the second sample inspected object by the millimeter wave radar, and a second sample foreign body annotated enhanced image corresponding to the second sample inspected object;

[0303] The second sample millimeter wave original data is used as input and the second sample foreign object annotated enhanced image is used as a label to train the AI ​​imaging network and the AI ​​enhanced network to obtain the trained millimeter wave radar security inspection equipment imaging model.

[0304] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 1202, the communication interface, and the memory 1201 communicate with each other via the communication bus.

[0305] The communication bus mentioned in the electronic device mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0306] The communication interface is used for communication between the above electronic device and other devices.

[0307] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk memory. Alternatively, the memory may be at least one storage device located away from the processor.

[0308] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0309] In another embodiment provided in the present application, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned millimeter-wave radar security inspection equipment imaging and model training methods are implemented.

[0310] In another embodiment provided in the present application, a computer program product containing instructions is also provided. When the computer is run on the computer, the computer executes any of the millimeter wave radar security inspection equipment imaging and model training methods in the above embodiments.

[0311] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0312] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0313] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device, electronic device, and storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simple. For related portions, refer to the descriptions of the method embodiments.

[0314] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A method for training an imaging model for a millimeter-wave radar security inspection device, characterized in that: The method comprises: Obtaining first sample millimeter-wave raw data obtained by scanning a first sample inspected object by a millimeter-wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter-wave raw data, and a millimeter-wave radar security inspection device imaging model to be trained, wherein the first sample radar three-dimensional voxel data is used to represent a millimeter-wave imaging three-dimensional contour of the first sample inspected object, and the millimeter-wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network; Training the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as labels; Acquire a first sample foreign body annotated enhanced image of the first sample inspected object and first sample predicted radar three-dimensional voxel data output by the AI ​​imaging network; Training the AI ​​enhanced network using the first sample predicted radar three-dimensional voxel data as input and the first sample foreign object annotated enhanced image as a label; Acquire second sample millimeter wave raw data obtained by scanning a second sample inspected object by a millimeter wave radar and a second sample foreign object annotated enhanced image corresponding to the second sample inspected object; The second sample millimeter wave original data is used as input and the second sample foreign object annotated enhanced image is used as a label, and the AI ​​imaging network and the AI ​​enhanced network are trained to obtain a trained millimeter wave radar security inspection equipment imaging model.

2. The method according to claim 1, characterized in that The training of the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as a label includes: Inputting the first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data; determining a loss of the AI ​​imaging network based on the first sample predicted 3D radar voxel data and the first sample 3D radar voxel data; Parameters of the AI ​​imaging network are adjusted according to the loss of the AI ​​imaging network.

3. The method according to claim 2, characterized in that The AI ​​imaging network is a fully convolutional network including k convolutional layers. Inputting the first sample millimeter wave raw data into the AI ​​imaging network to obtain first sample predicted radar three-dimensional voxel data includes: Performing a convolution operation on the first sample millimeter wave raw data using a first convolution layer; The t-th convolutional layer is used to perform a convolution operation on the features output by the t-1-th convolutional layer to obtain the first sample predicted radar 3D voxel data, where t∈[2,K].

4. The method according to claim 2, characterized in that The determining the loss of the AI ​​imaging network according to the first sample predicted radar 3D voxel data and the first sample radar 3D voxel data includes: The loss of the AI ​​imaging network is determined based on the first sample predicted radar 3D voxel data and the first sample radar 3D voxel data using the following formula: Among them, L 2_loss is the loss of the AI ​​imaging network, y i is the first sample radar 3D voxel data of the i-th, x i is the first sample predicted radar 3D voxel data corresponding to the i-th first sample millimeter wave original data, and M is the number of the first sample predicted radar 3D voxel data.

5. The method according to claim 1, wherein The training of the AI ​​enhanced network using the first sample predicted radar three-dimensional voxel data as input and the first sample foreign object annotated enhanced image as a label includes: Inputting the first sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a first sample predicted foreign object annotated enhanced image, wherein the first sample foreign object annotated enhanced image is millimeter wave imaging data of the first sample inspected object annotated with a foreign object, and the foreign object is annotated with a color that is different from the first sample inspected object; Determining the loss of the AI ​​enhanced network based on the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image; Adjusting parameters of the AI ​​enhanced network according to the loss of the AI ​​enhanced network.

6. The method according to claim 5, characterized in that The determining the loss of the AI ​​enhanced network according to the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image includes: Determining the L1 loss and the structural similarity SSIM loss of the AI ​​enhanced network according to the first sample predicted foreign body annotated enhanced image and the first sample foreign body annotated enhanced image; Determine a loss of the AI ​​enhanced network based on the L1 loss and the SSIM loss of the AI ​​enhanced network.

7. The method according to claim 1, characterized in that The method uses the second sample millimeter wave raw data as input and the second sample foreign object annotated enhanced image as a label, trains the AI ​​imaging network and the AI ​​enhanced network, and obtains a trained millimeter wave radar security inspection device imaging model, including: Inputting the second sample millimeter wave raw data into the AI ​​imaging network to obtain second sample predicted radar three-dimensional voxel data; Inputting the second sample predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain a second sample predicted foreign object annotated enhanced image, where the second sample foreign object annotated enhanced image is millimeter wave imaging data of the second sample inspected object annotated with a foreign object, where the foreign object is annotated with a color that is different from the second sample inspected object; Determine the loss of the imaging model of the millimeter-wave radar security inspection device according to the second sample predicted foreign object annotated enhanced image and the second sample foreign object annotated enhanced image; Adjust the parameters of at least one of the AI ​​imaging network and the AI ​​enhancement network according to the loss of the imaging model of the millimeter wave radar security inspection device.

8. A millimeter wave radar security inspection device imaging method, characterized in that: The method comprises: Obtaining unprocessed millimeter wave raw data obtained by scanning the target object by the millimeter wave radar, and a pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network; Inputting the to-be-processed millimeter-wave raw data into an AI imaging network to obtain predicted radar three-dimensional voxel data, where the predicted radar three-dimensional voxel data is used to represent the millimeter-wave imaging three-dimensional contour of the target object; The predicted radar three-dimensional voxel data is input into the AI ​​enhancement network to obtain an intelligent enhanced image.

9. The method according to claim 8, characterized in that The AI ​​enhanced network includes an image segmentation subnetwork and an attention mechanism subnetwork; Inputting the predicted radar three-dimensional voxel data into an AI enhancement network to obtain an intelligent enhanced image includes: Inputting the predicted radar 3D voxel data into an image segmentation subnetwork to obtain a first segmentation feature map; Inputting the first segmentation feature map into the attention mechanism sub-network to obtain a second segmentation feature map; The predicted radar three-dimensional voxel data is labeled using the second segmentation feature map to obtain an intelligent enhanced image.

10. The method according to claim 8 or 9, characterized in that The AI ​​imaging network is a fully convolutional network including multiple convolutional layers, the AI ​​enhanced network includes a U-Net subnetwork and a convolutional attention mechanism subnetwork CBAM, the U-Net subnetwork includes n downsampling layers and n upsampling layers corresponding to the downsampling layers, wherein the i-th downsampling layer is jump-connected to the n-i+1-th upsampling layer.

11. The method according to claim 9, characterized in that The attention mechanism sub-network includes a channel attention module and a spatial attention module, and the first segmentation feature map is input into the attention mechanism sub-network to obtain a second segmentation feature map, including: Inputting the first segmentation feature map into the channel attention module, and bitwise multiplying the first segmentation feature map by the channel attention parameter to obtain a first segmentation feature map after channel attention correction; The first segmentation feature map after the channel attention correction is input into the spatial attention module, and the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map.

12. The method according to claim 11, characterized in that The channel attention module includes a first pooling submodule and a fully connected neural network submodule, wherein the fully connected neural network submodule includes a fully connected neural network, inputting the first segmentation feature map into the channel attention module, bitwise multiplying the first segmentation feature map by the channel attention parameter, and obtaining a first segmentation feature map after channel attention correction, including: Inputting the first segmentation feature map into the first pooling submodule, performing a maximum pooling operation and an average pooling operation on the first segmentation feature map to obtain a first maximum pooling feature and a first average pooling feature; Inputting the first maximum pooling feature and the first average pooling feature into a fully connected network to obtain a channel attention parameter; The first segmentation feature map is bitwise multiplied by the channel attention parameter to obtain the first segmentation feature map after channel attention correction.

13. The method according to claim 11, characterized in that The spatial attention module includes a second pooling submodule and a convolutional neural network submodule, the convolutional neural network submodule includes a convolutional neural network, the first segmentation feature map after the channel attention correction is input into the spatial attention module, the first segmentation feature map after the channel attention correction is bitwise multiplied by the spatial attention parameter to obtain a second segmentation feature map, including: Inputting the first segmentation feature map after the channel attention correction into the second pooling submodule, performing a maximum pooling operation and an average pooling operation on the first segmentation feature map after the channel attention correction to obtain a second maximum pooling feature and a second average pooling feature; Inputting the second maximum pooling feature and the second average pooling feature into a convolutional neural network to obtain a spatial attention parameter; The spatial attention parameter is bitwise multiplied by the first segmentation feature map corrected by the channel attention to obtain a second segmentation feature map.

14. A millimeter wave radar security inspection equipment imaging model training device, characterized in that: The device comprises: A second acquisition module is configured to acquire first sample millimeter-wave raw data obtained by scanning a first sample inspected object by a millimeter-wave radar, first sample radar three-dimensional voxel data corresponding to the first sample millimeter-wave raw data, and a millimeter-wave radar security inspection device imaging model to be trained, wherein the first sample radar three-dimensional voxel data is used to represent the millimeter-wave imaging three-dimensional contour of the first sample inspected object, and the millimeter-wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network; a first training module, configured to train the AI ​​imaging network using the first sample millimeter wave raw data as input and the first sample radar three-dimensional voxel data as labels; a third acquisition module, configured to acquire a first sample foreign body annotated enhanced image of the first sample inspected object and first sample predicted radar 3D voxel data output by the AI ​​imaging network; a second training module, configured to train the AI ​​enhanced network using the first sample predicted radar 3D voxel data as input and the first sample foreign object annotated enhanced image as a label; A fourth acquisition module is configured to acquire second sample millimeter wave raw data obtained by scanning a second sample inspected object by a millimeter wave radar and a second sample foreign body annotated enhanced image corresponding to the second sample inspected object; The third training module is used to take the second sample millimeter wave original data as input and the second sample foreign object annotated enhanced image as a label, train the AI ​​imaging network and the AI ​​enhanced network, and obtain the trained millimeter wave radar security inspection equipment imaging model.

15. A millimeter wave radar security inspection equipment imaging device, characterized in that: The device comprises: A first acquisition module is used to obtain unprocessed millimeter wave raw data obtained by the millimeter wave radar scanning the target object, and a pre-trained millimeter wave radar security inspection device imaging model, wherein the millimeter wave radar security inspection device imaging model includes an artificial intelligence (AI) imaging network and an AI enhancement network; A first determination module is configured to input the to-be-processed millimeter-wave raw data into an AI imaging network to obtain predicted radar three-dimensional voxel data, wherein the predicted radar three-dimensional voxel data is used to represent a millimeter-wave imaging three-dimensional contour of the target object; The second determination module is used to input the predicted radar three-dimensional voxel data into the AI ​​enhancement network to obtain an intelligent enhanced image.

16. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method described in any one of claims 1-7 or 8-13 when executing a program stored in a memory.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 or 8 to 13 is implemented.

Images