A millimeter wave imaging dangerous goods detection method based on FPGA and depth learning

A millimeter wave imaging and dangerous goods technology, applied in character and pattern recognition, biological neural network models, instruments, etc., can solve the problem of inability to accurately extract the irregular shape and outline of dangerous goods, and achieve fine concurrent operation granularity and concurrent execution efficiency. , fast processing speed, good function and customizability advantage

Inactive Publication Date: 2019-03-08
博微太赫兹信息科技有限公司
5 Cites 21 Cited by

AI-Extracted Technical Summary

Problems solved by technology

[0005] The technical problem to be solved by the present invention is that the existing technology cannot accurately extract the irregular conto...
View more

Method used

[0057] Deep learning neural network has the advantages of high detection rate and fast detection speed, but the amount of calculation is huge, and the effect of real-time detection cannot be achieved by using embedded processor or server CPU. In order to reduce the computing load of the FPGA, the millimeter-wave imaging algorithm is partly carried out in an FPGA independent of the detection of dangerous objects, and the forward transfer operation in the network model is carried out in another FPGA, thus realizing the acceleration of the deep learning network Running, so that the entire detection process of dangerous goods is real-time.
[0058] After the imaging algorithm, the FPGA for millimeter-wave imaging outputs a millimeter-wave image with a fixed image size to the back-end FPGA, and the image resolution is 200*512; the image is output to the back-end FPGA through a high-speed bus for network model calculation. The forward transfer process of the selected ...
View more

Abstract

The invention discloses a millimeter wave imaging dangerous goods detection method based on FPGA and depth learning, which comprises the steps of obtaining the millimeter wave imaging pictures and identifying dangerous goods in the pictures; constructing the depth learning network model of hazardous materials detection, and using the labeled millimeter-wave imaging images to train the model to getthe trained detection model; locating the trained detection model into the detection system of FPGA platform; using the FPGA to detect the millimeter wave image to be measured; using the trained detection model to detect the collected millimeter-wave images; if the image to be measured contains hazardous materials, marking the hazardous materials category, alarming, and marking the coordinates ofthe irregular outline position of the hazardous materials at the same time. Based on the FPGA-based platform, using the depth neural network model trained by Mask R-CNN object detection framework forthe real-time foreign body detection, so that compared with ASIC chips, the method has better customizability.

Application Domain

Character and pattern recognitionNeural architectures

Technology Topic

Object detectionNetwork model +7

Image

  • A millimeter wave imaging dangerous goods detection method based on FPGA and depth learning
  • A millimeter wave imaging dangerous goods detection method based on FPGA and depth learning

Examples

  • Experimental program(1)

Example Embodiment

[0038] The embodiments of the present invention are described in detail below. This embodiment is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation manner and a specific operation process, but the protection scope of the present invention is not limited to the following implementation. example.
[0039] like figure 1 and figure 2 As shown, this embodiment includes two parts: offline model training and online detection, which are described as follows:
[0040] 1. Offline training of deep learning network model part:
[0041] Using the image semantic segmentation annotation tool labelme, the outlines of all dangerous objects in each image are marked separately as the input of the deep learning network.
[0042] The convolutional neural network used to extract features has a total of 8 layers, including 5 convolutional layers and 3 max-pooling layers. Normalize the input image data, adjust the image size to 224*224, and use 8-layer convolutional neural network for feature extraction for each image.
[0043] An initialized image with a size of 224*224 is passed through a convolution filter with a size of 11*11 and a stride of 2 to generate 110*110*32 and enter the next layer.
[0044] The second layer is the maximum pooling layer, the size is 3*3, the step size is 2, the output of 55*55*32 is obtained, and the third layer is entered.
[0045] The third layer is the convolution layer, the convolution kernel is 5*5, the stride is 2, and the output of 26*26*86 is obtained.
[0046] The fourth layer is the maximum pooling layer with a size of 3*3 and a stride of 2, resulting in an output of 13*13*32.
[0047] The fifth layer is a convolutional layer, the size of the convolution kernel is 3*3, the stride is 1, and the output of 13*13*128 is obtained.
[0048] The sixth layer is the convolution layer, and the convolution kernel size and stride of the fifth layer are the same, and the output of 13*13*128 is obtained.
[0049] The seventh layer is the convolution layer, and the convolution kernel size and stride of the fifth layer are the same, and the output of 13*13*128 is obtained.
[0050] The eighth layer is the maximum pooling layer with a size of 3*3 and a stride of 2, resulting in a feature map of 13*13*128.
[0051] The region proposal network RPN, which generates candidate regions according to the feature map, includes three convolution layers. The size of the convolution kernel of the first layer is 3*3. The output results are divided into two channels, and the size of the two convolution kernels is 1*. A convolutional layer of 1, which generates the probability that the candidate region is the target and the bounding box coordinates of the candidate region. RPN adopts the anchor mechanism. On the feature map generated by the original image through the convolutional neural network, each point corresponds to the 9 anchors on the original image. After the region proposal network RPN generates the regression sum of the four coordinates of these 9 anchors as the target probability value.
[0052] Since the position of the candidate region is a floating-point number, and the pooled feature map requires a fixed size, it is necessary to use ROI Align to extract a fixed-size feature map for subsequent classification and bounding box regression operations. The process of ROI Align is to traverse each candidate area and divide the candidate area into k*k units. The floating-point boundary of the candidate area and the boundary of each unit are not quantized, and four fixed coordinates are calculated in each unit. position, the values ​​of these four positions are calculated by bilinear interpolation, and then the maximum pooling operation is performed.
[0053] According to the feature map of fixed size, softmax is used to separate the candidate regions, and at the same time, the final accurate position of the bounding box is obtained by regression calculation again.
[0054] The Mask branch that generates masks is a fully convolutional network FCN, the input is the fixed-size feature maps generated by ROI Align, and the output is their masks. The Mask branch includes 4 convolutional layers, and finally generates a binary mask map of size 28*28*80. The small size property of the mask helps keep the mask branch network lightweight. During training, we downscale the real mask to 28x28 to compute the loss function, and in inference, upscale the predicted mask to the size of the ROI bounding box to give the final mask result, one mask per target membrane.
[0055] 2. FPGA-based online detection part:
[0056] The FPGA-based millimeter wave imaging dangerous goods detection part mainly applies the deep learning network model generated by offline training to real-time dangerous goods detection. After the FPGA is initialized, the trained deep learning network model parameters are loaded; when the person to be tested enters the millimeter-wave detection area, the millimeter-wave imaging device images the millimeter-wave reflected by the human body and collects real-time data. The size is 200*512 pixels; the FPGA takes the image data as input, passes it to the deep learning program framework in the FPGA, and uses the loaded network model parameters to detect dangerous objects; if there are dangerous objects in the currently processed image, mark the danger The category of the item starts to alarm, and the area with the mask value of 1 in the image is obtained at the same time, which is used to identify the location coordinates of the dangerous item. If no hazardous material is present, proceed to the next image.
[0057] Deep learning neural network has the advantages of high detection rate and fast detection speed, but the calculation amount is huge, and the effect of real-time detection cannot be achieved by using embedded processor or server CPU. In order to reduce the computational load of the FPGA, the millimeter-wave imaging algorithm is partially carried out in a piece of FPGA independent of the detection of dangerous objects, and the forward transfer operation in the network model is carried out in another piece of FPGA, thus realizing the acceleration of the deep learning network. The operation makes the whole detection process of dangerous goods have real-time performance.
[0058] After the FPGA used for millimeter wave imaging passes the imaging algorithm, it outputs a fixed image size millimeter wave image to the back-end FPGA, and the image resolution is 200*512; the image is output to the back-end FPGA through the high-speed bus for network model operation. The forward transfer process of the selected target detection network is mainly composed of multi-channel convolution operations. Taking advantage of the concurrent operation of FPGA, the convolution operations of each channel can be performed at the same time, thereby accelerating the neural network execution process.
[0059] Compared with the traditional CPU, the FPGA platform has better concurrency and faster processing speed; compared with the traditional GPU, it has finer concurrent operation granularity and concurrent execution efficiency; compared with the application-specific integrated circuit chip, it has better functions and can be customized sexual advantage.
[0060] The traditional image detection algorithm and the deep learning network based on other models are used for testing, and the effects are not as good as the technical solution of the present invention using the deep learning network. Specifically, the detection rate of dangerous goods based on the Mask R-CNN deep network model is replaced by the traditional image detection algorithm, and the detection rate is significantly reduced; the detection of dangerous goods based on the Mask R-CNN deep network model is replaced by other deep learning networks, which cannot Accurately describe the irregular contours of dangerous goods, thereby increasing the workload of manual judgment.
[0061] A comparison was made between using the FPGA platform and the traditional processor platform to run the Mask R-CNN-based deep network model to detect dangerous goods. Using the traditional processor-based platform, the detection time was much longer than the FPGA-based platform.
[0062] The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.

Similar technology patents

Multithread concurrent data compression method based on FT processor platform

InactiveCN103984528Aaccelerategood concurrency
Owner:NAT UNIV OF DEFENSE TECH

Data processing method and device

Owner:ADVANCED NEW TECH CO LTD

Driver smoking behavior real-time monitoring method based on vehicle video monitoring

InactiveCN105976570Aprocessing speedSoftware and hardware resource requirements are not high
Owner:SHANDONG NORMAL UNIV

JPEG_LS rule coding hardware achieving method based on scanning sequence changing

InactiveCN104717497Aprocessing speed
Owner:北京润光泰力科技发展有限公司

Multipicture synthesizing method and system

InactiveCN104168427Aprocessing speedgood effect
Owner:SUZHOU CODYY NETWORK SCI & TECH

Preparation method and application for graphene multi-metal composite material

InactiveCN110015746AExcellent organic matter degradation abilityprocessing speed
Owner:SHANGHAI UNIV

Classification and recommendation of technical efficacy words

  • processing speed
  • good concurrency

Truth random number producing method and device

InactiveCN102176199Agood randomnessprocessing speed
Owner:XI'AN INST OF OPTICS & FINE MECHANICS - CHINESE ACAD OF SCI

Vision auxiliary device and method for automatic landing of unmanned aerial vehicle

Owner:CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI

Multithread concurrent data compression method based on FT processor platform

InactiveCN103984528Aaccelerategood concurrency
Owner:NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products