Millimeter wave radar security inspection method and system based on deep learning and imaging inversion technology
Through deep learning and imaging inversion technology, combined with DT, RMA and pyramid fusion algorithms, the problems of millimeter wave radar security inspection system in resolution, accuracy and detection speed are solved, and more efficient object recognition and classification are achieved, suitable for Detection of complex environments.
Patent Information
- Application Number
- CN202410471039.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-18
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-04-18
AI Technical Summary
The existing millimeter-wave radar security system is difficult to ensure resolution and accuracy and slow detection speed.
Using a method based on deep learning and imaging inversion technology, radar signals are generated through the FMCW module, combined with DT algorithm, RMA algorithm and pyramid fusion algorithm, signal processing and image optimization are performed, and finally input to the YOLOv8 model for object classification.
Significantly improves the resolution and accuracy of the system, improves detection speed, and reduces overall costs, enhancing adaptability and image quality in complex environments.
Smart Images

Figure CN118191826B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of millimeter wave radar technology, and in particular to a millimeter wave radar security inspection method and system based on deep learning and imaging inversion technology. Background Art
[0002] Millimeter-wave radar technology has unique advantages in the field of security inspection due to its wavelength characteristics. It can penetrate clothing and other non-metallic materials to effectively detect hidden objects. However, traditional millimeter-wave radar systems have certain limitations in image clarity, detection speed, cost control, and adaptability to complex environments. For example, early systems may have difficulty accurately identifying small or hidden objects due to insufficient resolution, or their wide application is limited by high hardware costs. For example, a package security inspection method, device, storage medium and electronic device disclosed in patent publication number CN114779241A use traditional millimeter-wave radar systems for security inspection. In addition, traditional algorithms may have problems with insufficient accuracy and slow processing speed when processing data in complex scenarios. Therefore, there is an urgent need to improve and upgrade traditional millimeter-wave radar systems by integrating more advanced imaging algorithms, data processing technologies and more cost-effective hardware designs. Summary of the invention
[0003] The technical problem to be solved by the present invention is that the millimeter wave radar security inspection method and system in the prior art have the problem that the resolution and accuracy are difficult to guarantee and the detection speed is slow.
[0004] The present invention solves the above technical problems by the following technical means: a millimeter wave radar security inspection method based on deep learning and imaging inversion technology, comprising the following steps:
[0005] Step 1: The FMCW module generates a radar signal, and the transmitting switch array turns on the transmitting antenna array to transmit the radar signal;
[0006] Step 2: The servo scanning control system adjusts the direction of the antenna to ensure that the radar beam covers the entire detection area;
[0007] Step 3: The receiving switch array turns on the receiving antenna array, and the signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module. The FMCW module down-converts the received signal to obtain an intermediate frequency signal; the intermediate frequency signal is converted into a digital signal by digital-to-analog conversion, and the digital signal is processed by digital signal processing to obtain a processed signal;
[0008] Step 4: Process the processed signal using the DT algorithm and the RMA algorithm to obtain an image of the object to be measured;
[0009] Step 5: The image of the object to be tested is subjected to detail fusion through a pyramid fusion algorithm to obtain an optimized image;
[0010] Step 6: Input the optimized image into the YOLOv8 model and output the classification result of the object to be tested.
[0011] Furthermore, the processing of the DT algorithm includes:
[0012] The scattered field of the object to be measured at position r in the signal processed in step 3 is expressed by the following integral equation:
[0013]
[0014] Among them, u s (r) is the scattered field of the object to be measured at position r, r′ is the position of the source point, f(r′) is the scattering potential of the object to be measured, G(r,r′) is the Green's function, and V is the volume of the object to be measured;
[0015] The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as:
[0016] f(r)=k 2 (∈(r)-∈0)+iωσ(r)
[0017] Where k is the wave number, ∈(r) and σ(r) are the dielectric constant and conductivity of the object to be measured at position r, ∈0 is the background dielectric constant, and ω is the angular frequency;
[0018] By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured can be obtained.
[0019] Furthermore, the processing of the RMA algorithm includes:
[0020] Step 401: preprocess the signal S(t) processed in step 3 to obtain a preprocessed signal S pre (t)
[0021] S pre (t) = S(t)·W(t)
[0022] Where W(t) is the window function;
[0023] Step 402: preprocess the signal S pre (t) Perform distance compression:
[0024] S range (f r )=FFT{S pre (t)}·H r (f r )
[0025] Among them, S range (f r) is the signal after range compression, FFT stands for Fourier transform, H r (f r ) is a range compression filter;
[0026] Step 403: perform azimuth compression on the range-compressed signal:
[0027] S final (f r ,f a )=IFFT{S range (f r )·H a (f a )}
[0028] Among them, S final (f r ,f a ) is the signal after compression processing, IFFT stands for inverse Fourier transform, H a (f a ) is an azimuth compression filter;
[0029] Step 404: The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal:
[0030] I(x,y)=IFFT{S final (f r ,f a )}
[0031] Where I(x,y) represents the intensity of each pixel in the image;
[0032] Step 405: Invert the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
[0033] Furthermore, the processing process of the pyramid fusion algorithm is as follows:
[0034] Step 501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample to generate the next layer, which can be expressed as
[0035] I next =Downsample(GaussianBlur(I current ))
[0036] Among them, I next is the image of the next layer of the Gaussian pyramid, I current is the current layer image of the Gaussian pyramid, GaussianBlur is Gaussian filtering, and Downsample is downsampling;
[0037] Step 502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain a Laplacian layer, which can be expressed as
[0038] L i =G i -Upsample(G i+1 )
[0039] Among them, L i represents the i-th Laplacian layer of the Laplacian pyramid, G i+1 Represents the current level of the Gaussian pyramid, G i Indicates the previous layer of the Gaussian pyramid, and Upsample indicates upsampling;
[0040] Step 503, merging Laplacian pyramids: averaging and merging the Laplacian pyramids at each level;
[0041] Step 504, image reconstruction: reconstruct the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
[0042] Furthermore, the digital signal processing includes filtering, amplification and feature extraction.
[0043] Furthermore, the millimeter wave radar security inspection method further includes step seven:
[0044] The detection and recognition results of the objects to be tested are displayed on a computer display screen, showing the location and category of the detected objects to be tested.
[0045] The present invention also provides a millimeter wave radar security inspection system based on deep learning and imaging inversion technology, including:
[0046] The signal transmitting unit is used for the FMCW module to generate radar signals, and the transmitting switch array turns on the transmitting antenna array to transmit the radar signal;
[0047] Scan control unit, used to servo the scan control system to adjust the direction of the antenna to ensure that the radar beam covers the entire detection area;
[0048] The signal receiving and processing unit is used to receive the switch array to open the receiving antenna array. The signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module. The FMCW module down-converts the received signal to obtain an intermediate frequency signal. The intermediate frequency signal is converted into a digital signal through digital-to-analog conversion. The digital signal is processed into a processed signal through digital signal processing.
[0049] An image inversion unit is used to process the processed signal using a DT algorithm and an RMA algorithm to obtain an image of the object to be measured;
[0050] An image optimization unit, used for fusing details of the image of the object to be measured through a pyramid fusion algorithm to obtain an optimized image;
[0051] The object detection unit is used to input the optimized image into the YOLOv8 model and output the classification result of the object to be tested.
[0052] Furthermore, the processing of the DT algorithm includes:
[0053] The scattered field of the object to be measured at position r in the signal processed by the signal receiving and processing unit is expressed by the following integral equation:
[0054]
[0055] Among them, u s (r) is the scattered field of the object to be measured at position r, r′ is the position of the source point, f(r′) is the scattering potential of the object to be measured, G(r,r′) is the Green's function, and V is the volume of the object to be measured;
[0056] The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as:
[0057] f(r)=k 2 (∈(r)-∈0)+iωσ(r)
[0058] Where k is the wave number, ∈(r) and σ(r) are the dielectric constant and conductivity of the object to be measured at position r, ∈0 is the background dielectric constant, and ω is the angular frequency;
[0059] By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured can be obtained.
[0060] Furthermore, the processing of the RMA algorithm includes:
[0061] Step 401: preprocess the signal S(t) processed by the signal receiving and processing unit to obtain a preprocessed signal S pre (t)
[0062] S pre (t) = S(t)·W(t)
[0063] Where W(t) is the window function;
[0064] Step 402: preprocess the signal S pre (t) Perform distance compression:
[0065] S range (fr )=FFT{S pre (t)}·H r (f r )
[0066] Among them, S range (f r ) is the signal after range compression, FFT stands for Fourier transform, H r (f r ) is a range compression filter;
[0067] Step 403: perform azimuth compression on the range-compressed signal:
[0068] S final (f r ,f a )=IFFT{S range (f r )·H a (f a )}
[0069] Among them, S final (f r ,f a ) is the signal after compression processing, IFFT stands for inverse Fourier transform, H a (f a ) is an azimuth compression filter;
[0070] Step 404: The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal:
[0071] I(x,y)=IFFT{S final (f r ,f a )}
[0072] Where I(x,y) represents the intensity of each pixel in the image;
[0073] Step 405: Invert the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
[0074] Furthermore, the processing process of the pyramid fusion algorithm is as follows:
[0075] Step 501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample to generate the next layer, which can be expressed as
[0076] I next =Downsample(GaussianBlur(I current ))
[0077] Among them, I next is the image of the next layer of the Gaussian pyramid, I current is the current layer image of the Gaussian pyramid, GaussianBlur is Gaussian filtering, and Downsample is downsampling;
[0078] Step 502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain a Laplacian layer, which can be expressed as
[0079] L i =G i -Upsample(G i+1 )
[0080] Among them, L i represents the i-th Laplacian layer of the Laplacian pyramid, G i+1 Represents the current level of the Gaussian pyramid, G i Indicates the previous layer of the Gaussian pyramid, and Upsample indicates upsampling;
[0081] Step 503, merging Laplacian pyramids: averaging and merging the Laplacian pyramids at each level;
[0082] Step 504, image reconstruction: reconstruct the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
[0083] Furthermore, the digital signal processing includes filtering, amplification and feature extraction.
[0084] Furthermore, the millimeter wave radar security inspection system further includes a data display unit, which is used to:
[0085] The detection and recognition results of the objects to be tested are displayed on a computer display screen, showing the location and category of the detected objects to be tested.
[0086] The advantages of the present invention are:
[0087] (1) The present invention transmits and receives radar signals through precise switch array technology. The DT algorithm deeply analyzes weak scattered signals to reveal the characteristics of objects, and the RMA algorithm ensures high-definition imaging even when the target is moving. The pyramid fusion algorithm further improves the image quality, enabling the YOLOv8 model to identify and classify more accurately. The combination of multiple algorithms greatly improves the resolution and accuracy of the entire solution, and significantly improves the detection speed.
[0088] (2) The present invention simplifies the hardware structure of the system and reduces the overall cost.
[0089] (3) The present invention improves the directionality and coverage of the signal through the application of the switch array, and significantly improves the accuracy and resolution of radar scanning.
[0090] (4) The present invention improves the system's ability to reconstruct high-quality images under weak scattering conditions through the application of the DT inversion algorithm. Combined with the RMA imaging technology, the system can maintain imaging clarity and accuracy in dynamic environments. The pyramid image fusion technology enhances the image details and contrast and improves the image quality by combining different resolution levels. The image fusion technology is used to improve the detection accuracy of the system in different environments and the adaptability of the system in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0091] Figure 1 This is a schematic diagram of a millimeter wave radar security inspection method based on deep learning and imaging inversion technology disclosed in an embodiment of the present invention. DETAILED DESCRIPTION
[0092] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0093] Example 1
[0094] like Figure 1 As shown, the present invention provides a millimeter wave radar security inspection method based on deep learning and imaging inversion technology, comprising the following steps:
[0095] S1, FMCW module generates radar signal, the transmitting switch array opens the transmitting antenna array and transmits the radar signal; wherein, the FMCW module is composed of crystal oscillator, low phase noise source, amplification filter module, mixer, etc. The crystal oscillator outputs 100MHz signal, generates 3.2GHz signal through low phase noise source, and then mixes with the signal input by external DDS, and finally outputs 27GHz~34GHz signal. The signal received by the FMCW module is mixed with the transmitting signal through the delay module to generate 6.5MHz~7.8MHz intermediate frequency signal for subsequent signal processing. The switch array includes a transmitting switch array and a receiving switch array, and the antenna array includes a transmitting antenna array and a receiving antenna array. The transmitting switch array is used to control the start and stop of the transmitting antenna array, and the receiving switch array is used to control the start and stop of the receiving antenna array. The transmitting antenna array is used to transmit radar signals, and the receiving antenna array is used to receive the signal reflected by the object to be measured. The configuration principle of the switch array is to ensure the isolation between the transmitting and receiving switches, and the received echo power is strong and the signal-to-noise ratio is high. The switch array uses a horizontal metal plate as a reference plane to ensure that the path distance of the signal through the transceiver switch is consistent.
[0096] S2, the servo scanning control system adjusts the direction of the antenna according to the preset scanning scheme to ensure that the radar beam can cover the entire detection area. During the scanning process, the servo scanning control system controls the antenna array to perform precise mechanical movement;
[0097] S3, the receiving switch array turns on the receiving antenna array, the signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module, the FMCW module down-converts the received signal to obtain an intermediate frequency signal, the intermediate frequency signal contains distance and speed information; the intermediate frequency signal is converted into a digital signal through digital-to-analog conversion, and the digital signal is processed digitally to obtain a processed signal, and the digital signal processing includes filtering, amplification and feature extraction.
[0098] S4, using the DT algorithm and the RMA algorithm to process the processed signal to obtain an image of the object to be measured;
[0099] Scattering occurs when electromagnetic waves encounter objects with different electromagnetic properties. The properties of the scattered waves (such as amplitude, phase, direction) vary depending on the electromagnetic properties of the object. Scattering data is the result of the interaction between the electromagnetic waves and the object, and can be used to infer the properties of the object. This data usually includes information about the amplitude and phase of the scattered waves.
[0100] Diffraction Tomography (DT) is a linear method for solving electromagnetic inversion problems, and can calculate the relationship between the retrievable information content and the measurement configuration, which is of great significance to the inversion system involved. The DT algorithm of the present invention is based on the weak scattering assumption, and the contrast space spectrum at a specific frequency is a linear mapping of the scattered field data. First, a coordinate transformation is performed on the multi-frequency scattering data, then the dielectric constant and conductivity are decoupled and the aliased degree space spectrum is modified, and finally the objective function is reconstructed to achieve higher quantitative inversion accuracy. The processing process of the DT algorithm includes:
[0101] The scattered field of the object to be measured at position r in the signal processed in step S3 is expressed by the following integral equation:
[0102] u s (r) = ∫ V f(r′)G(r,r′)dr′
[0103] Among them, u s (r) is the scattered field of the object to be measured at position r, r′ is the position of the source point, f(r′) is the scattering potential of the object to be measured, G(r,r′) is the Green's function, and V is the volume of the object to be measured;
[0104] The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as:
[0105] f(r)=k 2 (∈(r)-∈0)+iωσ(r)
[0106] Where k is the wave number, ∈(r) and σ(r) are the dielectric constant and conductivity of the object to be measured at position r, ∈0 is the background dielectric constant, and ω is the angular frequency;
[0107] By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured are obtained. The solution process of the integral equation of the scattered field is as follows:
[0108] 1) Perform Fourier transform on the integral equation of the scattered field, then multiply it by the spatial spectrum under multiple frequencies to obtain the compensated contrast spatial spectrum, interpolate the contrast spatial spectrum, perform inverse Fourier transform on the contrast spatial spectrum after interpolation, and obtain the decoupled relative dielectric constant and conductivity under single frequency. The specific process is as follows:
[0109] a. Perform Fourier transformation on the integral equation of the scattered field, then multiply it by the spatial spectrum at multiple frequencies to obtain the compensated contrast spatial spectrum:
[0110]
[0111] Among them, γ resis the spatial spectrum coordinate of the receiving antenna, γ′ res is the spatial spectrum coordinate of the transmitting antenna, represents the spatial spectrum of the scattered field, ε0 and μ0 are the dielectric constant and conductivity in free space, respectively, I(ω) represents the spectrum of the excitation source, and z′ represents the spatial coordinate of the antenna array in the z direction.
[0112] b. Perform one-dimensional interpolation operation in the distance direction of the contrast space spectrum;
[0113] c. The contrast spatial spectrum after interpolation is subjected to inverse Fourier transform (first inversion) and decoupled to obtain the decoupled relative dielectric constant and conductivity under single frequency.
[0114] 2) Perform two-dimensional Fourier transform on the antenna array signal to obtain relative permittivity spatial spectrum and conductivity spatial spectrum, perform coherent accumulation on the relative permittivity spatial spectrum and conductivity spatial spectrum, perform two-dimensional inverse Fourier transform on the relative permittivity spatial spectrum and conductivity spatial spectrum after coherent accumulation, and obtain decoupled relative permittivity and conductivity under multiple frequencies; the specific process is as follows:
[0115] d. Perform a two-dimensional Fourier transform on the antenna array signal to obtain N f The spatial spectrum of relative permittivity at each frequency and conductivity spatial spectrum Performing two-dimensional Fourier transform on the antenna array signal refers to performing Fourier transform on both the transmitting antenna array and the receiving antenna array.
[0116] e. Spatial spectrum of relative dielectric constant and conductivity spatial spectrum Perform coherent accumulation;
[0117] f. Perform two-dimensional Fourier inverse transform (second inversion) on the relative permittivity and conductivity spatial spectrum after coherent accumulation to obtain the decoupled relative permittivity and conductivity at multiple frequencies:
[0118]
[0119]
[0120] Among them, x is the spatial coordinate of the object to be measured in the x direction, z is the spatial coordinate of the object to be measured in the z direction, To estimate the relative permittivity, ε rb is the relative permittivity of the free space background, is the relative permittivity of multi-frequency decoupling, K x is the x-direction spectral domain coordinate of the object to be measured, K z is the spectral domain coordinate of the object to be measured in the z direction, is the decoupled conductivity at multiple frequencies, ωn is the nth angular frequency.
[0121] Therefore, according to the relative permittivity and conductivity decoupled under a single frequency and the relative permittivity and conductivity decoupled under multiple frequencies, the complete relative permittivity and conductivity information of the object to be measured can be obtained.
[0122] The Range Migration Algorithm (RMA) is a method used in Synthetic Aperture Radar (SAR) imaging to process radar data and create high-resolution images. The RMA algorithm process includes:
[0123] S401, preprocess the signal S(t) processed by S3 to obtain a preprocessed signal S pre (t)
[0124] S pre (t) = S(t)·W(t)
[0125] Where W(t) is the window function;
[0126] S402, preprocessing signal S pre (t) Perform distance compression:
[0127] S range (f r )=FFT{S pre (t)}·H r (f r )
[0128] Among them, S range (f r ) is the signal after range compression, FFT stands for Fourier transform, H r (f r ) is a distance compression filter;
[0129] S403, performing azimuth compression on the distance-compressed signal:
[0130] S final (f r ,f a )=IFFT{S range (f r )·H a (f a )}
[0131] Among them, S final (f r ,f a ) is the signal after compression processing, IFFT stands for inverse Fourier transform, H a (f a) is an azimuth compression filter;
[0132] S404. The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal:
[0133] I(x,y)=IFFT{S final (f r ,f a )}
[0134] Where I(x,y) represents the intensity of each pixel in the image;
[0135] S405 , finally inverting the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
[0136] S5, performing detail fusion on the image of the object to be measured by using a pyramid fusion algorithm to obtain an optimized image;
[0137] The pyramid fusion algorithm uses Gaussian pyramid and Laplacian pyramid to output fused images. Gaussian pyramid is used to generate multi-scale representation of the image, and each level is created by downsampling layer by layer and applying Gaussian filtering. Laplacian pyramid is composed of the difference of Gaussian pyramid layers, capturing the details in the image. By fusing the Laplacian pyramid layers of different images and then reconstructing the final fused image, the details and features in the original images can be better preserved. The processing process of the pyramid fusion algorithm is as follows:
[0138] S501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample (reduce the size) to generate the next layer, which can be expressed as
[0139] I next =Downsample(GaussianBlur(I current ))
[0140] Among them, I next is the image of the next layer of the Gaussian pyramid, I current is the current layer image of the Gaussian pyramid, GaussianBlur is Gaussian filtering, and Downsample is downsampling; the number of layers of the Gaussian pyramid can be set as needed.
[0141] S502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain the Laplacian layer, which can be expressed as
[0142] L i =G i-Upsample(G i+1 )
[0143] Among them, L i represents the i-th Laplacian layer of the Laplacian pyramid, G i+1 Represents the current level of the Gaussian pyramid, G i Indicates the previous layer of the Gaussian pyramid, and Upsample indicates upsampling;
[0144] S503, merging Laplacian pyramids: performing average fusion on the Laplacian pyramids at each level;
[0145] S504, image reconstruction: reconstructing the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
[0146] S6. Input the optimized image into the YOLOv8 model and output the classification result of the object to be tested.
[0147] S7. The detection and recognition results of the object to be tested are displayed on a computer display screen, showing the position and category of the detected object to be tested.
[0148] Through the above technical scheme, the present invention first extracts the intensity, dielectric constant and conductivity information of the object from the radar data through the radar inversion algorithm. This information is stored in the corresponding matrix. Then, these matrices are normalized and scaled to the same size. Next, each matrix is processed using the Gaussian pyramid and Laplacian pyramid algorithms to create a multi-scale image representation. The Gaussian pyramid is generated by applying Gaussian filtering and downsampling layer by layer, while the Laplacian pyramid is constructed by extracting detail information from each layer of the Gaussian pyramid. Finally, using these pyramids, images with different physical properties are fused at each level, and then the fused pyramid data is used to reconstruct the final image. This method allows a comprehensive and clear image representation to be obtained from multiple physical dimensions. The overall scheme combines the Gaussian pyramid and Laplacian pyramid techniques, not only providing a more comprehensive view when fusing intensity, dielectric constant and conductivity images, but also enhancing the overall structure and large-scale features of the image through the Gaussian pyramid, while the Laplacian pyramid highlights high-frequency information such as edges and details, significantly improving image clarity. This multi-scale fusion method achieves an effective balance between details and overall features, so that the final image is not only improved in clarity, but also superior to traditional methods in terms of content richness and accuracy. Overall, the present invention provides a more efficient and accurate radar image processing and analysis solution, which is suitable for a variety of complex application scenarios.
[0149] Example 2
[0150] Based on Example 1, Example 2 of the present invention further provides a millimeter wave radar security inspection system based on deep learning and imaging inversion technology, including:
[0151] The signal transmitting unit is used for the FMCW module to generate radar signals, and the transmitting switch array turns on the transmitting antenna array to transmit the radar signal;
[0152] Scan control unit, used to servo the scan control system to adjust the direction of the antenna to ensure that the radar beam covers the entire detection area;
[0153] The signal receiving and processing unit is used to receive the switch array to open the receiving antenna array. The signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module. The FMCW module down-converts the received signal to obtain an intermediate frequency signal. The intermediate frequency signal is converted into a digital signal through digital-to-analog conversion. The digital signal is processed into a processed signal through digital signal processing.
[0154] An image inversion unit is used to process the processed signal using a DT algorithm and an RMA algorithm to obtain an image of the object to be measured;
[0155] An image optimization unit, used for fusing details of the image of the object to be measured through a pyramid fusion algorithm to obtain an optimized image;
[0156] The object detection unit is used to input the optimized image into the YOLOv8 model and output the classification result of the object to be tested.
[0157] Specifically, the processing process of the DT algorithm includes:
[0158] The scattered field of the object to be measured at position r in the signal processed by the signal receiving and processing unit is expressed by the following integral equation:
[0159] u s (r) = ∫ V f(r′)G(r,r′)dr′
[0160] Among them, u s (r) is the scattered field of the object to be measured at position r, r′ is the position of the source point, f(r′) is the scattering potential of the object to be measured, G(r,r′) is the Green's function, and V is the volume of the object to be measured;
[0161] The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as:
[0162] f(r)=k 2 (∈(r)-∈0)+iωσ(r)
[0163] Where k is the wave number, ∈(r) and σ(r) are the dielectric constant and conductivity of the object to be measured at position r, ∈0 is the background dielectric constant, and ω is the angular frequency;
[0164] By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured can be obtained.
[0165] More specifically, the processing of the RMA algorithm includes:
[0166] Step 401: preprocess the signal S(t) processed by the signal receiving and processing unit to obtain a preprocessed signal S pre (t)
[0167] S pre (t) = S(t)·W(t)
[0168] Where W(t) is the window function;
[0169] Step 402: preprocess the signal S pre (t) Perform distance compression:
[0170] S range (f r )=FFT{S pre (t)}·H r (f r )
[0171] Among them, S range (f r ) is the signal after range compression, FFT stands for Fourier transform, H r (f r ) is a distance compression filter;
[0172] Step 403: perform azimuth compression on the range-compressed signal:
[0173] S final (f r ,f a )=IFFT{S range (f r )·H a (f a )}
[0174] Among them, S final (f r ,f a ) is the signal after compression processing, IFFT stands for inverse Fourier transform, H a (f a ) is an azimuth compression filter;
[0175] Step 404: The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal:
[0176] I(x,y)=IFFT{S final (f r ,f a )}
[0177] Where I(x,y) represents the intensity of each pixel in the image;
[0178] Step 405: Invert the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
[0179] More specifically, the processing process of the pyramid fusion algorithm is as follows:
[0180] Step 501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample to generate the next layer, which can be expressed as
[0181] I next =Downsample(GaussianBlur(I current ))
[0182] Among them, I next is the image of the next layer of the Gaussian pyramid, I current is the current layer image of the Gaussian pyramid, GaussianBlur is Gaussian filtering, and Downsample is downsampling;
[0183] Step 502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain a Laplacian layer, which can be expressed as
[0184] L i =G i -Upsample(G i+1 )
[0185] Among them, L i represents the i-th Laplacian layer of the Laplacian pyramid, G i+1 Represents the current level of the Gaussian pyramid, G i Indicates the previous layer of the Gaussian pyramid, and Upsample indicates upsampling;
[0186] Step 503, merging Laplacian pyramids: averaging and merging the Laplacian pyramids at each level;
[0187] Step 504, image reconstruction: reconstruct the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
[0188] Specifically, the digital signal processing includes filtering, amplification and feature extraction.
[0189] Specifically, the millimeter wave radar security inspection system further includes a data display unit, and the data display unit is used to:
[0190] The detection and recognition results of the objects to be tested are displayed on a computer display screen, showing the location and category of the detected objects to be tested.
[0191] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A millimeter wave radar security inspection method based on deep learning and imaging inversion technology, characterized in that: The following steps are involved: Step 1: The FMCW module generates a radar signal, and the transmitting switch array turns on the transmitting antenna array to transmit the radar signal; Step 2: The servo scanning control system adjusts the direction of the antenna to ensure that the radar beam covers the entire detection area; Step 3: The receiving switch array turns on the receiving antenna array. The signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module. The FMCW module down-converts the received signal to obtain an intermediate frequency signal. The intermediate frequency signal is converted into a digital signal through digital-to-analog conversion, and the digital signal is processed to obtain a processed signal; Step 4: Process the processed signal using the DT algorithm and the RMA algorithm to obtain an image of the object to be measured; Step 5: The image of the object to be tested is subjected to detail fusion through a pyramid fusion algorithm to obtain an optimized image; Step 6: Input the optimized image into the YOLOv8 model and output the classification result of the object to be tested; The processing of the DT algorithm includes: The position of the object to be measured in the signal processed in step 3 The scattered field is expressed by the following integral equation: in, is the position of the object to be measured The scattered field of is the location of the source, is the scattering potential of the object to be measured, is the Green's function, is the volume of the object to be measured; The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as: in, is the wave number, and are the dielectric constant and conductivity of the object to be measured at position r, is the background dielectric constant, is the angular frequency; By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured are obtained; The integral equation of the scattered field is solved as follows: 1) Perform Fourier transform on the integral equation of the scattered field, then multiply it by the spatial spectrum under multiple frequencies to obtain the compensated contrast spatial spectrum, interpolate the contrast spatial spectrum, perform inverse Fourier transform on the contrast spatial spectrum after interpolation, and obtain the decoupled relative dielectric constant and conductivity under single frequency. The specific process is as follows: a. Perform Fourier transformation on the integral equation of the scattered field, then multiply it by the spatial spectrum at multiple frequencies to obtain the compensated contrast spatial spectrum: in, is the spatial spectrum coordinate of the receiving antenna, is the spatial spectrum coordinate of the transmitting antenna, represents the spatial spectrum of the scattered field, and are the dielectric constant and conductivity in free space, represents the excitation source spectrum, Represents the spatial domain coordinates of the antenna array in the z direction; b. Perform one-dimensional interpolation operation in the distance direction of the contrast space spectrum; c. Perform inverse Fourier transform on the contrast spatial spectrum after interpolation, and decouple to obtain the relative permittivity and conductivity decoupled under single frequency; 2) Perform two-dimensional Fourier transform on the antenna array signal to obtain relative permittivity spatial spectrum and conductivity spatial spectrum, perform coherent accumulation on the relative permittivity spatial spectrum and conductivity spatial spectrum, perform two-dimensional inverse Fourier transform on the relative permittivity spatial spectrum and conductivity spatial spectrum after coherent accumulation, and obtain the decoupled relative permittivity and conductivity at multiple frequencies; the specific process is as follows: d. Perform a two-dimensional Fourier transform on the antenna array signal to obtain The spatial spectrum of relative permittivity at each frequency and conductivity spatial spectrum , ; Performing a two-dimensional Fourier transform on the antenna array signal means performing a Fourier transform on both the transmitting antenna array and the receiving antenna array; e. Spatial spectrum of relative dielectric constant and conductivity spatial spectrum Perform coherent accumulation; f. Perform a two-dimensional Fourier inverse transform on the relative permittivity and conductivity spatial spectrum after coherent accumulation to obtain the decoupled relative permittivity and conductivity at multiple frequencies: in, is the x-direction spatial coordinate of the object to be measured, is the spatial coordinate of the object to be measured in the z direction, To estimate the relative permittivity, is the relative permittivity of the free space background, is the relative permittivity decoupled at multiple frequencies, is the spectral domain coordinate of the object to be measured in the x direction, is the spectral domain coordinate of the object to be measured in the z direction, is the decoupled conductivity at multiple frequencies, is the nth angular frequency; Therefore, according to the relative permittivity and conductivity decoupled under a single frequency and the relative permittivity and conductivity decoupled under multiple frequencies, the complete relative permittivity and conductivity information of the object to be measured can be obtained.
2. The millimeter wave radar security inspection method based on deep learning and imaging inversion technology according to claim 1 is characterized in that: The processing of the RMA algorithm includes: Step 401: the signal processed in step 3 Perform preprocessing to obtain preprocessed signal in, is the window function; Step 402: preprocess the signal To perform distance compression: in, is the signal after distance compression, represents the Fourier transform, is the range compression filter; Step 403: perform azimuth compression on the range-compressed signal: in, It is the signal after compression processing, IFFT means inverse Fourier transform, is the azimuth compression filter; Step 404: The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal: in, Represents the intensity of each pixel in the image; Step 405: Invert the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
3. The millimeter wave radar security inspection method based on deep learning and imaging inversion technology according to claim 2 is characterized in that: The processing process of the pyramid fusion algorithm is as follows: Step 501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample to generate the next layer, which can be expressed as in, is the image of the next layer of the Gaussian pyramid, is the current layer image of the Gaussian pyramid, is a Gaussian filter, is downsampling; Step 502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain a Laplacian layer, which can be expressed as in, represents the i-th Laplacian layer of the Laplacian pyramid, represents the current level of the Gaussian pyramid, represents the previous layer of the Gaussian pyramid, represents upsampling; Step 503, merging Laplacian pyramids: averaging and merging the Laplacian pyramids at each level; Step 504, image reconstruction: reconstruct the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
4. The millimeter wave radar security inspection method based on deep learning and imaging inversion technology according to claim 1 is characterized in that: The digital signal processing includes filtering, amplification and feature extraction.
5. The millimeter wave radar security inspection method based on deep learning and imaging inversion technology according to claim 1 is characterized in that: Also includes step seven: The detection and recognition results of the objects to be tested are displayed on a computer display screen, showing the location and category of the detected objects to be tested.
6. The millimeter wave radar security inspection system based on deep learning and imaging inversion technology is characterized by: include: The signal transmitting unit is used for the FMCW module to generate radar signals, and the transmitting switch array turns on the transmitting antenna array to transmit the radar signals; Scan control unit, used to servo the scan control system to adjust the direction of the antenna to ensure that the radar beam covers the entire detection area; A signal receiving and processing unit is used to receive the switch array to turn on the receiving antenna array. The signal reflected by the object to be measured is received by the receiving antenna array and transmitted to the FMCW module. The FMCW module down-converts the received signal to obtain an intermediate frequency signal. The intermediate frequency signal is converted into a digital signal through digital-to-analog conversion, and the digital signal is processed to obtain a processed signal; An image inversion unit is used to process the processed signal using a DT algorithm and an RMA algorithm to obtain an image of the object to be measured; An image optimization unit, used for fusing details of the image of the object to be measured through a pyramid fusion algorithm to obtain an optimized image; The object detection unit is used to input the optimized image into the YOLOv8 model and output the classification result of the object to be tested; The processing of the DT algorithm includes: The position of the object to be measured in the signal processed by the signal receiving and processing unit The scattered field is expressed by the following integral equation: in, is the position of the object to be measured The scattered field of is the location of the source, is the scattering potential of the object to be measured, is the Green's function, is the volume of the object to be measured; The scattering potential is related to the dielectric constant and conductivity of the object to be measured and is expressed as: in, is the wave number, and are the dielectric constant and conductivity of the object to be measured at position r, is the background dielectric constant, is the angular frequency; By solving the integral equation of the scattered field, the dielectric constant and conductivity of the object to be measured are obtained; The integral equation of the scattered field is solved as follows: 1) Perform Fourier transform on the integral equation of the scattered field, then multiply it by the spatial spectrum under multiple frequencies to obtain the compensated contrast spatial spectrum, interpolate the contrast spatial spectrum, perform inverse Fourier transform on the contrast spatial spectrum after interpolation, and obtain the decoupled relative dielectric constant and conductivity under single frequency. The specific process is as follows: a. Perform Fourier transformation on the integral equation of the scattered field, then multiply it by the spatial spectrum at multiple frequencies to obtain the compensated contrast spatial spectrum: in, is the spatial spectrum coordinate of the receiving antenna, is the spatial spectrum coordinate of the transmitting antenna, represents the spatial spectrum of the scattered field, and are the dielectric constant and conductivity in free space, represents the excitation source spectrum, Represents the spatial domain coordinates of the antenna array in the z direction; b. Perform one-dimensional interpolation operation in the distance direction of the contrast space spectrum; c. Perform inverse Fourier transform on the contrast spatial spectrum after interpolation, and decouple to obtain the relative permittivity and conductivity decoupled under single frequency; 2) Perform two-dimensional Fourier transform on the antenna array signal to obtain relative permittivity spatial spectrum and conductivity spatial spectrum, perform coherent accumulation on the relative permittivity spatial spectrum and conductivity spatial spectrum, perform two-dimensional inverse Fourier transform on the relative permittivity spatial spectrum and conductivity spatial spectrum after coherent accumulation, and obtain the decoupled relative permittivity and conductivity at multiple frequencies; the specific process is as follows: d. Perform a two-dimensional Fourier transform on the antenna array signal to obtain The spatial spectrum of relative permittivity at each frequency and conductivity spatial spectrum , ; Performing a two-dimensional Fourier transform on the antenna array signal means performing a Fourier transform on both the transmitting antenna array and the receiving antenna array; e. Spatial spectrum of relative dielectric constant and conductivity spatial spectrum Perform coherent accumulation; f. Perform a two-dimensional Fourier inverse transform on the relative permittivity and conductivity spatial spectrum after coherent accumulation to obtain the decoupled relative permittivity and conductivity at multiple frequencies: in, is the x-direction spatial coordinate of the object to be measured, is the spatial coordinate of the object to be measured in the z direction, To estimate the relative permittivity, is the relative permittivity of the free space background, is the relative permittivity decoupled at multiple frequencies, is the spectral domain coordinate of the object to be measured in the x direction, is the spectral domain coordinate of the object to be measured in the z direction, is the decoupled conductivity at multiple frequencies, is the nth angular frequency; Therefore, according to the relative permittivity and conductivity decoupled under a single frequency and the relative permittivity and conductivity decoupled under multiple frequencies, the complete relative permittivity and conductivity information of the object to be measured can be obtained.
7. The millimeter wave radar security inspection system based on deep learning and imaging inversion technology according to claim 6 is characterized in that: The processing of the RMA algorithm includes: Step 401: receiving and processing the signal processed by the signal receiving and processing unit Perform preprocessing to obtain preprocessed signal in, is the window function; Step 402: preprocess the signal To perform distance compression: in, is the signal after distance compression, represents the Fourier transform, is the range compression filter; Step 403: perform azimuth compression on the range-compressed signal: in, It is the signal after compression processing, IFFT means inverse Fourier transform, is the azimuth compression filter; Step 404: The signal after compression is subjected to inverse Fourier transform to obtain the reflection intensity of the signal: in, Represents the intensity of each pixel in the image; Step 405: Invert the image of the object to be measured according to the dielectric constant and conductivity of the object to be measured and the reflection intensity of the signal.
8. The millimeter wave radar security inspection system based on deep learning and imaging inversion technology according to claim 7 is characterized in that: The processing process of the pyramid fusion algorithm is as follows: Step 501, Gaussian pyramid construction: input the image of the object to be tested into the first layer of the Gaussian pyramid, perform Gaussian filtering on the current layer of the Gaussian pyramid, and then downsample to generate the next layer, which can be expressed as in, is the image of the next layer of the Gaussian pyramid, is the current layer image of the Gaussian pyramid, is a Gaussian filter, is downsampling; Step 502, Laplacian pyramid construction: Upsample each layer of the Gaussian pyramid, and subtract the upsampled layer from the previous layer of the Gaussian pyramid to obtain a Laplacian layer, which can be expressed as in, represents the i-th Laplacian layer of the Laplacian pyramid, represents the current level of the Gaussian pyramid, represents the previous layer of the Gaussian pyramid, represents upsampling; Step 503, merging Laplacian pyramids: averaging and merging the Laplacian pyramids at each level; Step 504, image reconstruction: reconstruct the final image using the average fusion results of each level of the Laplacian pyramid to obtain an optimized image of the object to be measured.
