Face Recognition Method and Device Based on Millimeter-Wave Radar Phase Features

Abstract

This invention discloses a face recognition method and apparatus based on millimeter-wave radar phase features. The method includes: acquiring face radar echo signals; selecting target points in the frequency domain; using the target point data as data to be processed containing face information; extracting phase features from the data to be processed; performing a deconvolution operation on the phase features to obtain the phase change difference of the target points; performing 3D convolution and max pooling on the phase change differences of all target points to obtain dimensionality-reduced phase features; downsampling the dimensionality-reduced phase features using a convolutional autoencoder; and inputting the downsampled data into a convolutional neural network to achieve face recognition. Unlike vision-based face recognition, this invention uses millimeter-wave radar, which has higher resolution, is not limited by lighting conditions, and can effectively protect user privacy.

Description

Technical Field

[0001] This invention relates to the field of radar signal data, and in particular to a face recognition method and apparatus based on millimeter-wave radar phase features. Background Technology

[0002] With the continuous development and progress of information technology, the research on identity recognition technology has gradually shifted from traditional keys, fingerprints, and vision towards automation, non-intrusiveness, and persistence. The growing security issues in various mobile applications and smart devices also urgently require a reliable and convenient user verification method to protect user privacy from being leaked.

[0003] Currently, the most widely used method of identity recognition is vision-based facial recognition. This type of research uses traditional optical cameras to capture facial images and then performs a series of processing steps, such as image segmentation and classification, to achieve the effect of identity recognition. However, vision-based methods have high requirements for image quality, consume a lot of power from the cameras, and have high requirements for lighting conditions. Furthermore, the facial image data involves important user privacy, and any leakage could have serious consequences.

[0004] Compared to vision-based face recognition, millimeter-wave radar face recognition has the following advantages: (1) Radar transmits and receives echo signals with very low light requirements, making it widely applicable to various scenarios. (2) It has low power consumption and can work around the clock. (3) Compared to image signals, radar echo signals have a smaller data volume, making them easier to process and effectively protecting user privacy. Summary of the Invention

[0005] In view of this, the present invention provides a face recognition method and device based on millimeter-wave radar phase features, which can solve the problems of current vision-based face recognition having high requirements for lighting conditions and privacy and security risks.

[0006] To address the aforementioned technical problems, this invention proposes the following technical solution: a face recognition method based on millimeter-wave radar phase features, comprising the following steps:

[0007] Step 1: Read the radar echo signal reflected back from the face received by the millimeter-wave radar sensor, and simulate the acquisition process of the face at different distances from the radar and at different angles to the radar.

[0008] Step 2: For single-channel face radar echo signals, obtain the range Doppler map through Fourier transform, select several points near the intensity peak point, and use them together with the peak point as target points. Use the target point data as the data to be processed containing face information.

[0009] Step 3: Extract the phase features from the data to be processed, and perform an unwinding operation on the phase features to obtain the phase change difference of the target point in the channel;

[0010] Step 4: Perform 3D convolution and max pooling on the phase change difference of all channel target points to obtain the dimensionality-reduced phase features;

[0011] Step 5: Downsample the phase features obtained from step 4 using a convolutional autoencoder;

[0012] Step 6: Input the downsampled data from Step 5 into the convolutional neural network to achieve face recognition.

[0013] Furthermore, in step 1, the acquisition process is specifically as follows: acquisition is carried out in multiple scenarios at 24cm 0°, 30cm 0°, 30cm+15°, 30cm+24°, 30cm+15°, 30cm 0°, 30cm-15°, 30cm-24°, 30cm-15°, 30cm 0°, and 50cm 0°. That is, the subject is first 24cm away from the radar, then moves to 30cm away, rotates his head from facing the radar to the right, then rotates from right to left and returns to the center, and finally moves to 50cm away.

[0014] Furthermore, in step 2, the method for selecting target points is as follows: select 8 points, which are 3 points before and 4 points after the intensity peak point on the Doppler image.

[0015] Furthermore, in step 2, if there are multiple intensity peak points, the peak point closest to the radar is selected as the target point, and the other peak points are regarded as noise interference.

[0016] Furthermore, in step 2, if the radar sensor has multiple receiving channels, the same data extraction operation is performed on all the obtained face radar echo signals.

[0017] Furthermore, in step 3, when performing the unwinding operation, the phase of the specified target point is used as the initial phase of all target points to obtain the phase change difference of all target points; for example, the phase of the target point closest to the radar can be used as the initial phase.

[0018] Furthermore, in step 5, the convolutional autoencoder adopts a 3D-CNN structure, which includes an input layer, a 3D convolutional layer, a max pooling layer, a max upsampling layer, and a transposed convolutional layer connected in sequence.

[0019] Furthermore, in step 6, the convolutional neural network adopts a 2D-CNN structure, including an input layer and a 2D convolutional module and a 2D dilated convolutional module connected to the input layer. The 2D dilated convolutional module can expand the receptive field and take into account both overall features and local features. The outputs of the 2D convolutional module and the 2D dilated convolutional module are concatenated and then flattened, and finally the face category is output through a fully connected layer.

[0020] The present invention also provides a face recognition device based on millimeter-wave radar phase features, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the above-mentioned face recognition method based on millimeter-wave radar phase features.

[0021] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the above-described face recognition method based on millimeter-wave radar phase features.

[0022] The beneficial effects of this invention are as follows: the face recognition method based on millimeter-wave radar phase features proposed in this invention is not affected by lighting conditions; the data volume is small and easy to process; it effectively protects user privacy, has a certain degree of penetration, and solves the pain points of traditional identity recognition. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method provided in an embodiment of the present invention;

[0024] Figure 2 This is a structural diagram of a convolutional autoencoder provided in an embodiment of the present invention;

[0025] Figure 3 This is a diagram of the convolutional neural network structure provided in an embodiment of the present invention;

[0026] Figure 4 This is the test set confusion matrix provided in the embodiments of the present invention;

[0027] Figure 5 This is a structural diagram of the device provided in an embodiment of the present invention. Detailed Implementation

[0028] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0029] Without loss of generality, this embodiment provides a face recognition method based on millimeter-wave radar phase features. A 60GHz millimeter-wave radar was used to perform face recognition on a total of 196 individuals. Figure 1As shown, it includes the following steps.

[0030] Step 1: Read the radar echo signal reflected back from the face received by the millimeter-wave radar sensor. The radar device consists of two vertically stacked antenna arrays; the bottom array is used to transmit signals, and the top array is used to receive signals. This radar array can record 32*32=1024 channels of radar echo signals.

[0031] The study simulated the acquisition process of a human face at different distances from a radar and at different angles to it. Data was collected in multiple scenarios: 24cm 0°, 30cm 0°, 30cm+15°, 30cm+24°, 30cm+15°, 30cm 0°, 30cm-15°, 30cm-24°, 30cm-15°, 30cm 0°, and 50cm 0°. Specifically, the subject first stood 24cm from the radar, then moved to 30cm, rotating their head from facing the radar to the right, then from right to left back to the center, and finally moving to 50cm. 200 frames of radar echo data were collected for each subject in each scenario, resulting in a sample size of 200×11×32×32 for each subject.

[0032] Step 2: For single-channel face radar echo signals, obtain the range Doppler map through Fourier transform. Take the first 3 points and the last 4 points of the intensity peak points on the range Doppler map, a total of 8 points, as target points. Use the target point data as the data to be processed containing face information.

[0033] Step 3: Extract the phase features from the data to be processed, and perform an unwinding operation on the phase features to obtain the phase change difference of the target point in that channel. The original radar signal suffers from phase winding during the calculation process. Let the unwound phase be... The true phase is φ(x), and can be expressed as:

[0034]

[0035]

[0036] Using the phase of the target point closest to the radar as the initial phase, the phase difference of all target points is obtained. The phase after unwinding is a relative quantity, which can remove the error of absolute distance, thus more accurately reflecting the depth of each position of the face.

[0037] Step 4: Perform 3D convolution and max pooling on the phase change differences of all target points across all channels. There are a total of 1024 radar echo signals. Combining the phase change differences of all target points across these 1024 channels yields a 32×32×8 dimension sample of phase change difference data. It was observed that very little of this data actually contains significant information. Therefore, 3D convolution and max pooling are further performed on the phase change difference sample data to reduce the sample size from 32×32×8 to 32×32×3. The purpose of 3D convolution and max pooling is to filter out data containing redundant information, reducing the amount of data while retaining feature information.

[0038] Step 5: For each sample data obtained in Step 4, downsample it using a convolutional autoencoder to represent it with a smaller dimension. For example... Figure 2 As shown, the convolutional autoencoder consists of an encoder and a decoder. The main network adopts a 3D-CNN structure, including an input layer, a 3D convolutional layer, a max pooling layer, a max upsampling layer, and a transposed convolutional layer connected in sequence. The input samples simulate the movement of a subject's face towards the radar. There is correlation information between these continuous radar images, and an additional dimension is added. Using 3D convolution can effectively extract the correlation information between the images.

[0039] Step 6: Input the downsampled data obtained in Step 5 into a convolutional neural network for classification and recognition. The convolutional neural network adopts a 2D-CNN structure. CNN processing can reduce the impact of subtle human movements during data acquisition. Figure 3 As shown, the network framework includes an input layer and a 2D convolutional module and a 2D dilated convolutional module connected to the input layer. The 2D dilated convolutional module can expand the receptive field and take into account both overall and local features. The outputs of the 2D convolutional module and the 2D dilated convolutional module are concatenated and then flattened. Finally, the face category is output through a fully connected layer.

[0040] In this implementation example, data from 196 subjects were processed using the radar echo signals as described above. Each subject had 200 data points, resulting in a total of 196 × 200 = 39,200 data points. This dataset was randomly divided into a training set and a test set at an 8:2 ratio, with 31,360 samples in the training set and 7,840 samples in the test set. The final accuracy was 99.7%, and the 20-person confusion matrix is ​​shown below. Figure 4 As shown.

[0041] To verify the effectiveness of the method of this invention, this embodiment compares different feature information extraction methods with different classification algorithms. The extraction methods include extracting amplitude and phase from radar echoes, and the classification algorithms include random forest and AlexNet. The final results are shown in Table 1:

[0042] Table 1 Comparison of accuracy of different feature extraction and classification algorithms

[0043]

[0044] In summary, the face recognition method based on millimeter-wave radar phase features provided by this invention can solve the problems of high requirements for lighting conditions and privacy and security risks of current vision-based face recognition.

[0045] Corresponding to the aforementioned embodiments of the face recognition method based on millimeter-wave radar phase features, the present invention also provides embodiments of a face recognition device based on millimeter-wave radar phase features.

[0046] See Figure 5 The present invention provides a face recognition device based on millimeter-wave radar phase features, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the face recognition method based on millimeter-wave radar phase features in the above embodiment.

[0047] The embodiments of the face recognition device based on millimeter-wave radar phase features of this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 5 The diagram shown is a hardware structure diagram of any data processing-capable device, including the millimeter-wave radar phase feature-based face recognition device of the present invention. (Except for...) Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0048] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0049] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0050] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the face recognition method based on millimeter-wave radar phase features described in the above embodiments.

[0051] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0052] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0053] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A face recognition method based on millimeter-wave radar phase features, characterized in that, The method includes the following steps: Step 1: Read the radar echo signal reflected back from the face received by the millimeter-wave radar sensor, and simulate the acquisition process of the face at different distances from the radar and at different angles to the radar. Step 2: For single-channel face radar echo signals, obtain the range Doppler map through Fourier transform, select several points near the intensity peak point, and use them together with the peak point as target points. Use the target point data as the data to be processed containing face information. Step 3: Extract the phase features from the data to be processed, and perform an unwinding operation on the phase features to obtain the phase change difference of the target point in the channel; when performing the unwinding operation, use the phase of the specified target point as the initial phase of all target points to obtain the phase change difference of all target points. Step 4: Perform 3D convolution and max pooling on the phase change difference of all channel target points to obtain the dimensionality-reduced phase features; Step 5: Downsample the phase features after dimensionality reduction in Step 4 using a convolutional autoencoder; the convolutional autoencoder adopts a 3D-CNN structure, including an input layer, a 3D convolutional layer, a max pooling layer, a max upsampling layer, and a transposed convolutional layer connected in sequence. Step 6: Input the downsampled data from Step 5 into a convolutional neural network to achieve face recognition. The convolutional neural network adopts a 2D-CNN structure, including an input layer and a 2D convolutional module and a 2D dilated convolutional module connected to the input layer. The 2D dilated convolutional module can expand the receptive field and take into account both overall and local features. The outputs of the 2D convolutional module and the 2D dilated convolutional module are concatenated and flattened. Finally, the face category is output through a fully connected layer.

2. The face recognition method based on millimeter-wave radar phase features according to claim 1, characterized in that, In step 1, the data collection process is as follows: data is collected in multiple scenarios at 24cm 0°, 30cm 0°, 30cm+15°, 30cm+24°, 30cm+15°, 30cm 0°, 30cm-15°, 30cm-24°, 30cm-15°, 30cm 0°, and 50cm 0°. That is, the subject first stands at a distance of 24cm from the radar, then moves to 30cm, rotates his head from facing the radar to the right, then rotates from right to left and returns to the center, and finally moves to 50cm.

3. The face recognition method based on millimeter-wave radar phase features according to claim 1, characterized in that, In step 2, the method for selecting target points is as follows: select 8 points, which are 3 points before and 4 points after the intensity peak point on the Doppler image.

4. The face recognition method based on millimeter-wave radar phase features according to claim 1, characterized in that, In step 2, if there are multiple intensity peak points, the peak point closest to the radar is selected as the target point, and the other peak points are regarded as noise interference.

5. The face recognition method based on millimeter-wave radar phase features according to claim 1, characterized in that, In step 2, if the radar sensor has multiple receiving channels, the same data extraction operation will be performed on all the multiple face radar echo signals obtained.

6. A face recognition device based on millimeter-wave radar phase features, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that, When the processor executes the executable code, it is used to implement the face recognition method based on millimeter-wave radar phase features as described in any one of claims 1-5.

7. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the face recognition method based on millimeter-wave radar phase features as described in any one of claims 1-5.

Images