A millimeter-wave based face recognition method and device for preventing forgery attacks and resisting occlusion

Abstract

The present invention discloses a face recognition method and device based on millimeter waves that are resistant to counterfeiting attacks and occlusion. The method includes user registration and user authentication. In order to reduce the registration overhead of the wireless face recognition system, the user only needs to provide three facial photos taken at different angles to complete the registration. The method converts a given facial photo into a virtual registration signal by constructing a millimeter wave signal propagation model, thereby avoiding the user from performing tedious on-site registration. The algorithm extracts facial curvature information of the face as a distance-robust facial structural feature based on the SAR imaging results. In three different environments, and with an authentication distance within the range of 10cm-20cm, an authentication success rate of more than 90% can be achieved. The activity detection method can effectively distinguish between real faces and three-dimensional printed masks of different materials, which shows that the method can effectively resist anti-counterfeiting attacks, thereby greatly improving the security of the face recognition system.

Description

Technical Field

[0001] The present invention relates to a millimeter wave-based face recognition method, and in particular to a millimeter wave-based face recognition method and device that is resistant to counterfeiting attacks and occlusion. Background Art

[0002] Due to their high accuracy and user-friendliness, facial recognition systems are widely used in daily life, for example, in access control, mobile payments, and identity authentication. Despite these advantages, existing facial recognition systems are typically camera-based and therefore subject to several limitations. First, camera-based facial recognition systems are susceptible to poor lighting conditions, which limits their practical applicability. Second, with the advancement of artificial intelligence, camera-based approaches are vulnerable to emerging forgery attacks. Although incorporating some liveness information, such as skin texture and facial movements (e.g., blinking), can enhance the security of facial recognition systems, attackers can still deceive existing liveness detection methods using fake videos or 3D-printed masks. Furthermore, the authentication accuracy of camera-based approaches is also affected by facial occlusion. In particular, the COVID-19 pandemic has made wearing surgical masks a daily necessity. In this context, camera-based facial recognition systems cannot effectively handle mask occlusion because they only capture partial facial features.

[0003] With the development of science and technology, millimeter wave sensing technology has been continuously improved and matured. Previous studies have shown that millimeter wave-based sensing has attractive advantages. First, they are able to adapt to complex lighting conditions. At the same time, millimeter wave signals bring new possibilities for anti-occlusion face authentication. Unlike cameras that are susceptible to facial occlusion, the penetrability of millimeter wave signals allows them to pass through obstacles, such as surgical masks, to capture facial features behind them. In addition, the material sensitivity of millimeter wave signals can be used to distinguish real faces from other materials (for example, silicone in 3D printed masks), which is crucial for defending against counterfeit attacks.

[0004] Although millimeter wave signals have the above advantages such as penetration and material sensitivity, there are also several defects that cannot be ignored when using millimeter wave signals to build a face recognition system: (1) Complicated registration process: Signal-based identity authentication systems often require users to register on-site and need to collect a large amount of registration data. Therefore, the cost of user registration is too high, which limits the actual deployment of the system; (2) Not robust to authentication distance: Millimeter wave signals are sensitive to authentication distance. When the distance between the user and the millimeter wave radar is inconsistent during user registration and authentication, there will be a huge difference in the millimeter wave signals collected in the two processes, which will affect the accuracy of user authentication; (3) Difficulty in extracting reliable material properties: Millimeter wave signals are affected by both facial structure and material. Because the structure of the face is complex, it is difficult to extract reliable material properties from the mixed millimeter wave signals to resist forgery attacks. Summary of the Invention

[0005] Therefore, the present invention aims to utilize the penetration and material sensitivity of millimeter waves to build a millimeter-wave-based face recognition system that is unaffected by occlusion and resistant to forgery attacks. This invention leverages the penetration, material sensitivity, and fine-grained sensing capabilities of millimeter waves to build a forgery-resistant and occlusion-resistant face recognition method and system. Millimeter-wave signals are used to capture the material and geometric characteristics of the face to achieve effective face recognition.

[0006] The present invention discloses a face recognition method based on millimeter waves that is resistant to counterfeiting attacks and occlusion. The method includes user registration and user authentication. User registration includes the following steps:

[0007] 1) Obtain at least three face images, collected from different directions;

[0008] 2) Construct a virtual registration scene based on the collected face images;

[0009] 3) Generate a virtual registration signal based on the virtual registration scenario;

[0010] 4) Using the generated virtual registration signal to extract distance-robust facial structural features and save them in the database as a template;

[0011] User authentication includes the following steps:

[0012] 5) Collecting millimeter wave signals reflected by the user’s face;

[0013] 6) Extract facial biometric features based on the collected signals to detect forgery attacks;

[0014] 7) For the detected signal, extract distance-robust facial structural features;

[0015] 8) Compare the obtained facial structure features with the template in the user registration stage to achieve user authentication.

[0016] In step 3), the generation of the virtual registration signal is achieved by constructing a theoretical millimeter wave signal propagation model. The millimeter wave signal propagation model includes three factors for the transmitted millimeter wave signal: The influence of: propagation distance (r), antenna directional characteristics (p) and face reflection characteristics (g), that is, virtual millimeter wave signal Expressed as Where F represents a point on the face.

[0017] As a further improvement, the directional characteristic of the antenna described in the present invention is that the initial signal amplitude of the signal transmitted by the millimeter wave radar in different directions has a large difference, and this directional characteristic is modeled as a Gaussian beam; the Gaussian beam indicates that the initial amplitude (p) of the transmitted signal is determined by the angle between the transmitted signal and the center line of the millimeter wave beam in the horizontal (α) and vertical (β) planes: Among them, A, γ, α h and β h These are four fixed parameters determined by the physical characteristics of the antenna.

[0018] As a further improvement, the reflection characteristic of the human face described in the present invention is that the human face reflects the incident signal in different directions with different amplitudes, and the human face is modeled as a semi-mirror reflector. The semi-mirror reflector is characterized as: for a point F on the human face, given its incident signal direction (i) and surface normal (n), the amplitude distribution of its reflected signal is determined by the angle (φ) between the reflected signal and the mirror reflection direction: g(F) = exp(-φ 2 / 2σ 2 ), σ is an empirical threshold.

[0019] As a further improvement, the facial structure features with robust extraction distance described in the present invention support robust face recognition at a variable authentication distance. The facial structure features with robust extraction distance in step 4) extract the surface curvature distribution features of the user's face from the virtual millimeter wave signal, and store them as the user's template in the database; the facial structure features with robust extraction distance in step 7) extract the surface curvature distribution features of the face from the actual collected millimeter wave signal, and compare the features with the template in the database, thereby completing the matching of user identity.

[0020] As a further improvement, the facial structure feature extraction described in the present invention includes: reconstruction of a three-dimensional facial image based on a SAR algorithm and extraction of facial surface curvature distribution features. The reconstruction of the three-dimensional facial image based on the SAR algorithm utilizes a range migration algorithm to reconstruct the millimeter wave signal matrix s(x, y, t) into an intuitive three-dimensional facial image, and extracts facial surface curvature distribution features based on the three-dimensional facial image as facial geometric structure features for use in the subsequent user identity matching process.

[0021] As a further improvement, the extraction of facial surface curvature distribution features described in the present invention includes three steps: conversion of three-dimensional facial images to two-dimensional facial images, outlining of the "bright area" contour of the two-dimensional facial image, and extraction of the "bright area" contour features; the conversion of three-dimensional facial images to two-dimensional facial images further improves the computational efficiency of the face recognition system and retains effective facial structure information; outlining of the "bright area" contour of the two-dimensional facial image is distance robust, reflecting the distribution of facial surface curvature, and serving as a distance robust facial geometric structure feature; the extraction of the "bright area" contour feature includes removing discrete points and quantizing the "bright area" contour using Fourier descriptors; the contour features of the "bright area" finally extracted reflect the surface curvature distribution of the face, thereby serving as a distance robust facial geometric structure feature.

[0022] As a further improvement, the extraction of facial biometric features for forgery attack detection in step 6) of the present invention includes: selection of "flat" facial areas and identification of facial biometric material. First, the flat facial areas are selected by calculating the energy of the signals reflected from the face received by the antennas on the planar antenna array. Then, the reflection coefficients of these flat areas are calculated as the final biometric material features, thereby achieving liveness detection.

[0023] As a further improvement, the construction of the virtual registration scene described in the present invention includes three steps: generating a three-dimensional face model, constructing a virtual planar antenna array, and determining the relative positions between them; the three-dimensional face model is reconstructed based on the face image provided by the user using a machine learning method; the virtual planar antenna array is constructed based on the arrangement of the planar antenna array constructed during the actual authentication process; the relative position is adjusted based on the relative position between the face and the planar antenna array during the actual authentication process.

[0024] The present invention also discloses a millimeter-wave-based face recognition device that is resistant to counterfeiting attacks and occlusion, comprising:

[0025] Face image acquisition module: used to obtain at least 3 or more face images, and the face images are collected from different directions;

[0026] Virtual registration scene construction module: used to construct a virtual registration scene based on the collected face images;

[0027] Virtual registration signal generation module: used to complete virtual registration signal generation based on virtual registration scenarios;

[0028] Structural feature extraction module: used to extract distance-robust facial structural features using the generated virtual registration signal and save it in the database as a template;

[0029] Millimeter wave signal capture module: collects millimeter wave signals reflected by the user's face;

[0030] Liveness detection module: extracts facial biometric features based on collected signals to detect forgery attacks;

[0031] Facial structure feature extraction module: extracts distance-robust facial structure features from the detected signal;

[0032] User identity matching and authentication module: The obtained facial structure features are compared with the template in the user registration stage to achieve user authentication.

[0033] The beneficial effects of the present invention are as follows:

[0034] This paper proposes a facial recognition system that can effectively operate even when the face is obscured by a mask. To reduce the registration overhead of wireless facial recognition systems, this paper proposes a novel virtual registration method, in which users only need to provide three facial photos taken from different angles to complete registration. This method constructs a millimeter-wave signal propagation model to convert the given facial photos into virtual registration signals, thus avoiding the user's tedious on-site registration. Furthermore, to achieve robust face recognition at variable distances, this paper proposes a distance-suppressed facial geometric structure feature extraction algorithm. Based on SAR imaging results, this algorithm extracts facial curvature information as distance-robust facial structure features. Experimental results show that this algorithm can achieve an authentication success rate of over 90% in three different environments (conference room, laboratory, and office) and at an authentication distance ranging from 10cm to 20cm. Finally, this paper achieves reliable liveness detection by extracting biomaterial properties from the millimeter-wave signal reflected from the face. Experimental results show that the liveness detection method can effectively distinguish real faces from 3D printed masks made of different materials (metal, PLA, nylon, resin and silicone), which indicates that the method can effectively resist anti-counterfeiting attacks, thereby greatly improving the security of the face recognition system.

[0035] The present invention can utilize millimeter waves to realize anti-counterfeiting attack and anti-occlusion face recognition.

[0036] The present invention designs a user identity registration method of virtual registration, which avoids tedious user on-site registration.

[0037] The present invention constructs a theoretical millimeter wave signal propagation model to achieve cross-modal conversion from two-dimensional facial images to millimeter wave signals.

[0038] The present invention models the directional characteristics of the antenna to characterize the initial amplitude of the transmitted signal in a real acquisition scenario.

[0039] The present invention models the reflection characteristics of human faces to signals, so as to describe the reflection of real human faces to signals.

[0040] The present invention proposes a distance-suppressed facial geometric structure feature extraction method, which realizes distance-robust face authentication.

[0041] The present invention reconstructs a three-dimensional facial image from millimeter wave signals to associate non-semantic millimeter wave signals with intuitive facial structural features.

[0042] The present invention extracts the contour characteristics of the "bright area" of the reconstructed facial image to depict the geometric structure characteristics of the face.

[0043] The present invention proposes a liveness detection method for resisting forgery attacks.

[0044] The present invention extracts the reflection coefficient of the perceived object from the amplitude of the millimeter wave signal as a biomaterial characteristic, which is used to determine whether the object is a real human face. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the millimeter-wave-based face recognition method that is resistant to forgery attacks and occlusion;

[0046] Figure 2 Millimeter-wave radar antenna directivity model diagram;

[0047] Figure 3 Modeling diagram of the impact of facial reflection characteristics on millimeter wave signals;

[0048] Figure 4 Millimeter-wave reconstruction of 3D facial images;

[0049] Figure 5 Reconstructed 3D face images at different distances;

[0050] Figure 6 Equivalent schematic diagram of the flat area of the face. DETAILED DESCRIPTION

[0051] The present invention discloses a face recognition method based on millimeter waves that is resistant to counterfeiting attacks and occlusion. The method includes user registration and user authentication. User registration includes the following steps:

[0052] 1) Obtain at least 3 or more face images, and the face images are collected from different directions

[0053] 2) Construct a virtual registration scene based on the collected face images

[0054] 3) Generate virtual registration signals based on virtual registration scenarios

[0055] 4) Use the generated virtual registration signal to extract distance-robust facial structural features and save the database as a template

[0056] User authentication includes the following steps:

[0057] 5) Collecting millimeter wave signals reflected by the user’s face;

[0058] 6) Extract facial biometric features based on the collected signals to detect forgery attacks

[0059] 7) For the detected signal, extract distance-robust facial structural features;

[0060] 8) Compare the obtained facial structure features with the template in the user registration stage to achieve user authentication.

[0061] The generation of virtual registration signals is achieved by constructing a theoretical millimeter wave signal propagation model. The millimeter wave signal propagation model mainly considers three factors for the transmitted millimeter wave signal: The influence of: propagation distance (r), antenna directional characteristics (p) and face reflection characteristics (g), that is, virtual millimeter wave signal It can be expressed as Where F represents a point on the face. Based on the theoretical model of millimeter wave signal propagation, this method can estimate a virtual registration signal that is very similar to the signal collected in the real scene.

[0062] The millimeter-wave signal propagation model considers the impact of the antenna's directional characteristics on the virtual registration signal. The antenna's directional characteristics mean that the initial signal amplitude of the signal transmitted by the millimeter-wave radar in different directions varies significantly. This method models this directional characteristic as a Gaussian beam. A Gaussian beam indicates that the initial amplitude (p) of the transmitted signal is determined by the angle between the transmitted signal and the centerline of the millimeter-wave beam in the horizontal (α) and vertical (β) planes: Among them, A, γ, α h and β h These are four fixed parameters determined by the physical characteristics of the antenna.

[0063] The influence of the reflective characteristics of the face on the virtual registration signal. The reflective characteristics of the face are that the face can reflect the incident signal in different directions with different amplitudes, and the face is modeled as a semi-mirror reflector. The semi-mirror reflector can be characterized as follows: for a point F on the face, given its incident signal direction (i) and surface normal (n), the amplitude distribution of its reflected signal is determined by the angle (φ) between the reflected signal and the mirror reflection direction: g(F) = exp(-φ 2 / 2σ 2 ), σ is an empirical threshold.

[0064] Facial structural features enable robust face recognition at variable authentication distances. Specifically, this system designs distance-dependent facial geometric structural feature extraction. During the registration phase, the surface curvature distribution features of the user's face are extracted from the virtual millimeter-wave signal and stored as the user's template in the database. During the authentication phase, the surface curvature distribution features of the face are extracted from the real millimeter-wave signal and compared with the template in the database to match the user's identity.

[0065] The facial structural feature extraction module includes two parts: SAR-based 3D facial image reconstruction and facial surface curvature distribution feature extraction. The SAR-based 3D facial image reconstruction utilizes a range migration algorithm to reconstruct the millimeter-wave signal matrix s(x, y, t) into a visual 3D facial image. The facial surface curvature distribution features extracted from the 3D facial image serve as geometric structural features for subsequent user identification matching.

[0066] Extraction of facial surface curvature distribution features. The extraction of facial surface curvature distribution features includes three steps: conversion of three-dimensional facial image to two-dimensional facial image, outlining of the "bright area" contour of the two-dimensional facial image, and extraction of the "bright area" contour features. The purpose of converting three-dimensional facial image to two-dimensional facial image is to further improve the computational efficiency of the face recognition system while also retaining effective facial structure information. Specifically, the conversion projects the three-dimensional facial image w(x′, y′, z′) onto the two-dimensional facial image w by selecting the maximum intensity value along the z-axis. 2D (x′, y′, z′). Two-dimensional face image w 2D The purpose of outlining the “bright area” of (x′, y′, z′) is to make the “bright area” contour robust, reflecting the distribution of facial surface curvature, and serving as a distance-robust facial geometric structure feature. Specifically, this method uses the maximum inter-class variance method to set an adaptive threshold β. If w 2D If the value of a pixel in (x′, y′, z′) is greater than β, the value of this pixel is set to 1; otherwise, it is set to 0. In this case, the two-dimensional face image w 2D(x′, y′, z′) is converted to a binary image W Binary (x′, y′, z′), where the binary image w Binary The pixel values of 1 in (x′, y′, z′) are the "bright areas." Extracting the contour features of these "bright areas" involves removing discrete points and quantizing the contours using Fourier descriptors. The resulting contour features reflect the surface curvature distribution of the face, thus serving as distance-robust facial geometric structure features.

[0067] In step 6), facial biometric features are extracted for forgery attack detection. This detection involves two steps: selecting "flat" facial areas and identifying the facial biometric material. Specifically, this module first selects flat facial areas by calculating the energy of the signal reflected from the face by antennas on a planar antenna array. The reflection coefficients of these flat areas are then calculated as the final biometric material features, enabling liveness detection.

[0068] In step 2), a virtual registration scene is constructed. This construction involves three steps: generating a 3D facial model, constructing a virtual planar antenna array, and determining their relative positions. The 3D facial model is reconstructed using machine learning methods based on the facial image provided by the user. The virtual planar antenna array is constructed based on the layout of the planar antenna array constructed during the actual authentication process. The relative position is adjusted based on the relative position between the face and the planar antenna array during the actual authentication process.

[0069] The present invention also discloses a millimeter-wave-based face recognition device that is resistant to counterfeiting attacks and occlusion, comprising:

[0070] Face image acquisition module: used to obtain at least 3 or more face pictures, the face pictures are collected from different directions;

[0071] Virtual registration scene construction module: used to construct a virtual registration scene based on the collected face images;

[0072] Virtual registration signal generation module: used to complete virtual registration signal generation based on virtual registration scenarios;

[0073] Structural feature extraction module: used to extract distance-robust facial structural features using the generated virtual registration signal and save it in the database as a template;

[0074] Millimeter wave signal capture module: collects millimeter wave signals reflected by the user's face;

[0075] Liveness detection module: extracts facial biometric features based on collected signals to detect forgery attacks;

[0076] Facial structure feature extraction module: extracts distance-robust facial structure features from the detected signal;

[0077] User identity matching and authentication module: The obtained facial structure features are compared with the template in the user registration stage to achieve user authentication.

[0078] The purpose of the present invention is to utilize the penetrability and material sensitivity of millimeter waves to construct a millimeter-wave-based face recognition system that is not affected by occlusion and can resist forgery attacks. Specifically, the system is divided into two stages. In the registration stage, in order to avoid tedious on-site registration, a virtual registration method is designed, and based on this method, a corresponding virtual registration signal is generated. Based on the generated virtual registration signal, the method extracts facial structural features from it as a template and stores it in a database. In the authentication stage, facial material characteristics are first extracted from the millimeter-wave signal reflected by the face, and based on this, user activity detection is implemented to resist forgery attacks. In addition, to achieve robust face authentication, the method designs a distance-suppressed facial structure feature extraction algorithm. Using this algorithm, the system will perform similarity matching between the facial geometric features extracted from the real millimeter-wave signal collected during the authentication process and the facial geometric features extracted from the virtual registration signal for identity verification. The following is a further explanation with reference to the specific steps and the accompanying drawings in the specification. Figure 1 A flow chart of a face recognition method based on millimeter waves that is resistant to counterfeiting attacks and occlusion; specifically, the method includes the following steps:

[0079] Step 1) Acquisition of facial images

[0080] The camera is used to collect the user's facial images from three different directions.

[0081] Step 2) Generate virtual registration signal

[0082] Step 2.1) Virtual registration scene construction

[0083] In order to generate a virtual registration signal, it is necessary to first build a virtual registration scene that is consistent with the real authentication scene. The construction of the virtual scene is mainly divided into three parts: generating a three-dimensional face model, building a virtual planar antenna array, and determining the relative positions between them. First, based on the three two-dimensional face images provided by the user, reconstruct the three-dimensional face model. Then, according to the arrangement of the planar antenna array constructed in the real authentication process, a virtual planar antenna array is constructed. Among them, the virtual transceiver antenna located in the mth row and nth column of the virtual planar antenna array can be expressed as TR m，n After completing the construction of the 3D face model and the virtual planar antenna array, the relative positions between them are adjusted to make them consistent with the real authentication process.

[0084] Step 2.2) Convert 2D face image to virtual signal

[0085] Based on the constructed virtual registration scenario, a millimeter wave signal propagation model is established to generate the virtual registration signal. The millimeter wave signal propagation model mainly considers the influence of three factors on the virtual registration signal: 1) propagation distance, 2) antenna directional characteristics, and 3) face reflection characteristics. Therefore, based on this theoretical propagation model, the antenna TR m，n (coordinates are (x m，n ,y m，n ,0)) is emitted and is detected by a point F on the 3D face model (coordinates are (x F ,y F , z F )) is reflected and is reflected by the antenna TR m，n Receiving is represented by:

[0086]

[0087] Among them, ∈ represents the reflection coefficient of the face material, which can be considered as a constant in the model. represents the effect of propagation distance on the virtual registration signal, where Indicates that the antenna TR m，n Transmitted signal, τ = 2r m，n / c represents the time delay of propagation, r m，n Indicates antenna TR m，n The Euclidean distance from point F. m，n (F) and g m，n (F) represents the influence of antenna direction characteristics and face reflection characteristics on virtual registration signal. For the convenience of representation, they are integrated into a face response function for millimeter wave. m，n (F). Next, we will introduce how to characterize the directional characteristics of the antenna (p) and the reflective characteristics of the face (g).

[0088] The antenna of the millimeter-wave radar has a strong directional characteristic. For example, the signal beam it transmits consists of a high-energy main lobe and low-energy side lobes. This means that the initial signal amplitudes of the signals emitted by the millimeter-wave radar in different directions are very different. Since face recognition is a near-field perception, this difference cannot be ignored when generating a virtual registration signal. Since the energy of the main lobe is much higher than that of the side lobe, for the convenience of modeling, the side lobes are ignored and only the main lobe is considered and modeled as a basic Gaussian beam. Therefore, Figure 2 Millimeter wave radar antenna directivity model diagram; from TR m，n The initial amplitude of the signal transmitted to point F on the face is determined by the transmitted signal The angle between the antenna directivity and the main lobe centerline c in the horizontal plane (H) and the vertical plane (E) is determined. Therefore, the effect of antenna directivity on the initial amplitude of the transmitted signal can be expressed as:

[0089]

[0090] in And A represents the peak value of the main lobe amplitude. In addition, γ, α h and β h are three fixed parameters determined by the physical characteristics of the antenna, where α h and β h are the half-energy points of the main lobe in the horizontal and vertical planes respectively.

[0091] Next, we will characterize the reflective properties of the human face. Specifically, we will model the face as a semi-specular reflector, meaning that the face can reflect the incident signal in different directions with varying amplitudes. Figure 3 Modeling diagram of the impact of facial reflectivity on millimeter-wave signals. For a point F on the face, given its incident signal direction (i) and surface normal (n), the amplitude distribution of its reflected signal is a Gaussian function centered on the specular reflection direction (s). Therefore, the impact of facial reflectivity on the reflected signal amplitude can be expressed as:

[0092] g m，n (F)=exp(-φ 2 / 2σ 2 ), (3)

[0093] Where σ is an empirical threshold, φ represents the angle between the received signal (r) and the mirror reflection signal (s), and can be calculated as In addition, the direction of the specular reflection signal (s) can be calculated as

[0094]

[0095] The surface normal (n) can be equivalent to the normal of a small plane consisting of point F and its eight adjacent points. Therefore, based on the above modeling, we can get a signal reflected by a point on the face received by a virtual transceiver antenna on the planar antenna array. Based on this, we can obtain the antenna TR by superimposing the reflection of the signal at each point on the face. m，n Theoretically, the virtual registration signal that should be captured is:

[0096]

[0097] Because a virtual planar antenna array contains N x *N y*ρ virtual transceiver antennas, so we can repeat the above operation for each virtual transceiver antenna to obtain a virtual registration signal matrix:

[0098]

[0099] Where (x, y) represents the coordinates of the virtual transceiver antenna, and O represents the coordinates of all virtual transceiver antennas. m，n , so its matrix size is N x *N y *ρ. Similar to O, R represents the r of all virtual transmit and receive antennas m，n At this point, the final virtual registration signal can be obtained.

[0100] Step 3) Acquisition of facial structural features

[0101] Unlike photos, the non-semantic nature of millimeter-wave signals prevents them from establishing an intuitive connection with the structural information of the face. Inspired by SAR imaging algorithms, a three-dimensional face image is reconstructed from the millimeter-wave signal matrix s(x, y, t) (both the virtual registration signal during enrollment and the real millimeter-wave signal during authentication). However, millimeter-wave signals are very sensitive to changes in the distance between the face and the millimeter-wave radar, resulting in inconsistent reconstructed three-dimensional facial images at different distances. Furthermore, in actual face recognition scenarios, it is difficult to ensure that the distance between the user's face and the millimeter-wave radar remains constant during each authentication. Therefore, a facial structural feature extraction algorithm based on SAR imaging is proposed to suppress the influence of the distance between the face and the millimeter-wave radar. During the enrollment phase, the virtual registration signal is used by this algorithm to obtain a user template for the subsequent authentication process.

[0102] Step 3.1) Construction of 3D facial image based on SAR imaging

[0103] Since face recognition is a near-field perception, the Range Migration Algorithm (RMA) suitable for near-field imaging scenarios is used to reconstruct the millimeter wave signal matrix s(x, y, t) into an intuitive three-dimensional face image. Specifically, the millimeter wave signal matrix s(x, y, t) is first subjected to a two-dimensional Fourier transform to convert it from the time-space domain to the wavenumber domain (i.e., s(k x , k y , k)), and then perform distance offset correction and target refocusing in the wavenumber domain. Finally, it is converted to the spatial domain to obtain the three-dimensional image w(x′, y′, z′) of the face:

[0104]

[0105] Among them, 1，1 (xF ,y F , z F ) represents the response of the entire face to the antenna in the first row and first column of the planar antenna array, revealing the curvature distribution of the face surface. In addition, A is a constant matrix related to the antenna arrangement on the planar antenna array. Figure 4 Millimeter wave reconstructed 3D face image; It can be seen that the 3D face image w(x′, y′, z′) is clearly divided into “bright areas” (i.e. areas with larger intensity values) and “dark areas” (i.e. areas with smaller intensity values). Although the intensity value of the 3D face image w(x′, y′, z′) is affected by the distance, according to Figure 5 The reconstructed 3D facial images at different distances show that the shape of the "bright area" of w(x', y', z') is stable at different distances and is determined solely by the curvature of the facial surface. Therefore, this feature extraction method uses the contour features of the "bright area" of the 3D facial image w(x', y', z') as the final facial structural features.

[0106] Step 3.2) Conversion of 3D face image to 2D face image

[0107] To reduce computational overhead, the 3D face image w(x′, y′, z′) is first projected onto the 2D face image w by selecting the maximum intensity value along the z-axis. 2D (x′, y′, z′). Since the intensity values of the pixels in the 3D face image w(x′, y′, z′) can reveal the curvature and depth information of the face surface, this conversion of 3D face images to 2D face images can both improve computational efficiency and preserve effective facial structural information.

[0108] Step 3.3) Outlining the “bright area”

[0109] In order to outline the two-dimensional face image w 2D The contour of the “bright area” of (x′, y′, z′) is set with an adaptive threshold β using the maximum inter-class variance method. 2D If the value of a pixel in (x′, y′, z′) is greater than β, the value of this pixel is set to 1; otherwise, it is set to 0. In this case, the two-dimensional face image w 2D (x′, y′, z′) is converted to a binary image w Binary (x′, y′, z′), where the binary image w Binary The part of (x′, y′, z′) where the pixel value is 1 is the “bright area”.

[0110] Step 3.4) Extraction of contour features

[0111] After outlining the contour of the "bright area", its discrete points are first removed and then the Fourier descriptor is used to quantify the contour of the "bright area", which is used as the final distance-suppressed facial structure feature.

[0112] Step 4) User authentication signal collection:

[0113] The user first places his face in front of the millimeter-wave radar, which then scans the face along a specific trajectory using a two-dimensional slide rail. y Row and N x Columns and finally obtain the millimeter wave signal matrix s(x, y, t).

[0114] Step 5) Activity detection

[0115] In order to improve the security of the face recognition system, a liveness detection method is proposed to resist forgery attacks. During the user authentication process, this method extracts the reflection coefficient of the face from the amplitude of the millimeter wave signal and uses it as a biomaterial feature to determine whether the object perceived by the millimeter wave radar is a real person. Specifically, according to formula (5), the antenna TR of the mth row and nth column of the planar antenna array is m，n The amplitude A of the received millimeter wave signal reflected from the face m，n It can be expressed as:

[0116]

[0117] From the above formula we can see that the amplitude A m，n It is related to the reflection coefficient (∈) of the face surface. Therefore, it is reasonable to extract the reflection coefficient of the face from the amplitude of the millimeter wave signal and use it as a biomaterial feature. However, in addition to the reflection coefficient, the amplitude of the millimeter wave signal A m，n It is also related to multiple factors: the curvature characteristics of the face surface m，n and the distance r between the face and the millimeter-wave radar m，n In order to eliminate the two effects on the amplitude A m，n To address the impact of facial smear, this liveness detection method extracts biomaterial characteristics (∈) by selecting regions with uniform facial curvature (i.e., relatively flat regions parallel to the planar antenna array). Specifically, this liveness detection method consists of two steps: 1) selecting flat regions on the face and 2) identifying biomaterial characteristics.

[0118] Step 5.1) Select a flat area on the face. First, select the antenna on the planar antenna array that detects the sensing target (i.e., the face). Specifically, calculate the distance between each antenna on the planar antenna array and the sensing target and form a set D. Then calculate the mode d in the set D. modeSince the sensing target (i.e., human face) occupies most of the area covered by the planar antenna array, the distance from those antennas to the target is equal to d mode The antennas with are considered to have detected the sensing target. Afterwards, these antennas are formed into a new set H and the antennas that detect the relatively flat areas on the sensing object are further selected. Although the antenna can receive the millimeter wave signals reflected from all points on the face, it is mainly affected by the area Γ closest to it. Therefore, each antenna mainly corresponds to a specific face area of similar size. In addition, according to formula (5), the signal amplitude A received by the antenna m，n The surface curvature o of the corresponding face area m，n The distribution determines the energy of the received signal. Therefore, when an antenna receives a signal with If it exceeds an empirical threshold m, the face region Γ corresponding to the antenna is a relatively flat area. Figure 6 Schematic diagram of the equivalent flat area on the face. This area Γ can be equivalent to a plane Γ' parallel to the planar antenna array. As described above, by calculating the energy of the signal received by the antenna, the antennas that detect the flat area on the face can be selected from the set H to form a new set P.

[0119] Step 5.2) Identification of biological materials

[0120] A relatively flat region Γ can be considered as consisting of N surfaces with similar surface curvature o m，n and distance r m，n The reflection coefficient of the surface of a set of points can be expressed as:

[0121]

[0122] Where F is an arbitrary point on the region Γ'. Since the region Γ' is parallel to the planar antenna array, m，n (F) is equal to 1. In addition, r m，n It can be calculated from the frequency of the received signal. Next, the reflection coefficients of the perceived object measured by each antenna in set P are calculated to form a reflection coefficient set Q. Outliers in set Q are then removed based on the Pauta criterion, and its six statistical features (mean, standard deviation, mode, median absolute deviation, mean absolute deviation, and range) are calculated as a biometric feature vector. Finally, the vector is input into a pre-trained support vector machine (SVM) to determine whether the perceived target is a true human face.

[0123] Step 6) User matching

[0124] Suppose m users have registered with the facial recognition system. Each user has a template stored in the database. During authentication, the user repeats step 5 and obtains the facial structural feature R. R is compared with each template, resulting in m corresponding Euclidean distances. The system then selects the smallest of these m distances and compares it with a pre-set threshold. If the minimum distance is less than the threshold, the system deems the user legitimate. Otherwise, the system rejects the authentication request.

[0125] The present invention also discloses a millimeter-wave-based face recognition device that is resistant to counterfeiting attacks and occlusion, comprising:

[0126] Face image acquisition module: used to obtain at least 3 or more face images, and the face images are collected from different directions;

[0127] Virtual registration scene construction module: used to construct a virtual registration scene based on the collected face images;

[0128] Virtual registration signal generation module: used to complete virtual registration signal generation based on virtual registration scenarios;

[0129] Structural feature extraction module: used to extract distance-robust facial structural features using the generated virtual registration signal and save it in the database as a template;

[0130] Millimeter wave signal capture module: collects millimeter wave signals reflected by the user's face;

[0131] Liveness detection module: extracts facial biometric features based on collected signals to detect forgery attacks;

[0132] Facial structure feature extraction module: extracts distance-robust facial structure features from the detected signal;

[0133] User identity matching and authentication module: The obtained facial structure features are compared with the template in the user registration stage to achieve user authentication.

[0134] The above description is not intended to limit the present invention. It should be noted that a person skilled in the art may make several changes, modifications, additions or substitutions without departing from the essential scope of the present invention. Such improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A millimeter-wave-based face recognition method that is resistant to counterfeiting attacks and occlusion, characterized in that: The method includes user registration and user authentication; the user registration includes the following steps: 1) Obtain at least three facial images, where the facial images are collected from different directions; 2) Construct a virtual registration scene based on the collected face images; 3) Generate a virtual registration signal based on the virtual registration scenario; 4) Using the generated virtual registration signal to extract distance-robust facial structural features and save them in the database as a template; The user authentication includes the following steps: 5) Collecting millimeter wave signals reflected by the user’s face; 6) Extract facial biometric features based on the collected signals to detect forgery attacks; 7) For the detected signal, extract distance-robust facial structural features; 8) Compare the obtained facial structure features with the template in the user registration stage to achieve user authentication.

2. The millimeter wave-based face recognition method for anti-counterfeiting attacks and anti-occlusion according to claim 1, characterized in that: In the step 3), the generation of the virtual registration signal is achieved by constructing a theoretical millimeter wave signal propagation model, which includes three factors for the transmitted millimeter wave signal: The influence of: propagation distance (r), antenna directional characteristics (p) and face reflection characteristics (g), that is, virtual millimeter wave signal Expressed as Where F represents a point on the face.

3. The millimeter wave-based anti-counterfeiting attack and anti-occlusion face recognition method according to claim 2, characterized in that: The directional characteristic of the antenna is that the initial signal amplitude of the signal emitted by the millimeter-wave radar in different directions varies greatly. This directional characteristic is modeled as a Gaussian beam. The Gaussian beam indicates that the initial amplitude (p) of the transmitted signal is determined by the angle between the transmitted signal and the center line of the millimeter-wave beam in the horizontal (α) and vertical (β) planes: Among them, A, γ, α h and β h are four fixed parameters determined by the physical characteristics of the antenna, and F represents any point on the face.

4. The millimeter wave-based anti-counterfeiting attack and anti-occlusion face recognition method according to claim 2, characterized in that: The reflection characteristic of the face is that the face reflects the incident signal in different directions with different amplitudes. The face is modeled as a semi-mirror reflector. The semi-mirror reflector is characterized as follows: for a point F on the face, given its incident signal direction (i) and surface normal (n), the amplitude distribution of the reflected signal is determined by the angle (φ) between the reflected signal and the mirror reflection direction: g(F) = exp(-φ 2 / 2σ 2 ), σ is an empirical threshold.

5. The millimeter wave-based anti-counterfeiting attack and anti-occlusion face recognition method according to claim 1, 2, 3, or 4, characterized in that: The facial structural features with robust extraction distance support robust face recognition at a variable authentication distance. The facial structural features with robust extraction distance in step 4) extract the surface curvature distribution features of the user's face from the virtual millimeter wave signal and store them as the user's template in the database; the facial structural features with robust extraction distance in step 7) extract the surface curvature distribution features of the face from the real collected millimeter wave signal and compare the features with the template in the database to complete the matching of the user identity.

6. The millimeter wave-based anti-counterfeiting attack and anti-occlusion face recognition method according to claim 5, characterized in that: The facial structure feature extraction includes: reconstruction of a three-dimensional facial image based on the SAR algorithm and extraction of facial surface curvature distribution features. The three-dimensional facial image reconstruction based on the SAR algorithm uses a range migration algorithm to reconstruct the millimeter wave signal matrix s(x, y, t) into an intuitive three-dimensional facial image, and extracts facial surface curvature distribution features based on the three-dimensional facial image as facial geometric structure features for use in the subsequent user identity matching process.

7. The millimeter wave-based face recognition method for anti-counterfeiting attacks and anti-occlusion according to claim 6, characterized in that: The extraction of facial surface curvature distribution features includes three steps: conversion of a three-dimensional facial image into a two-dimensional facial image, outlining of the "bright area" contour of the two-dimensional facial image, and extraction of the "bright area" contour features; the conversion of the three-dimensional facial image into a two-dimensional facial image further improves the computational efficiency of the face recognition system and retains effective facial structural information; the outlining of the "bright area" contour of the two-dimensional facial image is distance-robust, reflecting the distribution of facial surface curvature, and serving as a distance-robust facial geometric structure feature; the extraction of the "bright area" contour feature includes removing discrete points and quantizing the "bright area" contour using a Fourier descriptor; the "bright area" contour feature finally extracted reflects the surface curvature distribution of the face, thereby serving as a distance-robust facial geometric structure feature.

8. The millimeter wave-based anti-counterfeiting attack and anti-occlusion face recognition method according to claim 1, 2, 3, or 4, characterized in that: Extracting facial biometric features for forgery attack detection in step 6) includes selecting "flat" facial areas and identifying facial biometric material. First, the flat facial areas are selected by calculating the energy of the signal reflected from the face received by the antennas on the planar antenna array. The reflection coefficients of these flat areas are then calculated as the final biometric material features to achieve liveness detection.

9. The millimeter wave-based face recognition method for anti-counterfeiting attacks and anti-occlusion according to claim 1, characterized in that: The construction of the virtual registration scene includes three steps: generating a three-dimensional face model, constructing a virtual planar antenna array, and determining the relative position between them; the three-dimensional face model is reconstructed based on the face image provided by the user using a machine learning method; the virtual planar antenna array is constructed based on the arrangement of the planar antenna array constructed during the actual authentication process; and the relative position is adjusted based on the relative position between the face and the planar antenna array during the actual authentication process.

10. A millimeter-wave-based face recognition device that is resistant to counterfeiting attacks and occlusion, characterized in that: include: Face image acquisition module: used to obtain at least 3 or more face pictures, the face pictures are collected from different directions; Virtual registration scene construction module: used to construct a virtual registration scene based on the collected face images; Virtual registration signal generation module: used to complete virtual registration signal generation based on virtual registration scenarios; Structural feature extraction module: used to extract distance-robust facial structural features using the generated virtual registration signal and save it in the database as a template; Millimeter wave signal capture module: collects millimeter wave signals reflected by the user's face; Liveness detection module: extracts facial biometric features based on collected signals to detect forgery attacks; Facial structure feature extraction module: extracts distance-robust facial structure features from the detected signals; user identity matching and authentication module: compares the obtained facial structure features with the template in the user registration phase for similarity to achieve user authentication.

Images