A living body detection method based on environment correlation and transformer
By using a liveness detection method based on environmental correlation and Transformer, the problems of poor recognition performance and spoofing attacks in low-light environments for face recognition are solved, and the accuracy and stability of liveness detection in different environments are achieved.
Patent Information
- Application Number
- CN202211426803.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-14
- Publication Date
- 2026-02-06
- Estimated Expiration
- 2042-11-14
AI Technical Summary
Existing facial recognition technologies perform poorly in low-light conditions, and biometric-based liveness detection is easily affected by the external environment. Deep learning detection technology has poor generalization ability, making it difficult to prevent spoofing attacks.
A liveness detection method based on environmental correlation and Transformer is adopted. By acquiring face video data, five regions of signals are extracted, preprocessed and Fourier transformed, similarity is calculated, and liveness is determined by Transformer. During training, environmental correlation labels are used to distinguish between fake and live objects.
It achieves accurate identification of live subjects in different environments, avoids spoofing attacks, and improves the stability and accuracy of detection.
Smart Images
Figure CN115641540B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of biometric identification, and in particular to a living body detection method based on environmental correlation and a Transformer. BACKGROUND
[0002] With the continuous development of artificial intelligence technology and big data technology, face recognition technology is increasingly applied in identity authentication fields such as payment, access control, and safety monitoring. However, in actual applications, face data as an open biological feature is easily exploited by malicious individuals, leading to the existence of prosthetic attacks.
[0003] Therefore, the authenticity of face images has become an urgent problem for the industry to solve.
[0004] According to the search, the Chinese patent with the publication number CN109086728B discloses a living body detection method, which aims to solve the problem of poor recognition effect in environments with poor lighting conditions due to the large influence of light on face recognition technology. However, in addition to handling light, biological information technology is also susceptible to external environmental influences.
[0005] In existing living body detection technologies, there are mainly two directions: one is living body detection based on biological information, such as through the rPPG information of a face, and the other is to extract distinguishing features between prosthetics and living bodies based on deep learning technology.
[0006] However, biological information technology is susceptible to external environmental influences, leading to unstable measurement results, and deep learning detection technology requires a large amount of common data, resulting in poor generalization ability of measurement. SUMMARY
[0007] The present application is to solve the defects in the prior art and proposes a living body detection method based on environmental correlation and a Transformer.
[0008] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:
[0009] A living body detection method based on environmental correlation and a Transformer, comprising the following steps:
[0010] S1: obtaining video data containing a face;
[0011] S2: intercepting video data of five regions of the face;
[0012] S3: averaging the pixel values of each region in the G channel and taking the average value as the only pixel value of this region;
[0013] S4: removing the linear trend from the pixel signals of the same region at different times to eliminate baseline drift;
[0014] S5: Normalizing the data of the same region;
[0015] S6: Fast Fourier Transforming the generated signal wave;
[0016] S7: Filtering out the frequencies less than 50 and greater than 180;
[0017] S8: Inverse Fast Fourier Transforming the filtered frequency signal;
[0018] S9: Calculating the similarity of the signal wave of the nose region and the signal waves of the other four regions, wherein: if the four similarities are all higher than the threshold, it is judged that the face in the video is a living body, and if one of the four similarities is lower than the threshold, the processed signal waves of the five regions are input into the trained Transformer to judge whether the face in the video is a living body.
[0019] Preferably, in S1, the face video data contains 100 frames of face data, the video time is controlled at about 5s, and the face data must be continuous and cannot have a flash-off.
[0020] Preferably, in S2, the five regions of the face include: the nose region, the upper left corner, the upper right corner, the lower left corner and the lower right corner. Dlib is used to locate the key points of the face, and points 28 to 32 are used to generate the nose region, and the other four regions are the four corner regions of the picture, and the size of each region is 30*30 pixels.
[0021] Further, the five regions are divided into two categories, one is the nose region, and the other is the environment region: the four corner regions. The rPPG signal of the nose region is the strongest, and the rPPG signal of the environment region is the weakest.
[0022] Preferably, in S3, the G channel in each region is the intensity value of the green (G) color channel in the RGB color mode, because the blood has the strongest absorption ability to green light, so the green color in the collected three-channel signal can best reflect the information of heartbeats.
[0023] Preferably, in S4, the linear trend is removed from the pixel signals of the same region at different times in order to eliminate the baseline drift, and the least square method is used. If the baseline drift / trend item is not eliminated, the trend item will be regarded as the truly collected original signal, which will affect the accuracy of the signal and the subsequent data processing results.
[0024] Preferably, in S5, the data of the same region is normalized, and the data of each region is subtracted by the mean value and then divided by the standard deviation.
[0025] Preferably, in S6, the generated signal wave is subjected to fast Fourier transform to obtain a power spectrum, for the optical signal of the human skin surface, noise is often a high-frequency signal, and the normal heart rate is 60-100 beats per minute, and the frequencies less than 50 and greater than 180 are filtered out.
[0026] Optionally, in S7, the filtered frequency signal is subjected to inverse fast Fourier transform to obtain a relatively pure signal containing the heart rate.
[0027] Optionally, in S8, the signal wave of the nose region and the signal waves of the other four regions are subjected to similarity calculation to calculate the Pearson correlation coefficient; S0: similarity of the nose region and the upper left region; S1: similarity of the nose region and the upper right region; S2: similarity of the nose region and the lower left region; S3: similarity of the nose region and the lower right region.
[0028] Further, if the four similarities are all higher than the threshold value 0.8, it is judged that the face in the video is not a living body, the four corner regions generally do not contain face information, and there is no rPPG information in all region signals, if the similarity is high, it means that the environment of the nose region is highly correlated with the four corner regions, which means that the nose region has no rPPG signal, and all are false bodies;
[0029] If one of the four similarities is lower than the threshold value 0.8, it is judged that the face in the video may be a living body, if the similarity is low, it means that the environment of the nose region is weakly correlated with the four corner regions, which means that the environment of the nose region is different from that of the four corner regions, and there may be a living body.
[0030] Optionally, in S8, the processed signal waves of the five regions are input into the trained Transformer to judge whether the face in the video is a living body, first, the Transformer needs to be trained, and the process of training the Transformer is as follows:
[0031] The first step is to collect data of living bodies, the data of the five regions is a 100-dimensional vector, the embedding of the Transformer is 100, the five 100-dimensional vectors [v1, v2, v3, v4, v5] are spliced into a matrix with a dimension of [batch, 5, 100], and a corresponding label array [1 v1 ,1 v2 ,1 v3 ,1 v4 ,0 v5 ] is generated, 1 v1 represents the autocorrelation of the upper left region (v1), and this value is always 1.1 v2 represents the correlation coefficient of v2 and v1,1 v3represent the correlation coefficient of v2 and v1, the value is 1 by default.1 v4 represent the correlation coefficient of v4 and v3, v2, v1, the value is 1 by default. Because v1, v2, v3, v4 correspond to the vectors of the four corner regions, and the environment of the four corner regions is the same, the theoretical value of the correlation between the four corner regions is 1.0 v5 represent the correlation coefficient of v5 and v1, v2, v3, v4, because v5 is the vector of the nose region, and the data is a living body, so the theoretical value of the correlation of v5 and v1, v2, v3, v4 is 0.
[0032] Second step: collect the data of the prosthesis, the data of the five regions is a 100-dimensional vector, Transfo
[0033] rmer's embedding is 100, and the five 100-dimensional vectors
[0034] [v1, v2, v3, v4, v5] are spliced into a matrix with dimensions [batch, 5, 100], and the corresponding label array [1 v1 ,1 v2 ,1 v3 ,1 v4 ,1 v5 ],1 v1 represent the autocorrelation of the upper left region (v1), the value is 1 by default.1 v2 represent the correlation coefficient of v2 and v1,1 v3 represent the correlation coefficient of v3 and v2, v1, the value is 1 by default.1 v4 represent the correlation coefficient of v4 and v3, v2, v1, the value is 1 by default. Because v1, v2, v3, v4 correspond to the vectors of the four corner regions, and the environment of the four corner regions is the same, the theoretical value of the correlation between the four corner regions is 1.1 v5 represent the correlation coefficient of v5 and v1, v2, v3, v4, because v5 is the vector of the nose region, and the data is a living body, so the theoretical value of the correlation of v5 and v1, v2, v3, v4 is 1.
[0035] Further, input the processed signal wave of the five regions to the trained Transformer and judge whether the face in the video is a living body. Input the vector group [v1, v2, v3, v4, v5] (v5 is the nose region) of the test picture to the encoder to obtain the output tensor, and then input it to the decoder (not as the direct input of the decoder), and then use the start symbol <bos>When taken as input to a decoder, the output S is obtained v1 , and again <bos>+S v1 When S is input, output S v2 . Again <bos>+S v1 +S v2 When S is obtained as input v3 . With <bos>+S v1 +S v2 +S v3 When S is obtained as input v3 . Again, S <bos>+S v1 +S v2 +S v3 +S v4 When S v5 If at least three of S v1 , S v2 , S v3 , S v4 are greater than 0.6, and S v5 is greater than 0.8, it is a living body, otherwise it is a fake body.
[0036] Compared with the prior art, the beneficial effects of the present application are that the living body detection function can be effectively realized, fake body attacks can be avoided, and based on the environment correlation and the Transformer, the living body detection can better judge and detect whether it is a living body, distinguish the features of fake bodies and living bodies, and obtain accurate and stable results. BRIEF DESCRIPTION OF DRAWINGS
[0037] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application and explain the technical solutions of the present application, and do not constitute a limitation on the present application.
[0038] Figure 1 The flow logic diagram of the living body detection method based on environment correlation and Transformer proposed by the present application. DETAILED DESCRIPTION
[0039] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.
[0040] Embodiment one
[0041] With reference to Figure 1 , the living body detection method based on environment correlation and Transformer includes the following steps:
[0042] S1: acquiring video data containing a face;
[0043] S2: intercepting video data of five regions of the face;
[0044] S3: averaging the pixel values of each region in the G channel and taking the average value as the unique pixel value of the region;
[0045] S4: removing the linear trend from the pixel signals of different times in the same region to eliminate the baseline drift;
[0046] S5: normalizing the data in the same region for pretreatment;
[0047] S6: Fast Fourier transform is performed on the generated signal wave;
[0048] S7: Frequencies less than 50 and greater than 180 are filtered out;
[0049] S8: Fast inverse Fourier transform is performed on the filtered frequency signal;
[0050] S9: Similarity calculation is performed on the signal wave of the nose region and the signal waves of the other four regions, wherein: if the four similarities are all higher than the threshold value, it is judged that the face in the video is a living body, and if one of the four similarities is lower than the threshold value, the processed signal waves of the five regions are input into the trained Transformer to judge whether the face in the video is a living body.
[0051] On the basis of Embodiment One, in S1, the face video data contains 100 frames of face data, and the video time is controlled at about 5s, wherein the face data is continuous and there is no flashback.
[0052] In S2, the five regions of the face include: the nose region, the upper left corner, the upper right corner, the lower left corner and the lower right corner, wherein: Dlib is used to locate the face key points, and points 28-32 are used to generate the nose region, and the other four regions are the four corner regions of the picture, and the size of each region is 30*30 pixels.
[0053] Specifically, the five regions are divided into two categories, one is the nose region, and the other is the environment region: the four corner regions. The rPPG signal of the nose region is the strongest, and the rPPG signal of the environment region is the weakest.
[0054] In S3, the G channel in each region is the intensity value of the green (G) color channel in the RGB color mode, which is used to reflect the information of heartbeats.
[0055] It should be noted that since the blood has the strongest absorption ability to green light, the green color in the collected three-channel signal can best reflect the information of heartbeats.
[0056] In S4, the linear trend in the pixel signals of the same region at different times is removed to eliminate the baseline drift, and the least square method is used. If the baseline drift / trend term is not eliminated, the trend term will be regarded as the truly collected original signal, which will affect the accuracy of the signal and the subsequent data processing results.
[0057] In S5, the normalization preprocessing method for the data of the same region is to subtract the mean value of each region and then divide by the standard deviation.
[0058] In S6, the generated signal wave is subjected to a fast Fourier transform to obtain the power spectrum. For optical signals on the surface of human skin, noise is often a high-frequency signal. A normal heart rate beats 60-100 times per minute. Signals with frequencies less than 50 and greater than 180 are filtered out.
[0059] In S7, the filtered frequency signal is subjected to a fast inverse Fourier transform to obtain a signal containing the heart rate.
[0060] In S8, the signal waves from the nasal region are compared with those from the other four regions to calculate the similarity and Pearson correlation coefficient.
[0061] S0: Similarity between the nasal region and the upper left region; S1: Similarity between the nasal region and the upper right region; S2: Similarity between the nasal region and the lower left region; S3: Similarity between the nasal region and the lower right region.
[0062] Example 3
[0063] Based on Example 2, in S8, if all four similarities are higher than the threshold of 0.8, it is determined that the face in the video is not a live person. The four corner regions generally do not contain face information, and there is no rPPG information in the signals of all regions. If the similarity is high, it means that the environment of the nose region is very related to the four corner regions, which means that there is no rPPG signal in the nose region, and therefore it is a fake.
[0064] If any of the four similarity scores is below the threshold of 0.8, the face in the video may be a live person. If the similarity score is low, it means that the environment of the nose area is not strongly correlated with the four corner areas, which means that the environment of the nose area is different from the environment of the four corner areas, and there may be a live person.
[0065] In this embodiment, the processed signal waves corresponding to the five regions are input into the trained Transformer to determine whether the face in the video is a live person. First, the Transformer needs to be trained. The training process of the Transformer is as follows:
[0066] The first step is to collect live data. The data from the five regions is a 100-dimensional vector. The Transformer's embedding is 100. The five 100-dimensional vectors [v1, v2, v3, v4, v5] are concatenated into a matrix of dimensions [batch, 5, 100], and the corresponding label array is generated [1]. v1 ,1 v2 ,1 v3 ,1 v4 ,0 v5 ],1 v1 This represents the autocorrelation of the upper left region (v1). The default value is 1.
[0067] 1 v2 represent the correlation coefficient of v2 and v1, 1 v3 represent the correlation coefficient of v3 and v2, v1, this value is 1 by default, 1 v4 represent the correlation coefficient of v4 and v3, v2, v1, this value is 1 by default. Because v1, v2, v3, v4 correspond to the vectors of the four corner regions, and the environment of the four corner regions is the same, the theoretical value of the correlation between the four corner regions is 1.
[0068] 0 v5 represent the correlation coefficient of v5 and v1, v2, v3, v4, since v5 is the vector of the nose region, and the data is a living body, the theoretical value of the correlation between v5 and v1, v2, v3, v4 is 0;
[0069] Second step: collect the data of the prosthesis, the data of the five regions is a 100-dimensional vector, the embedding of the Transformer is 100, and the five 100-dimensional vectors
[0070] [v1, v2, v3, v4, v5] are spliced into a matrix with a dimension of [batch, 5, 100], and a corresponding label array [1 v1 , 1 v2 , 1 v3 , 1 v4 , 1 v5 ], 1 v1 represent the autocorrelation of the upper left region (v1), this value is 1 by default, wherein:
[0071] 1 v2 represent the correlation coefficient of v2 and v1, 1 v3 represent the correlation coefficient of v3 and v2, v1, this value is 1 by default, 1 v4 represent the correlation coefficient of v4 and v3, v2, v1, this value is 1 by default.
[0072] Because v1, v2, v3, v4 correspond to the vectors of the four corner regions, and the environment of the four corner regions is the same, the theoretical value of the correlation between the four corner regions is 1, 1 v5 represent the correlation coefficient of v5 and v1, v2, v3, v4, since v5 is the vector of the nose region, and the data is a living body, the theoretical value of the correlation between v5 and v1, v2, v3, v4 is 1.
[0073] In this embodiment, the processed five-region corresponding signal wave is input to the trained Transformer to determine whether the face in the video is a living body, and the specific process is as follows:
[0074] The vector group [v1, v2, v3, v4, v5] (v5 is the nose area) of the test picture is input to the encoder to obtain an output tensor, which is then sent to the decoder (not as the direct input of the decoder), and then the start symbol <bos>When taken as input to a decoder, the output S is obtained v1 and again <bos>+S v1 When S is input, output S v2 . Again <bos>+S v1 +S v2 When S is obtained as input v3 . With <bos>+S v1 +S v2 +S v3 When S is obtained as input v3 . Again, S <bos>+S v1 +S v2 +S v3 +S v4 When S v5 If at least three of S v1 , S v2 , S v3 , S v4 are greater than 0.6, and S v5 is greater than 0.8, then the subject is a living subject, otherwise the subject is a non-living subject.
[0075] The above description is merely preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art, according to the technical range disclosed by the present application and the inventive concept, should be covered within the protection scope of the present application.< / bos> < / bos> < / bos> < / bos> < / bos> < / bos> < / bos> < / bos> < / bos> < / bos>
Claims
1. A live detection method based on environmental correlation and Transformer, characterized in that: The method comprises the following steps: S1: obtaining video data containing a face; S2: intercepting video data of five regions of the face; S3: averaging pixel values of the G channel in each region and taking the average value as the unique pixel value of the region; S4: removing linear trends from pixel signals at different times in the same region to eliminate baseline drift; S5: normalizing the data in the same region; S6: performing fast Fourier transform on the generated signal wave; S7: filtering out frequencies less than 50 and greater than 180; S8: performing inverse fast Fourier transform on the filtered frequency signal; S9: calculating the similarity of the signal wave of the nose region and the signal waves of the other four regions, wherein: if the four similarities are all higher than the threshold value, it is judged that the face in the video is a living body, and if one of the four similarities is lower than the threshold value, the processed signal waves of the five regions are input into the trained Transformer to judge whether the face in the video is a living body.
2. The environment correlation and Transformer-based live detection method according to claim 1, wherein, In S1, the face video data contains 100 frames of face data, the video time is controlled at about 5s, and the face data is continuous without flashing out.
3. The living body detection method based on environment correlation and the Transformer according to claim 2, characterized in that, The five regions of the face in S2 include: the nose region, the upper left corner, the upper right corner, the lower left corner, and the lower right corner, wherein: Dlib is used to locate the key points of the face, points 28-32 are used to generate the nose region, and the other four regions are the four corner regions of the picture, and the size of each region is 30*30 pixels.
4. The living body detection method based on environment correlation and the Transformer according to claim 3, characterized in that, In S3, the G channel in each region is the intensity value of the green (G) color channel in the RGB color mode, which is used to reflect the information of heartbeats.
5. The living body detection method based on environment correlation and the Transformer according to claim 4, characterized in that, In S4, the linear trend is removed from the pixel signals at different times in the same region to eliminate baseline drift, which is achieved in a least squares manner.
6. The living body detection method based on environment correlation and the Transformer according to claim 5, characterized in that, In S5, the data in the same region is normalized by subtracting the mean value and dividing by the standard deviation.
7. The living body detection method based on environment correlation and the Transformer according to claim 6, characterized in that, In S6, the generated signal wave is subjected to fast Fourier transform to obtain the power spectrum, and for the optical signal on the surface of the human body, noise is often a high-frequency signal, and the normal heart rate beats 60-100 times per minute, so frequencies less than 50 and greater than 180 are filtered out.
8. The living body detection method based on environment correlation and the Transformer according to claim 7, characterized in that, In S8, the filtered frequency signal is subjected to inverse fast Fourier transform to obtain a signal containing the heart rate.
9. The living body detection method based on environment correlation and the Transformer according to claim 8, characterized in that, In S9, the similarity of the signal wave of the nose region and the signal waves of the other four regions is calculated, and the Pearson correlation coefficient is calculated; S0: the similarity of the nose region and the upper left region; S1: the similarity of the nose region and the upper right region; S2: the similarity of the nose region and the lower left region; S3: the similarity of the nose region and the lower right region.
10. The living body detection method based on environment correlation and the Transformer according to claim 9, characterized in that, In S9, if the four similarities are all higher than the threshold value 0.8, it is judged that the face in the video is not a living body, the four corner regions generally do not contain face information, and there is no rPPG information in all region signals, and if the similarity is high, it means that the environment of the nose region is highly related to the four corner regions, which means that the nose region has no rPPG signal, and all are fake. If there is one of the four similarities below the threshold value 0.8, it is possible that the face in the video is a live body, and if the similarity is low, it means that the environment of the nose region is weakly related to the four corner regions, which means that the environment of the nose region is different from that of the four corner regions, and it is possible that there is a live body.
Citation Information
Patent Citations
Liveness detection methods
CN109086728B
Human face living body detection method, system, device and readable storage medium
CN111222432A
User behavior recognition model training method, system and device and face recognition method, system and device
CN114220136A