Apparatus and method for detecting deepfake in real time through non-contact bio-signal measurement

Abstract

According to an apparatus and a method for detecting deepfake in real time through non-contact bio-signal measurement proposed in the present invention, a non-contact bio-signal is extracted from a face region in a video captured by an RGB camera, a deep learning model is trained using the extracted non-contact bio-signal, and it is determined whether the face is a real human face on the basis of a change in blood flow volume caused by a heartbeat, thereby enabling detection of a spoofing attack using deepfake technology.

Description

Device and method for real-time deepfake detection through non-contact biosignal measurement

[0001] The present invention relates to a real-time deepfake detection device and method, and more specifically, to a real-time deepfake detection device and method through non-contact biosignal measurement.

[0002] The content described in this section merely provides background information regarding an embodiment of the present invention and does not constitute prior art.

[0003]

[0004] Face recognition-based authentication systems are widely used on mobile devices and other platforms due to their ease of use and the advantages of contactless methods. However, face recognition systems have a vulnerability to deepfake attacks utilizing photos or videos of users' faces. In particular, as face-based authentication methods become increasingly common on mobile devices such as smartphones used frequently in daily life, there is a growing need for technologies to counter security attacks that exploit deepfake technology.

[0005]

[0006] Spoofing is a term derived from the word "spoof" (deception), referring to techniques that cause a victim to trust false information or connections due to the attacker's malicious intent, without directly attempting to intrude into a system. While DNS spoofing and IP spoofing were commonly used in the past, face spoofing attacks designed to deceive facial recognition systems are also on the rise recently, driven by the widespread adoption of such systems.

[0007]

[0008] There are several types of face spoofing attacks. 2D image attacks involve an attacker attempting to bypass authentication by using an image of an authorized person's face. 2D video attacks involve an attacker attempting to bypass authentication by showing a video of an authorized person's face to the authentication system. Because the face in a 2D video attack appears moving and lifelike, it can bypass motion-sensing spoofing detection systems that the most primitive 2D image attacks cannot bypass. 3D print / mask attacks are spoofing methods in which an attacker attempts to reproduce the 3D features of a real face by using a face mask or a 3D print of the actual face. These 3D print / mask attacks are problematic because they can bypass existing spoofing detection measures.

[0009]

[0010] Existing methods for detecting face spoofing attacks are broadly divided into texture-based and depth-based methods. Texture-based methods utilize texture information within an image to detect spoofing, analyzing brightness patterns using techniques such as Local Binary Patterns (LBP). However, this approach is ineffective for printed photographs or low-resolution images and can be vulnerable to 3D mask attacks. Additionally, depth-based methods utilize facial depth information to verify whether the face possesses actual physical three-dimensionality. However, this method has limitations, such as not functioning properly when using a 3D mask and requiring a depth camera.

[0011]

[0012] In particular, with the recent development of deepfake technology that uses artificial intelligence to generate fake content similar to the real thing, there is a need to develop technology capable of countering face spoofing attacks using deepfake videos.

[0013]

[0014] The aforementioned background technology is technical information that the inventor possessed for the derivation of the present invention or acquired during the process of deriving the present invention, and it cannot be considered as publicly known technology disclosed to the general public prior to the filing of the present invention.

[0015] The present invention is proposed to solve the aforementioned problems of previously proposed methods, and aims to provide a device and method for real-time deepfake detection through non-contact biosignal measurement that can detect spoofing attacks using deepfake technology by extracting non-contact biosignals from a face region in a video captured by an RGB camera, training a deep learning model with the extracted non-contact biosignals, and determining whether the face is a real human face through changes in blood flow volume caused by heartbeats.

[0016]

[0017] In addition, another objective of the present invention is to provide a device and method for real-time deepfake detection through non-contact biosignal measurement that can perform face recognition stably without being sensitive to changes in lighting and angles of the face, which are disadvantages of existing face recognition technology, by including a preprocessing unit that detects movement and lighting changes in a video and filters lighting change attack data.

[0018]

[0019] However, the technical problem that the present invention aims to solve is not limited to the technical problem described above, and other technical problems may exist. It goes without saying that objectives or effects that can be understood from the means of solving the problem or the embodiments, even if not explicitly mentioned, are also included.

[0020] A real-time deepfake detection device through non-contact biosignal measurement according to the features of the present invention for achieving the above-mentioned purpose,

[0021] As a real-time deepfake detection device that detects deepfake attacks in a face recognition system using video captured by an RGB camera,

[0022] A preprocessing unit that detects movement and lighting changes in the above video and filters lighting change attack data;

[0023] A biosignal extraction unit that detects a face region in the above video and extracts a non-contact biosignal; and

[0024] The configuration is characterized by including a deep learning model unit that learns the above-mentioned non-contact biosignal using a deep learning model and detects spoofing attacks using deepfake technology using a deepfake detection model that has completed training.

[0025]

[0026] Preferably, the pretreatment unit is,

[0027] It may include a motion detection unit that detects the direction and degree of movement of an object using optical flow values.

[0028]

[0029] More preferably, the motion detection unit is,

[0030] The detected movement direction and degree of movement are calculated as pixel values, and if the calculated pixel values ​​fall outside a preset threshold range, the corresponding data can be filtered.

[0031]

[0032] More preferably, the pretreatment unit is,

[0033] It may further include a lighting change detection unit that classifies and filters as lighting change attack data if the amount of change in the Y value in YCbCr is greater than or equal to a preset threshold.

[0034]

[0035] Preferably, the biosignal extraction unit is,

[0036] In the above video, the face region can be detected, rPPG (remote photoplethysmography) signals extracted, and pulse, blood pressure, and oxygen saturation can be calculated.

[0037]

[0038] More preferably, the biosignal extraction unit is,

[0039] For the above rPPG signal, a normalized signal can be obtained by using a chrominance (CHROM) algorithm to remove noise components and removing respiratory tendencies and trends, and by applying Butterworth bandpass filtering to remove components unrelated to cardiac activity, a filtered rPPG signal can be obtained.

[0040]

[0041] Preferably, the biosignal extraction unit is,

[0042] By using the filtered rPPG signal, the YCbCr color values ​​decomposed into three signals (Y, Cb, and Cr), and the power spectral density (PSD), the maximum power peak can be detected to calculate the pulse, blood pressure, and oxygen saturation.

[0043]

[0044] Preferably, the deep learning model unit is,

[0045] A first learning unit that trains an LSTM (Long Short-Term Memory) model using the above rPPG signal;

[0046] A second learning unit that converts the above rPPG signal into frequency and trains a CNN model; and

[0047] It may include a third learning unit that trains a CRNN (Convolutional Recurrent Neural Network) model using the above pulse, blood pressure, and oxygen saturation.

[0048]

[0049] A real-time deepfake detection method through non-contact biosignal measurement according to the features of the present invention for achieving the above-mentioned purpose is,

[0050] As a real-time deepfake detection method in which each step is performed on a computer to detect deepfake attacks in a face recognition system using video captured by an RGB camera,

[0051] (1) A preprocessing step for detecting movement and lighting changes in the above video and filtering lighting change attack data;

[0052] (2) A biosignal extraction step for detecting a face region in the above video and extracting a non-contact biosignal; and

[0053] (3) The configuration features include a deep learning model step that learns the above non-contact biosignal using a deep learning model and detects spoofing attacks using deepfake technology using a deepfake detection model that has completed learning.

[0054]

[0055] Preferably, the above pretreatment step is,

[0056] It may include a motion detection step that detects the direction and degree of movement of an object using optical flow values.

[0057]

[0058] More preferably, in the motion detection step,

[0059] The detected movement direction and degree of movement are calculated as pixel values, and if the calculated pixel values ​​fall outside a preset threshold range, the corresponding data can be filtered.

[0060]

[0061] More preferably, the above pretreatment step is,

[0062] If the amount of change in the Y value in YCbCr is greater than a preset threshold, a lighting change detection step may be further included to classify and filter it as lighting change attack data.

[0063]

[0064] Preferably, in the biosignal extraction step,

[0065] In the above video, the face region can be detected, rPPG (remote photoplethysmography) signals extracted, and pulse, blood pressure, and oxygen saturation can be calculated.

[0066]

[0067] More preferably, in the biosignal extraction step,

[0068] For the above rPPG signal, a normalized signal can be obtained by using a chrominance (CHROM) algorithm to remove noise components and removing respiratory tendencies and trends, and by applying Butterworth bandpass filtering to remove components unrelated to cardiac activity, a filtered rPPG signal can be obtained.

[0069]

[0070] Preferably, in the biosignal extraction step,

[0071] By using the filtered rPPG signal, the YCbCr color values ​​decomposed into three signals (Y, Cb, and Cr), and the power spectral density (PSD), the maximum power peak can be detected to calculate the pulse, blood pressure, and oxygen saturation.

[0072]

[0073] Preferably, the deep learning model step is,

[0074] A first training step of training an LSTM (Long Short-Term Memory) model with the above rPPG signal;

[0075] A second training step for converting the above rPPG signal into frequency and training a CNN model; and

[0076] It may include a third training step for training a CRNN (Convolutional Recurrent Neural Network) model using the above pulse, blood pressure, and oxygen saturation.

[0077] According to the device and method for real-time deepfake detection through non-contact biosignal measurement proposed in the present invention, non-contact biosignals are extracted from a face region in a video captured by an RGB camera, and a deep learning model is trained with the extracted non-contact biosignals to determine whether the face is a real person's face through changes in blood flow volume caused by heartbeats, thereby enabling the detection of spoofing attacks using deepfake technology.

[0078]

[0079] In addition, according to the device and method for real-time deepfake detection through non-contact biosignal measurement proposed in the present invention, by including a preprocessing unit that detects movement and lighting changes in a video and filters lighting change attack data, it is possible to perform stable face recognition without being sensitive to lighting changes and the angle of the face, which are disadvantages of existing face recognition technology.

[0080]

[0081] Furthermore, the various and beneficial advantages and effects of the present invention are not limited to those described above and may be more easily understood in the process of explaining specific embodiments of the present invention.

[0082] FIG. 1 is a diagram illustrating the overall system configuration including a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0083] FIG. 2 is a diagram illustrating the detailed configuration of a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0084] FIG. 3 is a drawing illustrating face recognition through a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0085] FIG. 4 is a diagram illustrating the detailed configuration of a preprocessing unit in a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0086] FIG. 5 is a diagram illustrating a biosignal extraction unit in a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0087] FIG. 6 is a diagram illustrating the detailed configuration of a deep learning model unit in a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0088] FIG. 7 is a diagram illustrating the generation of a deepfake detection model in the deep learning model unit of a real-time deepfake detection device through non-contact biosignal measurement according to an embodiment of the present invention.

[0089] FIG. 8 is a diagram illustrating the flow of a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention.

[0090] FIG. 9 is a diagram illustrating the detailed flow of a preprocessing step in a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention.

[0091] FIG. 10 is a diagram illustrating the detailed flow of the deep learning model step in a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention.

[0092] <Explanation of Symbols>

[0093] 100: Real-time Deepfake Detection Device

[0094] 110: Preprocessing section

[0095] 111: Motion detection unit

[0096] 112: Lighting change detection unit

[0097] 120: Biosignal extraction unit

[0098] 130: Deep Learning Model Section

[0099] 131: 1st Learning Department

[0100] 132: 2nd Learning Department

[0101] 133: 3rd Learning Department

[0102] 134: Prediction section

[0103] 200: RGB camera

[0104] S110: Preprocessing step

[0105] S111: Motion detection step

[0106] S112: Lighting change detection step

[0107] S120: Biosignal extraction step

[0108] S130: Deep learning model stage

[0109] S131: 1st learning stage

[0110] S132: Second learning stage

[0111] S133: Third learning stage

[0112] S134: Prediction step

[0113] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0114]

[0115] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "indirectly connected" with other elements interposed between them. Furthermore, terms such as "include," "have," or "have" described below should be interpreted as indicating the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. Additionally, singular expressions used in the present invention include plural expressions unless the context clearly indicates otherwise.

[0116]

[0117] In addition, each component, process, procedure, or method included in each embodiment of the present invention may be shared within a scope that is not technically contradictory to one another.

[0118]

[0119] Additionally, terms such as “…part,” “…unit,” and “module” described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0120]

[0121] In addition, some of the operations or functions described as being performed by a terminal, device, or device in the present invention may instead be performed by a server connected to said terminal, device, or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal, device, or device connected to said server.

[0122]

[0123] In particular, the means for executing the system according to each embodiment of the present invention may be an application or a web server, and the terminal, which is the means for reading the recording medium on which the application or web server is recorded, may include not only general PCs such as general desktops or laptops, but also mobile terminals such as smartphones and tablet PCs.

[0124]

[0125] The following examples are detailed descriptions to aid in understanding the present invention and are not intended to limit the scope of the present invention. Accordingly, inventions within the same scope that perform the same function as the present invention will also fall within the scope of the present invention.

[0126]

[0127] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0128]

[0129] FIG. 1 is a diagram illustrating the overall system configuration including a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 1, the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention can detect deepfake attacks in a face recognition system using video captured by an RGB camera (200). At this time, the RGB camera (200) may be a camera that captures a face video for face recognition. The real-time deepfake detection device (100) receives a video captured by the RGB camera (200), determines whether the face included in the video is a real face or a face image generated by a deepfake technique, and transmits the determination result to the face recognition system.

[0130]

[0131] FIG. 2 is a diagram illustrating the detailed configuration of a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 2, the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention is a real-time deepfake detection device (100) that detects deepfake attacks in a face recognition system using video captured by an RGB camera (200), and may be configured to include a preprocessing unit (110), a biosignal extraction unit (120), and a deep learning model unit (130).

[0132]

[0133] FIG. 3 is a diagram illustrating face recognition through a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 3, the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention detects deepfakes using non-contact biosignals extracted from a video captured by an RGB camera (200), and can perform face recognition if it is not a deepfake. Deepfake video is a fake video generated using artificial intelligence technology, and deepfake video used in face spoofing attacks is not a video of a real person. Therefore, non-contact biosignals extracted from such deepfake video do not represent the biosignal pattern of a real person. Accordingly, in a real-time deepfake detection device (100) through non-contact biosignal measurement according to one embodiment of the present invention, a non-contact biosignal is extracted from a face image to determine whether it is a face spoofing attack using deepfake technology, and if it is determined not to be a deepfake image, face recognition is performed with the face image, and if it is determined to be a deepfake image, face recognition is not performed.

[0134]

[0135] Hereinafter, with reference to FIGS. 2 and FIGS. 3, each component of the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention will be described in detail.

[0136]

[0137] The preprocessing unit (110) can detect movement and lighting changes in the video and filter lighting change attack data. In the present invention, a preprocessing step (S110) can be introduced to effectively detect spoofing attacks. In the preprocessing step (S110), unnecessary video data is filtered by detecting movement and lighting changes in the video, and only non-contact biosignals can be extracted in the biosignal extraction unit (120), which will be described in detail below.

[0138]

[0139] FIG. 4 is a diagram illustrating the detailed configuration of a preprocessing unit (110) in a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 4, the preprocessing unit (110) of the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention may be configured to include a motion detection unit (111) and a lighting change detection unit (112).

[0140]

[0141] The motion detection unit (111) can detect the direction of movement and the degree of movement of an object using optical flow values. More specifically, the motion detection unit (111) can calculate the detected direction of movement and the degree of movement as pixel values, and if the calculated pixel values ​​fall outside a preset threshold range, the corresponding data can be filtered. Thus, parts that are unsuitable for extracting biosignals due to large movement can be removed.

[0142]

[0143] The lighting change detection unit (112) can classify and filter the lighting change attack data if the amount of change in the Y value in YCbCr is greater than or equal to a preset threshold. More specifically, a video captured by an RGB camera (200) can be converted into YCbCr, and a lighting change attack can be detected using the amount of change in the Y value, which is the luminance component, in the converted YCbCr. That is, if the lighting change is greater than or equal to the threshold, it is determined to be a lighting change attack attempting spoofing through the lighting change, and the corresponding part can be removed.

[0144]

[0145] The biosignal extraction unit (120) can detect a face area in a video and extract a non-contact biosignal. That is, the biosignal extraction unit (120) is configured to extract biosignals in real time from a video, and the biosignals extracted here may include pulse, blood pressure, oxygen saturation, etc.

[0146]

[0147] FIG. 5 is a diagram illustrating a biosignal extraction unit (120) in a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 5, the biosignal extraction unit (120) of the real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention can detect a face region in a video captured by an RGB camera (200) and extract an rPPG (remote photoplethysmography) signal. In addition, pulse, blood pressure, and oxygen saturation can be calculated using the rPPG signal.

[0148]

[0149] More specifically, the biosignal extraction unit (120) can obtain a filtered rPPG signal by using a chrominance (CHROM) algorithm to remove noise components and remove breathing tendencies and trends for the rPPG signal, and by applying Butterworth bandpass filtering to remove components unrelated to heart activity. In addition, the biosignal extraction unit (120) can calculate the pulse, blood pressure, and oxygen saturation by detecting the maximum power peak using the YCbCr color values ​​decomposed into three signals (Y, Cb, and Cr) and the power spectral density (PSD) in addition to the filtered rPPG signal. The pulse in BPM units can be calculated by multiplying the detected maximum power peak by 60.

[0150]

[0151] The deep learning model unit (130) can learn non-contact biosignals using a deep learning model and detect spoofing attacks using deepfake technology using the learned deepfake detection model. That is, the deep learning model unit (130) can generate a learned deepfake detection model through supervised learning using a dataset and detect deepfakes in input videos using the learned deepfake detection model. The deep learning model unit (130) of the present invention can learn biosignals and detect spoofing attacks using three deep learning models.

[0152]

[0153] FIG. 6 is a diagram illustrating the detailed configuration of a deep learning model unit (130) in a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating the generation of a deepfake detection model in the deep learning model unit (130) of a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 6 and FIG. 7, the deep learning model unit (130) of a real-time deepfake detection device (100) through non-contact biosignal measurement according to an embodiment of the present invention may be configured to include a first learning unit (131), a second learning unit (132), and a third learning unit (133), and may further include a prediction unit (134).

[0154]

[0155] The first learning unit (131) can learn an LSTM (Long Short-Term Memory) model with an rPPG signal. An LSTM model suitable for learning time series data can be used to learn the sequential characteristics of the rPPG signal.

[0156]

[0157] The second learning unit (132) can convert the rPPG signal into frequency to train a CNN model. The CNN model is suitable for learning the frequency characteristics of a time series signal.

[0158]

[0159] The third learning unit (133) can train a CRNN (Convolutional Recurrent Neural Network) model with pulse, blood pressure, and oxygen saturation. The CRNN model is a model that combines CNN and RNN to learn features extracted from an image in a time-series manner, and can learn the time-series characteristics of rPPG, pulse, blood pressure, and oxygen saturation.

[0160]

[0161] The three model outputs trained in the first learning unit (131), the second learning unit (132), and the third learning unit (133), respectively, can be combined using an ensemble algorithm such as soft voting to produce a final output. In this way, a deepfake detection model can be constructed by combining an LSTM model, a CNN model, and a CRNN model to produce a final output.

[0162]

[0163] The first learning unit (131), the second learning unit (132), and the third learning unit (133) can be trained with a dataset consisting of normal videos and deepfake videos, and can apply data augmentation techniques to increase the number of deepfake videos. Data for training can be trained to classify rPPG, pulse, blood pressure, and oxygen saturation data extracted by performing preprocessing in the same manner as the preprocessing unit (110) into each model. Through training, the LSTM model trained in the first learning unit (131) had an accuracy of 98.7979 and an AUC of 0.9970, the CNN model trained in the second learning unit (132) had an accuracy of 99.7424 and an AUC of 0.9997, and the CRNN model trained in the third learning unit (133) had an accuracy of 97.6817 and an AUC of 0.9961.

[0164]

[0165] The prediction unit (134) receives a preprocessed video from a preprocessing unit (110) that processes a video captured by an RGB camera (200) in real time, and inputs it into a deepfake detection model to output a deepfake detection result. At this time, the prediction unit (134) can perform binary classification as deepfake or normal.

[0166]

[0167] FIG. 8 is a diagram illustrating the flow of a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 8, the real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention is a real-time deepfake detection method in which each step is performed on a computer to detect deepfake attacks in a face recognition system using video captured by an RGB camera (200). It may be implemented by including: a preprocessing step (S110) that detects movement and lighting changes in the video and filters lighting change attack data; a biosignal extraction step (S120) that detects a face region in the video and extracts non-contact biosignals; and a deep learning model step (S130) that learns non-contact biosignals using a deep learning model and detects spoofing attacks using deepfake technology using a deepfake detection model that has completed learning.

[0168]

[0169] FIG. 9 is a diagram illustrating the detailed flow of a preprocessing step (S110) in a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 9, the preprocessing step (S110) of the real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention may be implemented by including: a motion detection step (S111) that detects the direction and degree of movement of an object using an optical flow value; and a lighting change detection step (S112) that classifies and filters the data as lighting change attack data if the amount of change in the Y value in YCbCr is greater than or equal to a preset threshold value.

[0170]

[0171] FIG. 10 is a diagram illustrating the detailed flow of the deep learning model step (S130) in a real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention. As shown in FIG. 10, the deep learning model step (S130) of the real-time deepfake detection method through non-contact biosignal measurement according to an embodiment of the present invention comprises: a first learning step (S131) ​​of learning an LSTM (Long Short-Term Memory) model using an rPPG signal; and a second learning step (S132) of learning a CNN model by converting the rPPG signal into a frequency. It can be implemented by including a third training step (S133) for training a CRNN (Convolutional Recurrent Neural Network) model with pulse, blood pressure, and oxygen saturation, and can further include a prediction step (S134) for receiving a preprocessed video from a preprocessing step (S110) that processes a video captured by an RGB camera (200) in real time, inputting it into a deepfake detection model, and outputting a deepfake detection result.

[0172]

[0173] Since the details regarding each step have been sufficiently explained in relation to the real-time deepfake detection device (100) through non-contact biosignal measurement according to one embodiment of the present invention, a detailed explanation will be omitted.

[0174]

[0175] Meanwhile, the present invention may include a computer-readable medium comprising program instructions for performing operations implemented by various communication terminals. For example, the computer-readable medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory.

[0176]

[0177] Such a computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. In this case, the program instructions recorded on the computer-readable medium may be those specifically designed and configured to implement the present invention, or they may be those known and available to those skilled in the art of computer software. For example, they may include not only machine code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

[0178]

[0179] As described above, according to the real-time deepfake detection device (100) and method through non-contact biosignal measurement proposed in the present invention, non-contact biosignals are extracted from a face region in a video captured by an RGB camera (200), and a deep learning model is trained with the extracted non-contact biosignals to determine whether the face is a real person's face through changes in blood flow volume caused by heartbeats, thereby detecting spoofing attacks using deepfake technology.

[0180]

[0181] In addition, according to the real-time deepfake detection device (100) and method through non-contact biosignal measurement proposed in the present invention, by including a preprocessing unit (110) that detects movement and lighting changes in a video and filters lighting change attack data, it is possible to perform stable face recognition without being sensitive to lighting changes and the angle of the face, which are disadvantages of existing face recognition technology.

[0182]

[0183] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0184]

[0185] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

Claims

1. A real-time deepfake detection device (100) for detecting deepfake attacks in a face recognition system using video captured by an RGB camera (200), A preprocessing unit (110) that detects movement and lighting changes in the above video and filters lighting change attack data; A biosignal extraction unit (120) that detects a face region in the above video and extracts a non-contact biosignal; and A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by including a deep learning model unit (130) that learns the non-contact biosignal using a deep learning model and detects spoofing attacks using deepfake technology using a deepfake detection model that has completed learning.

2. In paragraph 1, the preprocessing unit (110) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by including a motion detection unit (111) that detects the direction and degree of movement of an object using an optical flow value.

3. In paragraph 2, the motion detection unit (111) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by calculating the detected direction of movement and degree of movement as pixel values, and filtering the data when the calculated pixel values ​​fall outside a preset threshold range.

4. In paragraph 2, the preprocessing unit (110) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by further including a lighting change detection unit (112) that classifies and filters as lighting change attack data if the amount of change in the Y value in YCbCr is greater than or equal to a preset threshold value.

5. In paragraph 1, the biosignal extraction unit (120) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by detecting a face region in the above video, extracting rPPG (remote photoplethysmography) signals, and calculating pulse, blood pressure, and oxygen saturation.

6. In paragraph 5, the biosignal extraction unit (120) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by obtaining a filtered rPPG signal by removing noise components using a chrominance (CHROM) algorithm and removing breathing tendencies and trends from the rPPG signal, and removing components unrelated to heart activity by applying Butterworth bandpass filtering.

7. In paragraph 1, the biosignal extraction unit (120) is, A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by detecting a maximum power peak using the filtered rPPG signal, YCbCr color values ​​decomposed into three signals (Y, Cb, Cr), and power spectral density (PSD), to calculate pulse, blood pressure, and oxygen saturation.

8. In paragraph 1, the deep learning model unit (130) is, A first learning unit (131) that learns an LSTM (Long Short-Term Memory) model using the above rPPG signal; A second learning unit (132) that converts the above rPPG signal into a frequency to train a CNN model; and A real-time deepfake detection device (100) through non-contact biosignal measurement, characterized by including a third learning unit (133) that trains a CRNN (Convolutional Recurrent Neural Network) model using the pulse, blood pressure, and oxygen saturation.

9. A real-time deepfake detection method in which each step is performed on a computer to detect deepfake attacks in a face recognition system using video captured by an RGB camera (200), (1) A preprocessing step (S110) that detects movement and lighting changes in the above video and filters lighting change attack data; (2) A biosignal extraction step (S120) for detecting a face region in the above video and extracting a non-contact biosignal; and (3) A method for detecting real-time deepfake through non-contact biosignal measurement, characterized by including a deep learning model step (S130) of learning the non-contact biosignal using a deep learning model and detecting a spoofing attack using deepfake technology using a deepfake detection model that has been learned.

10. In claim 9, the preprocessing step (S110) is, A real-time deepfake detection method through non-contact biosignal measurement, characterized by including a motion detection step (S111) that detects the direction and degree of movement of an object using an optical flow value.

11. In Clause 10, in the motion detection step (S111), A real-time deepfake detection method through non-contact biosignal measurement, characterized by calculating the detected movement direction and degree of movement as pixel values, and filtering the data when the calculated pixel values ​​fall outside a preset threshold range.

12. In Clause 10, the above preprocessing step (S110) is, A real-time deepfake detection method through non-contact biosignal measurement, characterized by further including a lighting change detection step (S112) that classifies and filters as lighting change attack data if the amount of change in the Y value in YCbCr is greater than or equal to a preset threshold value.

13. In claim 9, in the biosignal extraction step (S120), A real-time deepfake detection method through non-contact biosignal measurement, characterized by detecting a face region in the above video, extracting rPPG (remote photoplethysmography) signals, and calculating pulse, blood pressure, and oxygen saturation.

14. In Clause 13, in the biosignal extraction step (S120), A real-time deepfake detection method through non-contact biosignal measurement, characterized by removing noise components and breathing tendencies and trends using a chrominance (CHROM) algorithm for the above rPPG signal to obtain a normalized signal, and applying Butterworth bandpass filtering to remove components unrelated to cardiac activity to obtain a filtered rPPG signal.

15. In claim 9, in the biosignal extraction step (S120), A real-time deepfake detection method through non-contact biosignal measurement, characterized by detecting a maximum power peak using the filtered rPPG signal, YCbCr color values ​​decomposed into three signals (Y, Cb, and Cr), and power spectral density (PSD) to calculate pulse, blood pressure, and oxygen saturation.

16. In claim 9, the deep learning model step (S130) is, A first training step (S131) ​​of training an LSTM (Long Short-Term Memory) model with the above rPPG signal; A second training step (S132) for converting the above rPPG signal into frequency and training a CNN model; and A method for detecting real-time deepfake through non-contact biosignal measurement, characterized by including a third training step (S133) of training a CRNN (Convolutional Recurrent Neural Network) model using the pulse, blood pressure, and oxygen saturation.

Images