Non-invasive method and system for determining the probability of deception

Abstract

The invention relates to a non-invasive system and method for determining the probability of deception, said invention being based on remotely measuring the physiological changes exhibited by the subject being evaluated when they interact with stimuli provided by the system after an avatar-assisted pre-interview. The system comprises an interaction module, a facial recognition and localisation module, a processing module, and a probability of deception determination module. The system records the conscious responses of the subject being evaluated, and determines the subject's physiological variables and the changes therein caused by stress and cognitive load in response to the generated stimuli in order to calculate the probability of deception in the subject's conscious responses.

Description

[0001] NON-INVASIVE SYSTEM AND METHOD FOR DETERMINING THE PROBABILITY OF DECEPTION

[0002] FIELD OF INVENTION

[0003] The present invention relates to systems and methods for estimating the probability of deception of a subject, based on the variation of physiological signals caused by cognitive load and stress derived from deception.

[0004] BACKGROUND OF THE INVENTION

[0005] Deception testing systems are widely used in security procedures by government entities, the military, and private companies. The objective of these systems is to determine whether the person being tested is responding deceptively or honestly to questions.

[0006] Most deception detection methodologies have focused on the physical measurement of physiological responses resulting from activation of the ANS (autonomic nervous system). Numerous studies rely on polygraphs, which typically include cardiac activity sensors, electrodermal change sensors, and respiratory pattern sensors.

[0007] Other technologies use specialized sensors, such as infrared sensors to detect thermal changes in the skin, or changes in estimated percentage pupil diameters, changes in brain electrical patterns (EEG), or even a combination of all of the above. However, there are situations in which the subject to be evaluated cannot be physically present; instead, the interaction is conducted remotely, such as in personnel recruitment processes, insurance claim processes, and / or negotiation processes in general, among others. In these situations, it is necessary to detect the possibility of deception on the part of the subject in order to avoid an unfavorable negotiation, an inadequate insurance claim, or the hiring of individuals who do not meet the expectations of the entities or companies.

[0008] However, there are no deception detectors in the art that can operate remotely, since all existing systems are based on the premise of operating in physical contact with the subject being evaluated to obtain physiological signals, or at a reasonable distance from them, always in person, due to the complexity of the signals to be analyzed and the method of obtaining these signals.

[0009] There is therefore a need in the art for a remote system that evaluates changes in a plurality of physiological variables and relates them to responses to stimuli to obtain a calculation of the probability of deception, without the need for the subject being evaluated to be in the same location as the interviewer or the entity that requires the evaluation.

[0010] BRIEF DESCRIPTION OF THE INVENTION

[0011] The present disclosure seeks to provide a noninvasive deception detection system and method that can also be implemented remotely. Specifically, it eliminates the need to physically connect a series of sensors to the interviewee. The system and method disclosed herein extract information from the subject being evaluated; this information is captured through a device with a webcam, microphone, and / or speaker. The information obtained is used to determine the subject's physiological variables, along with their changes in response to different stimuli, to determine the probability of deception.

[0012] The remote deception detection system according to the present disclosure comprises several modules, including: an interaction module, a face recognition and localization module, a processing module, and a deception probability determination module.

[0013] The system is configured to create an avatar that reflects one or more physical features of the subject being evaluated. This avatar is configured to ask the subject interview-style questions during an initial stage. In addition, the system is configured to also provide visual, auditory, or combined stimuli other than the avatar.

[0014] Additionally, the system is configured to process remote photoplethysmographic signals acquired from sequenced images of the subject being evaluated and the background against which said subject is standing, thereby extracting physiological variables such as heart rate variability (HRV), vasoconstriction and vasodilation, among others. The screen in front of the interviewee and ambient light are used as light sources to acquire these signals.

[0015] Similarly, image analysis is performed to determine changes in blinking and facial expression variations caused by stress and cognitive load in response to the stimulus generated by the interaction module. Likewise, auditory signals and their changes are processed to infer respiratory patterns and artifacts such as apneas and others. The deception determination module uses the one or more variables obtained, their changes, and the supplied stimuli in a machine learning algorithm that outputs the probability of deception in the conscious responses made by the evaluated subject.

[0016] Another aspect of the invention relates to a non-intrusive method for determining the probability of deception. Said method includes the steps of: a) acquiring images of the subject being evaluated and the background in which they are located; b) locating the subject's face and identifying the subject's physical features; c) creating an avatar containing one or more of the subject's physical features; d) conducting a pre-interview on general topics using the avatar; e) providing visual, auditory, and / or combined stimuli to the subject being evaluated; f) locating points of interest on the subject's face and the background; g) obtaining photoplethysmographic signals of the face and changes in the background lighting where the subject being evaluated is located; h) processing the signals using filtering strategies; i) compensating the signals for movements of the subject being evaluated; j) processing the signals by eliminating artifacts; k) identifying blinking of the subject being evaluated;I) determine physiological variables inferred from the analysis of the signals obtained; k) determine changes in the physiological variables; and I) calculate the probability of deception from the changes in the physiological variables and the stimuli supplied to the subject being evaluated using machine learning methods based on artificial intelligence.

[0017] BRIEF DESCRIPTION OF THE FIGURES

[0018] Figure 1 shows a schematic diagram of the modules of the non-invasive system for detecting probability of deception according to the present disclosure.

[0019] Figure 2 shows a flowchart of the noninvasive method for determining the probability of deception according to the present disclosure. Figure 3 shows framing guides displayed on the screen that allow the subject's face to be correctly positioned within the system and method of the present disclosure.

[0020] DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention provides a non-invasive deception detection system and method that can be implemented remotely, eliminating the need to physically connect a series of sensors to the subject being evaluated. The system and method disclosed herein extract information from the evaluated subject's face, captured through a device that includes a webcam, microphone, and / or speaker, and determine facial expressions and physiological variables, and their changes with respect to different stimuli, to determine the probability of deception.

[0022] Among the physiological variables the system evaluates are heart rate (HRV) variables inferred from color changes produced by light reflection on a person's facial skin, changes in blinking frequency and duration, facial expressions, changes in breathing patterns, and changes in vasoconstriction and vasodilation.

[0023] As shown in Figure 1, the non-invasive system for detecting the probability of deception disclosed here comprises several modules, including: an interaction module (101), a face recognition and localization module (102), a processing module (103), and a deception probability determination module (104).

[0024] Preferably, the interaction module (101) comprises a video camera, a processor, a display, a microphone and a sound reproducing device. The interaction module (101) is configured to generate stimuli and to acquire information.

[0025] In a preferred embodiment, the generated stimuli may be visual, auditory, or a combination of these. Additionally, the interaction module is configured to create an avatar that has a humanoid voice and appearance and adopts one or more physical features of the subject being evaluated. The interaction module (101) is also configured to emit a sound different from the stimulus, at a frequency inaudible to the human ear.

[0026] In a preferred embodiment of the invention, the display of the interaction module, together with ambient light, illuminates the subject being evaluated and the background against which the subject stands during the acquisition of the photoplethysmographic signals. Additionally, the display is configured to indicate framing guides, which will serve as a guide for the evaluated subject to correctly position his or her face in front of the camera. The interaction module (101) also captures information from the microphone channel for subsequent processing.

[0027] Preferably, the light source coming from the screen is configured to implement lighting strategies, that is, it allows for changes in lighting projected from the screen, both in light intensity, modifying what is emitted on the screen in brightness and color spectrum. Most digital cameras include algorithms that change the contrast when capturing video. The calibration system used allows us to infer how the camera adapts to changes in lighting and color under known lighting levels for the processing stage. This also allows us to understand the variations between different hardware, something that represents an improvement over existing methods. In a preferred embodiment of the invention, the camera is configured to take video of the subject being evaluated, mainly the face and some points in the background in which the subject is located.The recording sampling frequency is stored between 15 to 30 Hz, typically being saved at 25 Hz, the resolution is at least 640x480 pixels in size within which the frame of the person's face and the background will be during the development of the interview.

[0028] In the context of the present invention, the processing module (103) comprises three units; a remote photoplethysmographic signal processing unit, which is configured to acquire physiological signals from the photoplethysmographic signals such as heart rate variability (HRV) and changes in vasoconstriction and vasodilation; an acoustic signal processing unit, which is configured to infer respiratory patterns; and an image processing unit, which is configured to determine the frequency and duration of blinking and micro expressions of the face of the evaluated subject. Additionally, the processing module (103) is configured to identify and count the number of people or faces detected in the frame, as well as to identify that the evaluated subject matches the subject that was initially validated.

[0029] In a preferred embodiment of the invention, the deception probability determination module (104) is configured to receive as input the data of one or more of the physiological variables obtained from the other modules, their changes, and information of the stimuli presented, so that the probability of deception of the conscious responses of the evaluated subject is obtained as a result.

[0030] According to one embodiment of the invention, the interaction module is configured to emit a sound at a frequency inaudible to the human ear during the interview and record the reflected sound, similar to sonar. Another aspect of the invention relates to a non-intrusive method for determining the probability of deception. Figure 2 shows a flowchart of said method according to one embodiment of the present invention, which includes the steps of: a) acquiring images of the subject being evaluated and the background in which he or she is located; b) locating the subject's face and identifying the subject's physical features; c) creating an avatar containing one or more of the subject's physical features; d) conducting a pre-interview on general topics using the avatar; e) providing visual, auditory, and / or combined stimuli to the subject being evaluated; f) locating points of interest on the subject's face and the background;g) obtain photoplethysmographic signals of the face and changes in background illumination where the subject being evaluated is located; h) process the signals using filtering strategies; i) compensate the signals for movements of the subject being evaluated; j) process the signals by eliminating artifacts; k) identify the blinking of the subject being evaluated; I) determine physiological variables inferred from the analysis of the signals obtained; k) determine changes in the physiological variables; and I) calculate the probability of deception from the changes in the physiological variables and the stimuli supplied to the subject being evaluated using machine learning methods based on artificial intelligence.

[0031] In a preferred embodiment of the invention, a pre-interview is conducted with the subject being evaluated, assisted by an avatar that has a humanoid voice and appearance and also contains one or more of the subject's physical features. Preferably, during the main interview, the subject is subjected to visual or auditory stimuli, or a combination of both.

[0032] To capture images of the subject being evaluated, the subject is positioned so that they align with framing guides displayed on the screen, which allow the subject's face to be correctly positioned. The light emitted from the screen is predominantly green and surrounds the framing guides to facilitate facial illumination at the spectrum frequency being analyzed.

[0033] Preferably, the images are acquired taking into account a color and light intensity setting adjustable by the screen located in front of the subject being evaluated. The acquired images contain information about both the face of the subject being evaluated and the background against which the subject is positioned. In this way, information is obtained about changes in ambient lighting.

[0034] In the context of the present invention, face localization is implemented using computer vision algorithms in a processor. Background subtraction techniques are employed to differentiate between the face of the subject being evaluated and the background against which the subject is standing. This additionally makes it possible to identify whether changes in light intensity and composition on the face are due to physiological changes or are generated by the environment. Since, if the changes are generated by the environment, they would also be reflected in areas other than the face.

[0035] In the context of the present invention, the non-intrusive method for determining the probability of deception analyzes the light reflected on the face of the subject being evaluated at a frequency compatible with that of his cardiac activity, identifying variations in the light frequencies emitted by the screen and the environment.

[0036] According to one embodiment of the invention, in an initial stage of the method, the evaluated subject undergoes a pre-interview, assisted by an avatar that has a humanoid voice and appearance and shares one or more physical features with the evaluated subject. This provides the subject with reassurance and familiarity, reducing the anxiety and stress associated with the situation and is not associated with deception. The pre-interview is conducted by raising general topics.

[0037] In a subsequent stage, the main interview is conducted, during which visual, auditory, or a combination of both stimuli are generated. Additionally, the subject being evaluated can interact in a way that allows them to accept, deny, or explain each of the questions presented.

[0038] In a preferred embodiment of the invention, the method also compensates the photoplethysmographic signals and the acoustic signal for respiratory movements or natural movements of the subject being evaluated. Likewise, the signals obtained are filtered and their artifacts are eliminated. The techniques used for filtering and artifact removal are selected from the group of Fast Fourier Transform (FFT), Welch, Kalman Filter, Adaptive Filter, among others. Likewise, the method disclosed here comprises generating a sound during the interview, different from the auditory stimulus, at a frequency inaudible to the human ear and obtaining the reflected sound signal, similar to sonar.

[0039] According to the method revealed here, the acoustic signal analyzed corresponds to the reflected sound signal of the signal emitted at a frequency inaudible to the human ear, so that the respiratory patterns are inferred.

[0040] To eliminate artifacts originating from ambient lighting, the principle of concurrency between background and facial illumination is used to distinguish between facial lighting artifacts of environmental and non-physiological origin. Additionally, the respiratory waveform inferred from the acoustic signal is compared with the respiratory signal derived from the cardiac signal, thereby identifying artifacts such as apneas and others. Once the photoplethysmographic signal is clean, cardiac peaks are estimated to analyze which ones should be considered in the analysis and those that should be rejected. In this way, the cardiovascular activity of the subject being evaluated is obtained, resulting in a cardiac graph.Additionally, information can be obtained on heart rate, heart rate variability, blood pressure, respiration, and an estimate of relative changes in blood oxygen levels.

[0041] In the context of the present invention, facial image processing employs machine vision techniques, which assess changes in facial expressions and evaluate the ocular structure to analyze the subject's blinking. This assessment comprises: tracking the detected points of the eye and their horizontal variation over time during blinking with a mathematical estimate of relative coordinates.

[0042] In a preferred embodiment of the invention, the probability of deception is calculated from one or more physiological variables obtained, such as: heart rate variability (HRV), changes in blink frequency and duration, changes in facial expressions, changes in breathing patterns, changes in vasoconstriction and vasodilation, and response time. The changes in these variables, along with information from the presented stimuli, are input into a machine learning algorithm that outputs the probability of deception from the conscious responses of the evaluated subject.

[0043] The following table presents examples of different combinations of stimuli, and the different changes in physiological variables that are prioritized to determine the probability of deception: STIMULI PRIORITIZED VARIABLES

[0044] The embodiments disclosed herein exemplify the invention, which is not limited to such embodiments, and its scope may include obvious variations and modifications of such embodiments.

Claims

CLAIMS 1. A non-invasive system for detecting the probability of deception by a subject, characterized in that it comprises: an interaction module, configured to generate stimuli for the evaluated subject, obtain images that include a face of an interviewed subject and a background where the evaluated subject is located, obtain photoplethysmographic signals, obtain acoustic signals; where the interaction module is configured so that an avatar carries out a pre-interview; a face recognition and localization module, which is configured to differentiate the subject's face from the background from the images acquired by the interaction module; a processing module, which comprises three processing units: a photoplethysmographic signal processing unit, an acoustic signal processing unit, and an image processing unit;wherein the processing module is configured to process the received signals and identify physiological variables and changes in said physiological variables in response to the stimuli that the subject has received from the interaction module; and a probability determination module that, based on the changes in the physiological variables and the given stimuli, determines the probability of the subject deceiving their conscious responses.

2. The system according to Claim 1, wherein the avatar that performs the pre-interview contains a humanoid appearance and adapts one or more physical features of the subject evaluated.

3. The system according to Claim 1, wherein the information acquisition module uses a screen as a light source.

4. The system according to Claim 1 wherein the interaction module is configured to conduct a main interview with the subject, in which the interviewed subject can interact by accepting, denying or explaining the statements.

5. The system according to Claim 1, wherein the interaction module is configured to emit a sound at a frequency inaudible to the human ear during the interview and record the reflected sound.

6. The system according to any of the preceding Claims, wherein the processing module is configured to filter the photoplethysmographic and acoustic signals.

7. The system according to any of the preceding Claims, wherein the processing module is configured to eliminate artifacts from the photoplethysmographic and acoustic signals.

8. The system according to any of the preceding Claims, wherein the processing module is configured to compensate the signal for movements of the interviewed subject.

9. The system according to any of the preceding Claims, wherein the processing module is configured to count the number of people or faces detected in the frame, as well as identify that the evaluated subject matches the subject that was initially validated.

10. A non-invasive method for determining the probability of deception of a subject, characterized in that it comprises: acquiring images of the subject and the background in which he or she is located; locating the subject's face; identifying physical features of the evaluated subject; creating an avatar that contains one or more of the physical features of the evaluated subject; conducting a pre-interview assisted by the avatar on general topics; providing visual, auditory and / or combined stimuli to the evaluated subject; locating points of interest on the face of the evaluated subject and the background against which the subject is located; processing the signals using filtering strategies; compensating the signals for movements of the interviewed subject; processing the signals by eliminating artifacts; identifying blinking of the interviewed subject; determining physiological variables from the analysis of the signals obtained; Determine changes in physiological variables; calculate the probability of deception based on changes in physiological variables and the stimuli provided to the interviewee using machine learning methods.

11. The method of Claim 8, wherein framing guides are provided on a screen for the subject to align his or her face.

12. The method of Claim 8, wherein the screen is additionally configured to adjust the colors and amount of light of the environment where the subject is.

13. The method of Claims 8 to 10, wherein the localization of the subject's face is performed by computer vision algorithms, such as background tracking.

14. The method of Claims 8 to 11, further comprising generating a sound during the interview, different from the auditory stimulus, at a frequency inaudible to the human ear and obtaining the reflected sound signal.

15. The method of Claims 8 to 12, further comprising contrasting the respiratory signal obtained from the processing of the acoustic signal and the plethysmographic signal in order to identify artifacts such as apneas.

16. The method of Claim 8, wherein to determine the blinking of the interviewed subject, a computer recognition of the ocular structure is performed and then the changes in geometry are computed.

17. The method of Claims 8 to 13 wherein the determined physiological variables comprise: - changes in facial expressions; - heart rate-related variables (HRV); - changes in blinking frequency and duration; - changes in breathing patterns; and - changes in vasoconstriction and vasodilation - response times.

Images