Sound-Based Presentation Attack Detection

Abstract

Techniques for presentation attack detection are described that include a computer-implemented method of emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing the video conference. The computer-implemented method further includes collecting video data of a user engaged in the video conference by a camera communicatively coupled to the computer and overlapping with the emitting of the predetermined frequency pattern. The computer-implemented method further includes determining that the video data is inconsistent with an expected response to the predetermined frequency pattern. The computer-implemented method further includes generating an indication that a user engaged in the video conference is conducting a presentation attack.

Description

[Technical field]

[0001] The present disclosure relates to cybersecurity, and more particularly, to sound-based presentation attack detection. [Background technology]

[0002] A presentation attack is implemented by a malicious actor pretending to be another individual in order to engage in malicious activities such as providing credible but misleading information, fraudulently authorizing the transfer of money, goods, or services, or otherwise leveraging false user authentication to achieve a malicious outcome. Presentation attacks may be implemented, for example, through masks, photographs, pre-recorded videos, live deepfake videos, etc. Summary of the Invention

[0003] Aspects of the present disclosure are directed to a computer-implemented method that includes emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing a video conference. The computer-implemented method further includes collecting video data of a user engaged in the video conference by a camera communicatively coupled to the computer and overlapping with the emitting of the predetermined frequency pattern. The computer-implemented method further includes determining that the video data is inconsistent with an expected response to the predetermined frequency pattern. The computer-implemented method further includes generating an indication that a user engaged in the video conference is conducting a presentation attack.

[0004] Additional aspects of the disclosure are directed to systems and computer program products configured to implement the aforementioned methods. This Summary is not intended to represent every implementation or embodiment of each aspect of the disclosure, or every combination thereof.

[0005] The drawings included in this application are incorporated in and form a part of this specification. These drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. The drawings are merely illustrative of certain embodiments and do not limit the present disclosure. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 illustrates a block diagram of an exemplary computing environment for deploying presentation attack detection software in accordance with some embodiments of the present disclosure. [Diagram 2] 1 illustrates a flow diagram of an example method for presentation attack detection in a video conference, according to some embodiments of the present disclosure. [Figure 3A] 1 illustrates a flow diagram of an example method for presentation attack detection using induced vibration, according to some embodiments of the present disclosure. [Figure 3B] 1 illustrates a flow diagram of an example method for presentation attack detection using user motion, according to some embodiments of the present disclosure. [Figure 3C] 1 illustrates a flow diagram of an example method for presentation attack detection using user facial expressions, according to some embodiments of the present disclosure. [Figure 4] 1 illustrates a flow diagram of an example method for downloading, deploying, metering, and billing usage of presentation attack detection software, according to some embodiments of the present disclosure. [Diagram 5] 1 illustrates a block diagram of an exemplary computer according to some embodiments of the present disclosure. [Figure 6] 1 illustrates a cloud computing environment, according to some embodiments of the present disclosure. [Figure 7] 1 illustrates abstraction model layers according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] While the present disclosure is susceptible to various modifications and alternative forms, details thereof have been shown by way of example in the drawings and will be described in detail. It is to be understood, however, that the intention is not to limit the disclosure to the particular embodiments described. Instead, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure.

[0008] Aspects of the present disclosure are directed to cybersecurity, and more particularly, to sound-based presentation attack detection. Although not limited to such applications, embodiments of the present disclosure may be better understood in light of the foregoing context.

[0009] Presentation attack detection (also referred to as spoof detection or liveness detection) may be used to detect presentation attacks by various methods. Presentation attack detection methods may rely on factors such as blink detection, eye movement, pupil dilation, biometric factors (e.g., respiration, body temperature, blood flow, blood pressure, etc.), two-way user queries, light reflectivity, three-dimensional cameras for depth-based assessment, infrared cameras for heat pattern detection, optical flow fields, background noise analysis, etc. Presentation attack detection methods may be active or passive based on whether the person being authenticated needs to perform any special actions to prove their authenticity. Presentation attack detection methods may utilize various types of hardware such as video cameras, single frame cameras, thermal imaging cameras, cameras with three-dimensional (3D) depth perception capabilities, speakers, headphones, earphones, etc. Presentation attack detection methods have been standardized in the International Organization of Standardization / International Electrotechnical Commission (ISO / IEC) 30107-3, which includes principles and methods for evaluating the performance of presentation attack detection mechanisms, reporting of test results from evaluation of presentation attack detection mechanisms, and a classification of known attack types.

[0010] Aspects of the present disclosure are directed to techniques for presentation attack detection. One aspect of the present disclosure is a passive presentation attack detection mechanism whereby a predetermined frequency pattern is emitted from one or more speakers and a high resolution, high speed camera captures induced vibrations caused by the predetermined frequency pattern. The predetermined frequency pattern can induce vibrations in clothing fibers and / or hair of genuine users engaged in a video conference, where such vibrations would not be induced in the individual conducting the presentation attack. Advantageously, this aspect of the present disclosure can distinguish between malicious users and genuine users employing presentation attacks of most known presentation attack vectors. Additionally, this aspect of the present disclosure is a passive presentation attack detection mechanism that may be utilized without the need for any specific action to be performed by the user engaged in the video conference.

[0011] Another aspect of the present disclosure is an active presentation attack detection mechanism whereby a predetermined frequency pattern is emitted from one or more speakers and a video camera captures user movements and / or facial expressions in response to the predetermined frequency pattern. Expected user movements and / or expected changes in facial expressions may indicate a genuine user, while unexpected movements and / or unexpected changes in facial expressions may indicate a user conducting a presentation attack. Advantageously, this aspect of the present disclosure does not require specialized equipment and may be implemented using most consumer level video conferencing equipment.

[0012] Referring now to the figures, Figure 1 illustrates a block diagram of an exemplary computing environment 100 for deploying presentation attack detection software 136 in accordance with some embodiments of the present disclosure. The computing environment 100 includes a data processing system 102, a remote data processing system 134, and a plurality of computers 126 (e.g., a first computer 126-1 through an nth computer 126-N, where N may refer to any positive integer greater than 1) communicatively coupled to each other via a network 138. The network 138 may refer to a local area network (LAN), a wide area network (WAN), an intranet, the Internet, or any other data processing network useful for connecting multiple computing devices.

[0013] A plurality of computers 126 may engage in a video conference using videoconferencing software 132 (e.g., a first incarnation of videoconferencing software 132-1 corresponding to a first computer 126-1, an nth incarnation of videoconferencing software 132-N corresponding to an nth computer 126-N). Each computer 126 may include a speaker 128 (e.g., a speaker 128-1 corresponding to computer 126-1 and a speaker 128-N corresponding to computer 126-N) and a camera 130 (e.g., a camera 130-1 corresponding to computer 126-1 and a camera 130-N corresponding to computer 126-N). A speaker 128 may be any speaker communicatively coupled to its corresponding computer 126 and may be capable of emitting audio from the videoconference. Speakers 128 may be built into computer 126, connected by wires to computer 126 (e.g., a universal serial bus (USB) connection, an auxiliary (AUX) input connection, etc.), or connected wirelessly to computer 126 (e.g., a Bluetooth connection). Camera 130 may be any video camera capable of collecting video data 114 associated with computer 126 and of users engaged in video conferencing. In some embodiments, camera 130 is a high speed and / or high resolution video camera 130. In some embodiments, camera 130 is capable of capturing video, at least temporarily, at frame rates exceeding 500 frames per second (FPS), 700 FPS, 900 FPS, or 1,000 FPS in various embodiments.

[0014] The data processing system 102 may implement presentation attack detection. In some embodiments, the data processing system 102 downloads presentation attack detection software 136 from a remote data processing system 134. In some embodiments, the presentation attack detection software 136 is a component of the video conferencing software 132 (e.g., providing extensible functionality for presentation attack detection to a user of the video conferencing software 132 for verifying the authenticity of other participants in a video conference), while in other embodiments, the presentation attack detection software 136 is standalone software that may be deployed in conjunction with various types of video conferencing software 132. While FIG. 1 illustrates the data processing system 102 implementing the presentation attack detection software 136 (which may be downloaded from the remote data processing system 134), in other embodiments, the presentation attack detection software 136 may be downloaded to and implemented by the computer 126.

[0015] Data processing system 102 may be any computer, server, or other data processing system in which presentation attack detection software 136 may be deployed. In some embodiments, data processing system 102 is a server managed by video conferencing software 132 for supporting video conferencing between multiple computers 126. Data processing system 102 includes a predetermined frequency pattern 104. Predefined frequency pattern 104 may comprise different frequencies (including audible and inaudible frequencies), different volumes, and different durations collectively configured to test the authenticity of a user in a video conference.

[0016] For example, the predetermined frequency pattern 104 may be a vibration-inducing frequency 106, or a reactive cue 108, such as a directional cue 110 or an emotional cue 112, or both. The vibration-inducing frequency 106 may be a frequency that induces vibrations in small objects, such as thread fibers on clothing or human hair (e.g., arm hair, facial hair, head hair, etc.), or both. In some embodiments, the vibration-inducing frequency 106 is tuned to induce vibrations that can be captured by the frame rate of the camera 130.

[0017] The reactive cues 108 may be audible cues that elicit a response (e.g., a body movement or facial expression) from a user engaged in a video conference. For example, the directional cues 110 may be used to cause a user engaged in a video conference to react to a perceived directional noise by flinching, turning around, shifting their eyes, etc. The directional cues 110 may be, for example, an audible sound emitted from only one of the multiple speakers 128, thereby giving the directional cue 110 the appearance of a direction. Additionally, the directional cues 110 may be an audible sound that elicits a response from a person, for example, an audible sound that mimics a mosquito or other insect, an error tone, or a voice giving a command (e.g., "look this way").

[0018] Another example reactive cue 108 is an emotional cue 112. The emotional cue 112 can be an audible sound intended to elicit a change in facial expression of a user engaged in a video conference. For example, the emotional cue 112 can be a sound that interrupts the video conference, a sound that is unpleasant to the user (e.g., a dial tone with noise), a sound that is confusing to the user (e.g., a crying baby), a command (e.g., "please smile"), or other sound that will elicit a change in facial expression, such as irritation, confusion, or happiness.

[0019] The data processing system 102 further collects video data 114. The video data 114 may be data collected from the camera 130, where the video data 114 overlaps with the emission of the predetermined frequency pattern 104 from the speaker 128. More specifically, the video data 114 may begin prior to the emission of the predetermined frequency pattern 104 (to capture baseline vibrations, movements, or facial expressions) and during the emission of the predetermined frequency pattern 104 (to capture changes to the vibrations, movements, or facial expressions from the baseline). In some embodiments, the video data 114 also extends past the emission of the predetermined frequency pattern 104 to capture remaining reactions from the user to the predetermined frequency pattern 104. In other words, the video data 114 may include video that lasts longer than the predetermined frequency pattern 104.

[0020] In some embodiments, the video data 114 includes upscaled video data 116. The upscaled video data 116 may be a portion of the video data 114 that is upscaled to assess subtle changes, such as induced vibrations in yarn fibers or hair. In embodiments that utilize upscaled video data 116, the camera 130 that captures the video data 114 may be a high resolution camera.

[0021] The data processing system 102 further includes a liveness threshold 118. The liveness threshold 118 may be used to distinguish between users performing a presentation attack and genuine users in a video conference. The liveness threshold 118 may include a vibration threshold 120, a movement threshold 122, or an expression threshold 124, or a combination thereof. The vibration threshold 120 determines whether the vibration induced by the vibration-inducing frequency 106 in the yarn fibers or hair matches the expected vibration. The vibration threshold 120 may be a minimum vibration frequency (or a range of vibration frequencies) that, if exceeded (or within a range), results in a determination that the user is genuine, and that, if not exceeded (or outside of a range), results in a determination that the user is performing a presentation attack.

[0022] The movement threshold 122 determines whether the user movements induced by the directional cues 110 match expected movement patterns. The movement threshold 122 may be a minimum movement (or range of movement) threshold that, if exceeded (or within a range), results in a determination that the user is authentic, and that, if not exceeded (or outside of a range), results in a determination that the user is engaging in a presentation attack. In some embodiments, the movement threshold 122 is localized to a particular part of the user's body, for example, the user's head.

[0023] The facial expression threshold 124 determines whether the user facial expressions induced by the emotional cues 112 match the expected facial expressions. The facial expression threshold 124 may be a minimum change in facial expression (or a range of changes) that, if exceeded (or within a range), results in a determination that the user is authentic, and that, if not exceeded (or outside of a range), results in a determination that the user is engaging in a presentation attack.

[0024] In some embodiments, only one of the vibration induction frequency 106, the directional cues 110, and the emotional cues 112 is utilized to determine the authenticity of a user. In other embodiments, more than one of the vibration induction frequency 106, the directional cues 110, or the emotional cues 112, or a combination thereof, is used to determine the authenticity of a user. In some embodiments, the presentation attack detection software 136 may be used to authenticate each user engaged in a video conference, while in other embodiments, users may be selectively authenticated (e.g., in response to another user engaged in a video conference deploying the presentation attack detection software 136).

[0025] 2 illustrates a flow diagram of an example method 200 for presentation attack detection in a video conference in accordance with some embodiments of the present disclosure. In some embodiments, the method 200 is implemented by the data processing system 102, the computer 126, the remote data processing system 134, a processor, or another configuration of hardware or software or both.

[0026] The operation 202 includes emitting the predetermined frequency pattern 104 using at least one speaker 128 communicatively coupled to a computer 126 implementing the video conference. In some embodiments, the predetermined frequency pattern 104 is a vibration-inducing frequency 106 or a reactive cue 108, where the reactive cue 108 can be a directional cue 110 or an emotional cue 112.

[0027] Operation 204 includes collecting video data 114 of users engaged in a video conference by a camera 130 communicatively coupled to computer 126 and overlapping with the emission of the predetermined frequency pattern 104. In some embodiments, the video data 114 spans a longer period of time than the predetermined frequency pattern 104. In some embodiments, the video data 114 is high definition video data having a frame rate of greater than 500 FPS, 700 FPS, 900 FPS, or 1,000 FPS.

[0028] Operation 206 includes determining that the video data 114 is inconsistent with an expected response to the predetermined frequency pattern 104. In some embodiments, operation 206 compares the video data 114 to one or more liveness thresholds 118, such as a vibration threshold 120, a movement threshold 122, and / or an expression threshold 124, to determine whether the video data 114 is inconsistent with an expected response to the predetermined frequency pattern 104.

[0029] Operation 208 includes generating an indication that a user engaged in the video conference is conducting a presentation attack. Operation 208 may include transmitting the indication to one or more other computers 126 engaged in the video conference, to the data processing system 102, or to another entity. In some embodiments, operation 208 performs a mitigating action, such as providing a video conference connection to any computers 126 associated with the user determined to be conducting a presentation attack.

[0030] 3A illustrates a flow diagram of an example method 300 for presentation attack detection using induced vibration, according to some embodiments of the present disclosure. In some embodiments, the method 300 is implemented by the data processing system 102, the computer 126, the remote data processing system 134, a processor, or another configuration of hardware or software or both. In some embodiments, the method 300 is a sub-method of operation 206 of FIG. 2.

[0031] Operation 302 includes magnifying a portion of the video data 114 to generate augmented video data 116. The augmented video data 116 may be sufficiently magnified to detect vibrations in the yarn fibers and / or hairs. Operation 304 determines whether there is induced vibrations in the augmented video data 116 originating from the vibration-inducing frequency 106. Operation 304 may use a vibration threshold 120 to determine whether there is induced vibrations. If so (304: yes), then the method 300 proceeds to operation 306, where it is determined that the user is authentic. If not (304: no), then the method 300 proceeds to operation 308, where it is determined that the user is conducting a presentation attack.

[0032] 3B illustrates a flow diagram of an exemplary method for presentation attack detection using user motion, according to some embodiments of the present disclosure. In some embodiments, the method 310 is implemented by the data processing system 102, the computer 126, the remote data processing system 134, a processor, or another configuration of hardware or software or both. In some embodiments, the method 310 is a sub-method of operation 206 of FIG. 2.

[0033] Operation 312 includes monitoring the video data 114 for user movement. In some embodiments, operation 312 monitors the video data 114 during and / or after emission of the directional cue 110 and compares the user position during and / or after emission of the directional cue 110 to the user position prior to emission of the directional cue 110. Operation 314 determines whether there was user movement that was expected as a result of emission of the directional cue 110. Operation 314 may utilize a movement threshold 122 to distinguish expected movement from unexpected movement. If so (314: yes), then method 310 proceeds to operation 316 and makes a determination that the user is a genuine user. If not (314: no), then method 310 proceeds to operation 318 and makes a determination that the user is conducting a presentation attack.

[0034] 3C illustrates a flow diagram of an exemplary method for presentation attack detection using user facial expressions, according to some embodiments of the present disclosure. In some embodiments, the method 320 is implemented by the data processing system 102, the computer 126, the remote data processing system 134, a processor, or another configuration of hardware or software or both. In some embodiments, the method 320 is a sub-method of operation 206 of FIG. 2.

[0035] Operation 322 includes monitoring the video data 114 for the user's facial expressions. In some embodiments, operation 322 monitors the video data 114 during and / or after the emission of the emotional cue 112 and compares the user's facial expressions during and / or after the emission of the emotional cue 112 to the user's facial expressions prior to the emission of the emotional cue 112. Operation 324 determines whether there was an expected change in facial expression as a result of the emission of the emotional cue 112. Operation 324 may utilize an expression threshold 124 to distinguish an expected change in facial expression from an unexpected change in facial expression (or no change in facial expression). If so (324: yes), then the method 320 proceeds to operation 326 and makes a determination that the user is a genuine user. If not (324: no), then the method 320 proceeds to operation 328 and makes a determination that the user is conducting a presentation attack.

[0036] 4 illustrates a flow diagram of an exemplary method 400 for downloading, deploying, metering, and billing usage of presentation attack detection software 136, in accordance with some embodiments of the present disclosure. In some embodiments, method 400 is implemented by data processing system 102, computer 126, remote data processing system 134, processor, or another configuration of hardware or software or both. In some embodiments, method 400 occurs concurrently with other methods discussed in this disclosure.

[0037] Operation 402 includes downloading the presentation attack detection software 136 from the remote data processing system 134 and to one or more computers (e.g., data processing system 102, computer 126). Operation 404 includes executing the presentation attack detection software 136. Operation 404 may include performing any of the methods discussed herein. Operation 406 includes metering usage of the presentation attack detection software 136. Usage may be metered, for example, by the amount of time the presentation attack detection software 136 is used, the number of workstations that deploy the presentation attack detection software 136, the amount of resources consumed by implementing the presentation attack detection software 136, or other usage metering metrics or combinations thereof. Operation 408 includes generating an invoice based on the usage metering.

[0038] FIG. 5 illustrates a block diagram of an exemplary computer 500 according to some embodiments of the present disclosure. In various embodiments, the computer 500 can perform any or all portions of the methods described in FIGS. 2 and 3A-3C or implement the functionality discussed in FIG. 1, or both. In some embodiments, the computer 500 receives instructions related to the aforementioned methods and functionality by downloading processor-executable instructions from a remote data processing system (e.g., the remote data processing system 134 of FIG. 1) over a network 550. In other embodiments, the computer 500 provides instructions for the aforementioned methods and / or functionality to a client machine (e.g., the data processing system 102 and / or the computer 126 of FIG. 1) such that the client machine executes the method, or a portion of the method, based on the instructions provided by the computer 500. In some embodiments, computer 500 is incorporated into (or functionality similar to computer 500 is virtually provided for) one or more of the entities illustrated in FIG. 1 and / or other aspects of the disclosure (e.g., remote data processing system 134, data processing system 102, and / or computer 126).

[0039] The computer 500 includes memory 525, storage 530, an interconnect 520 (e.g., a bus), one or more CPUs 505 (also referred to herein as processors), an I / O device interface 510, I / O devices 512, and a network interface 515.

[0040] Each CPU 505 retrieves and executes programming instructions stored in memory 525 or storage 530. The interconnect 520 is used to move data, e.g., programming instructions, between the CPU 505, the I / O device interface 510, the storage 530, the network interface 515, and the memory 525. The interconnect 520 may be implemented using one or more buses. The CPU 505 may be a single CPU, multiple CPUs, or a single CPU with multiple processing cores, in various embodiments. In some embodiments, the CPU 505 may be a digital signal processor (DSP). In some embodiments, the CPU 505 includes one or more 3D integrated circuits (3DICs) (e.g., 3D wafer level packaging (3DWLP), 3D interposer-based integration, 3D stacked IC (3D-SIC), monolithic 3DIC, 3D heterogeneous integration, 3D system-in-package (3DSiP), and / or package-on-package (PoP) CPU configurations). Memory 525 is included generally to represent random access memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), or flash). Storage 530 is included generally to represent non-volatile memory, e.g., a hard disk drive, a solid state device (SSD), a removable memory card, optical storage, or a flash memory device. In an alternative embodiment, storage 530 may be replaced by a storage area network (SAN) device, cloud, or other device connected to computer 500 via I / O device interface 510 or to network 550 via network interface 515.

[0041] In some embodiments, memory 525 stores instructions 560. However, in various embodiments, instructions 560 are stored partially in memory 525 and partially in storage 530, or instructions 560 are stored entirely in memory 525 or entirely in storage 530, or instructions 560 are accessed over network 550 via network interface 515.

[0042] The instructions 560 may be computer readable and computer executable instructions for performing any portion or all of the method of Figure 2 or Figures 3A-3B or a combination thereof, or may implement the functionality discussed in Figure 1, or both. Although the instructions 560 are shown in memory 525, the instructions 560 may include program instructions collectively stored across a number of computer readable storage media and executable by one or more CPUs 505.

[0043] In various embodiments, the I / O device 512 includes an interface capable of presenting information and receiving input. For example, the I / O device 512 can present information to, and receive input from, a user interacting with the computer 500.

[0044] The computer 500 is connected to a network 550 via a network interface 515. The network 550 may include physical, wireless, cellular, or different networks.

[0045] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings provided herein is not limited to cloud computing environments. Rather, embodiments of the invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0046] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0047] Its features include:

[0048] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capacity, e.g., server time and network storage, as needed automatically without the need for human interaction with the provider of the service.

[0049] Broad Network Access: The capability is available over the network and is accessed via standard mechanisms that facilitate use by heterogeneous thin- or thick-client platforms (e.g., cell phones, laptops, and PDAs).

[0050] Resource Pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated according to demand. There is a sense of location independence in that consumers generally have no control or knowledge of the exact location of the provided resources, but may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).

[0051] Rapid Elasticity: Capacity can be rapidly elastically provisioned, sometimes automatically, to rapidly scale out, and rapidly released to rapidly scale in. To the consumer, the capacity available for provisioning often appears unlimited and can be purchased in any quantity at any time.

[0052] Measured Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both providers and consumers of the services utilized.

[0053] The service models are as follows:

[0054] Software as a Service (SaaS): The capability offered to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0055] Platform as a Service (PaaS): The capability offered to the consumer is to deploy consumer-created or acquired applications written using programming languages ​​and tools supported by the provider onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but does control the deployed applications and sometimes the application hosting environment configuration.

[0056] Infrastructure as a Service (IaaS): The capability offered to the consumer is to supply processing, storage, network, and other underlying computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0057] The deployment models are as follows:

[0058] Private Cloud: The cloud infrastructure is operated solely for one organization. It can be managed by the organization or a third party and can exist on-premise or off-premise.

[0059] Community Cloud: The cloud infrastructure is shared by several organizations to support a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by the organization or a third party and may exist on-premise or off-premise.

[0060] Public cloud: The cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0061] Hybrid Cloud: The cloud infrastructure is a composition of multiple clouds (private, community, or public) that remain unique entities but are tied together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0062] A cloud computing environment is a service oriented approach with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0063] Referring now to FIG. 6, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or cell phone 54A, a desktop computer 54B, a laptop computer 54C, or an automobile computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. The nodes 10 may be physically or virtually grouped in one or more networks, such as a private, community, public, or hybrid cloud, or combinations thereof, as previously described (not shown). This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service without the cloud consumer having to maintain resources on the local computing device. It will be understood that the types of computing devices 54A-N illustrated in FIG. 6 are intended to be illustrative only, and that the computing node 10 and cloud computing environment 50 may communicate with any type of computerized device via any type of network and / or network addressable connection (e.g., using a web browser).

[0064] Referring now to Figure 7, a set of functional abstraction layers provided by cloud computing environment 50 (Figure 6) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 7 are intended to be merely illustrative, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0065] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: mainframe 61, RISC (reduced instruction set computing) architecture based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0066] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71, virtual storage 72, underlying networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0067] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources utilized to perform tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing of the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification of cloud consumers and tasks, and protection of data and other resources. User portal 83 provides consumer and system administrator access to the cloud computing environment. Service level management 84 provides cloud computing resource allocation and management such that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-arrangement and procurement of cloud computing resources where future requirements are anticipated according to SLAs.

[0068] The workload layer 90 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include: mapping and navigation 91, software development and lifecycle management 92, virtual classroom instructional delivery 93, data analytics results processing 94, transaction processing 95, and presentation attack detection 96.

[0069] The embodiments of the invention may be a system, a method, or a computer program product, or a combination thereof, at any possible level of technical detail integration. The computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to implement aspects of the invention.

[0070] A computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes: portable computer diskettes, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROMs or flash memories), static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or raised structures in grooves with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as being a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or even an electrical signal transmitted over a wire.

[0071] The computer readable program instructions described herein may be downloaded from the computer readable storage medium to the respective computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may comprise copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer readable program instructions from the network and forwards the computer readable program instructions for storage in the computer readable storage medium in the respective computing / processing device.

[0072] The computer readable program instructions for carrying out the operations of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages, e.g., the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry to implement aspects of the invention.

[0073] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0074] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via a processor of the computer or other programmable data processing apparatus, produce means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored includes an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0075] The computer readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause the computer, other programmable apparatus, or other device to perform a series of operational steps to produce a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0076] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or a subset of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0077] It is understood that the process software (e.g., any of the instructions stored in instructions 560 of FIG. 5 or any software configured to perform any portion of the method described with respect to FIG. 2 and / or FIG. 3A-3B and / or implement the functionality discussed in FIG. 1) may be deployed by manually loading it directly at the client, server and proxy computers via loading a storage medium such as a CD, DVD, etc., but the process software may also be automatically or semi-automatically deployed to computer systems by sending the process software to a central server or group of central servers. The process software is then downloaded to the client computers that will execute the process software. Alternatively, the process software is sent directly to the client systems via email. The process software is then isolated in a directory or loaded into the directory by executing a set of program instructions that isolates the process software into the directory. Another alternative is to send the process software directly to a directory on the client computer hard drive. When a proxy server is present, the process will select the proxy server code, determine on which computer to place the proxy server code, transmit the proxy server code, and then install the proxy server code on the proxy computer. The process software will be transmitted to the proxy server, and it will then be stored on the proxy server.

[0078] Embodiments of the invention may also be delivered as part of a service agreement with a client company, a non-profit organization, a government agency, an internal organizational structure, etc. These embodiments may include configuring an implementing computer system and deploying software, hardware, and web services that implement some or all of the methods described herein. These embodiments may also include analyzing client behavior, making recommendations in response to the analysis, building a system that implements a subset of the recommendations, integrating the system into existing processes and infrastructure, metering usage of the system, allocating costs to users of the system, and billing, invoicing (e.g., generating invoices), or otherwise receiving payment for use of the system.

[0079] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will be further understood that the terms "comprise" and / or "comprising", as used herein, specify the presence of stated features, integers, steps, operations, elements, or components or combinations thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof. In the preceding detailed description of exemplary embodiments of the various embodiments, reference has been made to the accompanying drawings, in which like numerals represent like elements, which show by way of illustration specific exemplary embodiments that form a part hereof and in which the various embodiments may be practiced. These embodiments have been described in sufficient detail to enable one skilled in the art to practice the embodiments, but other embodiments may be used and logical, mechanical, electrical and other changes may be made without departing from the scope of the various embodiments. In the description above, numerous specific details have been set forth in order to provide a thorough understanding of the various embodiments. However, various embodiments may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail so as not to obscure the embodiments.

[0080] Different instances of the word "embodiment" as used herein do not necessarily refer to the same embodiment, but may. Any data and data structures shown and described herein are merely examples, and in other embodiments, different amounts of data, types of data, fields, numbers and types of fields, field names, numbers and types of rows, records, entries, or organization of data may be used. Furthermore, any data may be combined with logic such that separate data structures may not be required. Thus, the foregoing detailed description should not be taken in a limiting sense.

[0081] The description of various embodiments of the present disclosure is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Numerous modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements to the technology found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0082] While the present disclosure has been described with respect to specific embodiments, it is anticipated that variations and modifications thereof will become apparent to those skilled in the art, and it is therefore intended that the following claims be interpreted as including all such variations and modifications that are within the true scope of the present disclosure.

[0083] Any advantages discussed in this disclosure are example advantages, and there may be embodiments of the disclosure that achieve all, some, or none of the advantages discussed while remaining within the scope of the disclosure.

[0084] A non-limiting list of examples is provided below to illustrate some aspects of the disclosure. Example 1 is a computer-implemented method. The method includes emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing a video conference, collecting video data of a user engaged in the video conference by a camera communicatively coupled to the computer and overlapping with the emitting of the predetermined frequency pattern, determining that the video data is inconsistent with an expected response to the predetermined frequency pattern, and generating an indication that the user engaged in the video conference is conducting a presentation attack.

[0085] Example 2 includes the features of example 1. In this example, determining that the video data is inconsistent with an expected response to the predetermined frequency pattern further includes: magnifying a portion of the video data and detecting a lack of oscillatory motion in the magnified portion of the video data.

[0086] Example 3 includes the features of any one of examples 1-2. In this example, the video data is high definition video data.

[0087] Example 4 includes the features of any one of examples 1 to 3. In this example, the predetermined frequency pattern is within a frequency range that causes vibrations that can be captured by the frame rate of the camera.

[0088] Example 5 includes the features of any one of examples 1 to 4. In this example, determining that the video data is inconsistent with an expected response to the predetermined frequency pattern further includes: monitoring the video data of the user's movements and determining that the user's movements are not expected movements.

[0089] Example 6 includes the features of any one of examples 1 to 5. In this example, the at least one speaker communicatively coupled to the computer is a headset headphone.

[0090] Example 7 includes the features of any one of examples 1 to 6. In this example, the predetermined frequency pattern is a directional audio cue.

[0091] Example 8 includes the features of any one of examples 1 to 7. In this example, determining that the video data is inconsistent with an expected response to the predetermined frequency pattern further includes: monitoring the video data of a facial expression of the user and determining that the facial expression of the user is not an expected facial expression.

[0092] Example 9 includes the features of any one of examples 1 to 8. In this example, the predetermined frequency pattern is an audible cue configured to elicit an emotional response from the user.

[0093] Example 10 includes the features of any one of examples 1 to 9. In this example, the method is performed by one or more computers in accordance with presentation attack detection software downloaded to the one or more computers from a remote data processing system.

[0094] Example 11 includes the features of any one of examples 1 to 10. In this example, the method further includes: measuring usage of the presentation attack detection software; and generating an invoice based on the usage measurement.

[0095] Example 12 is a system including one or more computer-readable storage media storing program instructions, including or excluding optional features, and one or more processors configured to perform a method according to any one of Examples 1 to 11 in response to executing the program instructions.

[0096] Example 13 is a computer program product, including or excluding optional features, that includes one or more computer-readable storage media and program instructions collectively stored on the one or more computer-readable storage media, the program instructions including instructions configured to cause one or more processors to perform a method according to any one of Examples 1 to 11.

Claims

1. emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing the video conference; collecting video data of users engaged in said video conference by a camera communicatively coupled to said computer and overlapping with said emitting said predetermined frequency pattern; determining that the video data is inconsistent with an expected response to the predetermined frequency pattern; generating an indication that the user engaged in the video conference is conducting a presentation attack; 4. A computer-implemented method comprising:

2. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: enlarging a portion of the video data; detecting a lack of vibratory motion in the enlarged portion of the video data; The computer-implemented method of claim 1 , comprising:

3. The computer-implemented method of claim 1 , wherein the video data is high definition video data.

4. The computer-implemented method of claim 1 , wherein the predetermined frequency pattern is within a frequency range that causes vibrations that can be captured by a frame rate of the camera.

5. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data of the user's movement; determining that the exercise of the user is not an expected exercise; The computer-implemented method of claim 1 , comprising:

6. The computer-implemented method of claim 5 , wherein the at least one speaker communicatively coupled to the computer is a headset headphone.

7. The computer-implemented method of claim 5 , wherein the predetermined frequency pattern is a directional audio cue.

8. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data for facial expressions of the user; determining that the facial expression of the user is not an expected facial expression; The computer-implemented method of claim 1 , comprising:

9. 10. The computer-implemented method of claim 8, wherein the predetermined frequency pattern is an audible cue configured to elicit an emotional response from the user.

10. 10. The computer-implemented method of claim 1, wherein the method is performed by the one or more computers in accordance with presentation attack detection software downloaded to the one or more computers from a remote data processing system.

11. The method further comprises: Measuring usage of the presentation attack detection software; and generating an invoice based on said usage measurements; The computer-implemented method of claim 10, comprising:

12. one or more computer readable storage media storing program instructions; emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing the video conference; collecting video data of users engaged in said video conference by a camera communicatively coupled to said computer and overlapping with said emitting said predetermined frequency pattern; determining that the video data is inconsistent with an expected response to the predetermined frequency pattern; generating an indication that the user engaged in the video conference is conducting a presentation attack; one or more processors configured to perform, in response to executing the program instructions, a method comprising: A system comprising:

13. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: enlarging a portion of the video data; detecting a lack of vibratory motion in the enlarged portion of the video data; The system of claim 12 , comprising:

14. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data of the user's movement; determining that the exercise of the user is not an expected exercise; The system of claim 12 , comprising:

15. The system of claim 14 , wherein the predetermined frequency pattern is a directional audio cue.

16. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data for facial expressions of the user; determining that the facial expression of the user is not an expected facial expression; The system of claim 12 , comprising:

17. 1. A computer program product comprising one or more computer readable storage media and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising: emitting a predetermined frequency pattern using at least one speaker communicatively coupled to a computer implementing the video conference; collecting video data of users engaged in said video conference by a camera communicatively coupled to said computer and overlapping with said emitting said predetermined frequency pattern; determining that the video data is inconsistent with an expected response to the predetermined frequency pattern; generating an indication that the user engaged in the video conference is conducting a presentation attack; 23. A computer program product comprising instructions configured to cause one or more processors to perform a method comprising:

18. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: enlarging a portion of the video data; detecting a lack of vibratory motion in the enlarged portion of the video data; 20. The computer program product of claim 17, comprising:

19. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data of the user's movement; determining that the exercise of the user is not an expected exercise; 20. The computer program product of claim 17, comprising:

20. Determining that the video data is inconsistent with the expected response to the predetermined frequency pattern further comprises: monitoring the video data for facial expressions of the user; determining that the facial expression of the user is not an expected facial expression; 20. The computer program product of claim 17, comprising:

Images