Visual and audio output authentication in virtual avatar systems

Abstract

The invention relates to visual and audio output authentication in virtual avatar systems. A method is described for combating digital impersonations and unauthorized digital or audio usage resulting from the rise of deep forgery techniques. The method employs biometric authentication to ensure real-time user authenticity during avatar or voice recording. Unique watermarks are generated at predetermined intervals and embedded into the audio and / or video output. The device integrity may be additionally validated by device attestation, thereby validating hardware and firmware integrity of the visual and audio systems. The device attestation may generate a device authentication key, which may be combined with a user authentication key based on biometric authentication to generate a combined watermark. Thus, the real-time verification tool allows authentication of watermarks, thereby ensuring continuous content authenticity.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to digital visual and audio output authentication, and more specifically, to techniques for generating cryptographic watermarks based on user biometric authentication to be incorporated into video and / or audio content to combat the rise of deepfakes. Background Art

[0002] Artificial intelligence and machine learning technologies have led to an increase in the misuse of digital audiovisual representations through deepfakes ("deepfakes"). Deepfakes are synthetic media that have been digitally manipulated to convincingly alter or replace an individual's image. Deepfakes leverage machine learning and AI technologies to manipulate or generate audiovisual content, such as videos, images, or audio, that can more easily deceive viewers or listeners. Deepfakes pose a serious threat to personal privacy and general security, and may lead to an increase in disinformation, fraud, identity theft, and other forms of online malfeasance unless preventative technologies are implemented.

[0003] Traditional methods for identifying or authenticating visual and audio content include digital watermarking. Digital watermarking, a technique commonly used to identify copyright ownership, embeds information into digital content. This embedded information is also used to identify or authenticate content and protect it from unauthorized modification. However, detecting unauthorized modifications relies on identifying whether the digital watermark has been altered or removed, and attackers may find ways to preserve the digital watermark even when making unauthorized modifications.

[0004] As previously mentioned, the field needs more effective techniques for authenticating digital visual and / or audio representations to combat the rise of deepfakes. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A block diagram illustrating a computing system configured to implement one or more aspects of at least one embodiment is shown;

[0006] Figure 2 shows a flow chart of a method for watermarking a digital visual and / or audio representation based on continuous user biometric authentication during recording, in accordance with at least one embodiment;

[0007] Figure 3 A process flow diagram for device certification according to at least one embodiment is shown;

[0008] Figure 4 A flow chart illustrating a method for watermarking digital visual and / or audio representations based on a combination of device proof and continuous user biometric authentication in accordance with at least one embodiment; and

[0009] Figure 5 A process flow diagram for watermark verification according to at least one embodiment is shown. DETAILED DESCRIPTION

[0010] Deepfakes are synthetic media in which machine learning and artificial intelligence technologies are used to digitally manipulate original media to alter or replace a person's image, often with remarkable fidelity. The increasing misuse of deepfakes poses threats to personal privacy, organizational security, disinformation, fraud, identity theft, and / or other forms of online illicit activity. For example, using machine learning and artificial intelligence technologies, virtual avatars resembling public figures can be generated and recorded making statements or endorsing products that the public figures did not make.

[0011] To combat the misuse of deepfakes and ensure the authenticity of media sources, the disclosed technology dynamically manages watermark embedding based on real-time biometric authentication. Users (or media creators) can be continuously authenticated using biometrics during avatar or voice rendering to ensure that the user's presence is continuously verified during the recording process. Authentication failures may result in the automatic deletion, expiration, or otherwise omission of the watermark from the content frame. In addition, device attestation can be performed at the beginning of avatar or voice rendering (or at intervals during avatar or voice rendering) to verify the integrity of the camera and audio system, thereby ensuring that the recording is done on a trustworthy device.

[0012] More specifically, at an instance during the recording of video or audio content, one or more biometric authentication operations are performed to authenticate the user. The one or more operations may be performed every frame, every n frames, every frame or n frames of the user's speech, or every spoken word or trigger word (e.g., a predefined meaningful word that may change the context or meaning of a statement spoken by the user). In response to a successful biometric authentication, a user authentication key is generated, and a cryptographic watermark representing the user's continued presence during the recording of the instance is generated based on the user authentication key. Thus, the cryptographic watermark is embedded in the frames of the content for the instance.

[0013] Furthermore, device attestation can be performed to verify the integrity of the camera and / or audio system of the computing device recording the video or audio content. In response to verifying the integrity of the camera and / or audio system, a device authentication key is generated, which can then be combined with the user authentication key to generate a cryptographic watermark. Thus, the combination of regularly performed user authentication, dynamic watermark management, and device integrity verification provides a strong defense against deepfake techniques and / or other unauthorized manipulation of digital content.

[0014] System Overview

[0015] Figure 1is a block diagram illustrating a computing system 100 configured to implement one or more aspects of at least one embodiment. In at least one embodiment, computing system 100 may include any type of computing device, including but not limited to a server machine, a server platform, a desktop computer, a laptop computer, a handheld / mobile device, a digital kiosk, an in-vehicle infotainment system, and / or a wearable device. In at least one embodiment, computing system 100 is a server machine operating in a data center or cloud computing environment that provides scalable computing resources as a service over a network.

[0016] In various embodiments, computing system 100 includes, but is not limited to, one or more processors 102 and one or more memories 104, which are coupled to parallel processing subsystem 112 via memory bridge 105 and communication path 113. Memory bridge 105 is also coupled to I / O (input / output) bridge 107 via communication path 106, which in turn is coupled to switch 116.

[0017] In one embodiment, I / O bridge 107 is configured to receive user input information from optional input devices 108 (e.g., but not limited to, a keyboard, mouse, touchscreen, sensor data analysis (e.g., evaluating gestures, voice, or other information about one or more uses within the field of view or sensor field of one or more sensors), VR / MR / AR headsets, gesture recognition systems, steering wheels, mechanical, digital, or touch-sensitive buttons or input components, and / or microphones) and forward the input information to processor 102 for processing. In at least one embodiment, computing system 100 may be a server machine in a cloud computing environment. In such an embodiment, computing system 100 may omit input device 108 and receive equivalent input information as commands (e.g., in response to one or more inputs from a remote computing device) and / or messages transmitted over a network and received via network adapter 118. In at least one embodiment, switch 116 is configured to provide connectivity between I / O bridge 107 and other components of computing system 100 (e.g., network adapter 118 and various add-in cards 120 and 121).

[0018] In at least one embodiment, I / O bridge 107 is coupled to system disk 114, which can be configured to store content and applications as well as data for use by processor 102 and parallel processing subsystem 112. In one embodiment, system disk 114 provides non-volatile storage for applications and data and can include a fixed or removable hard drive, a flash memory device, and a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc ROM), Blu-ray, HD-DVD (High Definition DVD), or other magnetic, optical, or solid-state storage device. In various embodiments, other components (e.g., a universal serial bus or other port connection, an optical disc drive, a digital versatile disc drive, a film recording device, etc.) can also be connected to I / O bridge 107.

[0019] In various embodiments, memory bridge 105 may be a northbridge chip, and I / O bridge 107 may be a southbridge chip. Furthermore, communication paths 106 and 113, as well as other communication paths within computing system 100, may be implemented using any technically suitable protocol, including but not limited to AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art.

[0020] In at least one embodiment, parallel processing subsystem 112 includes a graphics subsystem that delivers pixels to optional display device 110, which can be any conventional cathode ray tube, liquid crystal display, light emitting diode display, etc. In such an embodiment, parallel processing subsystem 112 can include circuitry optimized for graphics and video processing, including, for example, video output circuitry. Such circuitry can be incorporated into one or more parallel processing units (PPUs) (also referred to herein as parallel processors) included within parallel processing subsystem 112.

[0021] In at least one embodiment, parallel processing subsystem 112 includes circuitry optimized for general-purpose and / or computational processing (e.g., it is optimized). Similarly, such circuitry may be incorporated into one or more PPUs included within parallel processing subsystem 112, which are configured to perform such general-purpose and / or computational operations. In yet other embodiments, one or more PPUs included within parallel processing subsystem 112 may be configured to perform graphics processing, general-purpose processing, and / or computational processing operations. Memory 104 includes at least one device driver configured to manage processing operations of one or more PPUs within parallel processing subsystem 112. In addition, memory 104 includes an annotation engine 122, a dialog engine 124, and an execution engine 126, which may be executed by the processor and / or parallel processing subsystem 112.

[0022] In various embodiments, the parallel processing subsystem 112 may be coupled with Figure 1 For example, parallel processing subsystem 112 may be integrated with processor 102 and other connected circuitry on a single chip to form a system on a chip (SoC).

[0023] The processor 102 may include any suitable processor implemented as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an artificial intelligence (AI) accelerator, a deep learning accelerator (DLA), a parallel processing unit (PPU), a data processing unit (DPU), a vector or vision processing unit (VPU), a programmable vision accelerator (PVA) (which may include one or more VPUs and / or a direct memory access (DMA) system), any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate with a GPU. In general, the processor 102 may include any technically feasible hardware unit that is capable of processing data and / or executing software applications. In addition, in the context of the present disclosure, the computing elements shown in the computing system 100 may correspond to physical computing systems (e.g., systems in a data center or machine) and / or may correspond to virtual computing instances executed in a computing cloud.

[0024] In at least one embodiment, processor 102 issues commands that control the operation of the PPUs. In at least one embodiment, communication path 113 is a PCI Express link, where each PPU is allocated a dedicated lane. Other communication paths may also be used. The PPUs advantageously implement a highly parallel processing architecture, and the PPUs can be equipped with any number of local parallel processing memories (PP memories).

[0025] It should be understood that the systems shown herein are illustrative and are subject to variation and modification. The connection topology, including the number and arrangement of bridges, the number of processors 102, and the number of parallel processing subsystems 112, can be modified as desired. For example, in at least one embodiment, memory 104 can be directly connected to processor 102 rather than through memory bridge 105, and other devices can communicate with memory 104 via memory bridge 105 and processor 102. In other embodiments, parallel processing subsystem 112 can be connected to I / O bridge 107 or directly to processor 102 rather than to memory bridge 105. In still other embodiments, I / O bridge 107 and memory bridge 105 can be integrated into a single chip rather than existing as one or more discrete devices. In some embodiments, Figure 1One or more of the components shown in FIG may not exist. For example, switch 116 may be eliminated, and network adapter 118 and add-in cards 120, 121 may be connected directly to I / O bridge 107. Finally, in some embodiments, Figure 1 One or more components shown in FIG1 may be implemented as virtualized resources in a virtual computing environment (e.g., a cloud computing environment). Specifically, in at least one embodiment, parallel processing subsystem 112 may be implemented as a virtualized parallel processing subsystem. For example, parallel processing subsystem 112 may be implemented as a virtual graphics processing unit (vGPU) that renders graphics on a virtual machine (VM) that executes on a server machine whose GPU and other physical resources are shared among one or more VMs.

[0026] In some embodiments, each of the user authentication engine 122, the device authentication engine 124, and the watermark generation engine 126 includes functionality to combat the abuse of deep fakes and ensure the authenticity of media sources by dynamically managing cryptographic watermark embedding based on real-time biometric authentication. The user authentication engine 122 performs one or more biometric authentication operations to authenticate the user using the user's unique biometrics to verify their identity. Non-limiting examples of operations that can be performed to biometrically authenticate the user include voice recognition, facial recognition, iris scanning, fingerprint scanning, and the like. In addition, in some cases, two or more biometric authentication operations (such as voice and facial recognition, etc.) may be required to authenticate the user. Therefore, after the biometric authentication is successful, the user authentication engine 122 generates a user authentication key for the instance.

[0027] The device authentication engine 124 performs device attestation to verify the integrity of at least one of the camera or audio system of the user's computing device. In one embodiment, the Trusted Computing Group (TCG) Device Identifier Composition Engine (DICE) is used to perform device attestation to verify the integrity of a set of hardware components of the computing device. In addition, in one or more embodiments, DICE is extended beyond hardware to verify one or more software components, including a recording application that records video and / or audio content. As described in further detail herein, device attestation verifies each of a set of hardware components and a software component associated with the recording application separately. After successfully verifying a component, the device authentication engine 124 generates a verification certificate to be added to a verification chain. Each verification certificate corresponds to a component in a set of hardware and software components, and each component corresponds to a link in the verification chain, where each link adds a unique secret to the verification chain. Therefore, the device authentication engine 124 uses the verification chain to generate a device authentication key, or otherwise unlocks the device authentication key after successfully verifying the verification chain.

[0028] In one or more embodiments, the watermark generation engine 122 generates a cryptographic watermark by cryptographically binding the user authentication key, the device authentication key, and the timestamp of the recorded frame of the video or audio content corresponding to the instance. The watermark generation engine 122 then embeds the cryptographic watermark into the frame of the content for the instance.

[0029] Watermarked media based on continuous user biometric authentication during recording

[0030] Figure 2 A flow chart of a method 200 for generating a cryptographic watermark based on user biometric authentication for incorporation into video and / or audio content is shown in accordance with at least one embodiment. Alternative embodiments may include Figure 2 There are more, fewer or different steps shown in the figure, and these steps can be followed in the same way as Figure 2 The execution order is different from the order shown in .

[0031] like Figure 2 As shown, method 200 begins at operation 202, where a user initiates a video or audio recording session, such as an avatar and / or live stream recording. Accordingly, at each of a plurality of instances 204 of the video or audio content recording, the user authentication engine 122 performs one or more biometric authentication operations at operation 206 to authenticate the user. The term "instance" is used to refer to the time at which a biometric operation is performed, such as the number of frames or the time between authentication operations. In one embodiment, a biometric authentication operation is performed at each frame of the video or audio content to check for continued presence during the recording. In this example, instance 204 would occur at each frame. The biometric authentication operation may also be performed at each set of frames (e.g., every 2 frames, every 5 frames, or every n frames) or at time intervals (e.g., every second or every n seconds).

[0032] Furthermore, in one embodiment, the instance at which biometric authentication is performed is determined based on whether the user is speaking during the recording. Therefore, in this embodiment, the user authentication engine 122 analyzes the video or audio content and identifies when the user is speaking. In response to identifying that the user is speaking, the user authentication engine 122 causes a biometric operation to be performed for the duration of the user's speech. As described above, the biometric authentication operation can be performed for each frame, a group of frames, or at intervals while the user is speaking.

[0033] In another embodiment, the instance at which biometric authentication is performed is determined based on whether the user uttered one or more predetermined trigger words or phrases during the recording. Thus, in this embodiment, the user authentication engine 122 analyzes the recorded audio to identify the trigger words or phrases uttered by the user. In one embodiment, the user authentication engine 122 stores or accesses a data store of predefined words, groups of words, and / or phrases that are believed to have an impact and, if changed, would likely have an impact on the meaning of the statement that is greater than a threshold impact. Thus, in response to identifying that the user uttered a trigger word or phrase, the user authentication engine 122 causes the biometric authentication operation to be performed while the trigger word or phrase is being uttered.

[0034] In operation 208, the user authentication engine 122 determines whether the user has successfully authenticated. If the user has successfully authenticated, then in operation 212, the user authentication engine generates a user authentication key for the instance. Conversely, in response to unsuccessful user authentication in operation 208, in operation 210, the user authentication engine 122 does not generate a user authentication key, and the process ends without a watermark being generated for the instance of video or audio content. Thus, an authentication failure results in the automatic removal or deactivation of the watermark. In one embodiment, an authentication failure causes the video or audio recording session to end.

[0035] In operation 214, the watermark generation engine 126 generates a cryptographic watermark based on the user authentication key and the timestamp for the instance of the video or audio content. Thus, real-time biometric authentication ensures that the user interacting with or recording the content is the authorized individual, thereby preventing unauthorized users from misusing the digital avatar or live stream.

[0036] In operation 216, the watermark generation engine 126 embeds the cryptographic watermark into the recording for the instance. Thus, the watermark is embedded or removed in real time based on the ongoing authentication state. This dynamic approach ensures that the watermarked content is generated by an authenticated user, thereby effectively preventing attempts to misuse digital representations.

[0037] In various embodiments, watermarks can be either explicit or implicit, and can be applied to both the video and audio of an audiovisual file to ensure simultaneous authenticity verification. A covert watermark is a hidden, discreet pattern or digital mark that, when embedded in content, is not easily detectable to the viewer. Unlike explicit watermarks, which are intentionally visible, covert watermarks are designed to be subtle and imperceptible, for example, to the naked eye. These watermarks are typically detected using specialized software or techniques for authentication and ownership confirmation. Examples of covert watermarks include steganographic watermarks, which are hidden within the content, for example, by altering insignificant bits of pixel values (such as color changes or refresh rates that are imperceptible to the human eye). Additionally, audio watermarks can encode sounds that sound like background noise or are imperceptible to the human ear, but can be detected using software verification techniques. Steganographic embedding of watermarks provides a discreet verification method that preserves the aesthetic integrity of the content while ensuring its authenticity. Various other watermarking techniques may also be used within the scope of the present disclosure. Furthermore, while some previous solutions may watermark audio or video separately, the described approach unifies the watermarking method for both audio and video output, ensuring synchronized authenticity verification across different media types.

[0038] Equipment certification

[0039] Figure 3 A process flow diagram 300 for device attestation of a computing device is shown in accordance with at least one embodiment. Device attestation is the process of ensuring the integrity and authenticity of a computing device. In various embodiments, the process flow diagram 300 includes, but is not limited to, a BIOS 302, a SoC 304, motherboard components 306, a webcam 308, an audio codec 310, a firmware driver 312, an operating system (OS) 314, an OS driver, a recording application 318, a unique device secret (UDS) 320, a DICE 322, and a trusted platform module (TPM) 324. Alternative embodiments may include, but are not limited to, a BIOS 302, a SoC 304, a motherboard component 306, a webcam 308, an audio codec 310, a firmware driver 312, an operating system (OS) 314, an OS driver, a recording application 318, a unique device secret (UDS) 320, a DICE 322, and a trusted platform module (TPM) 324. Figure 3 There are more, fewer or different steps shown in the , and these steps can be followed in the same Figure 3 The order shown in the following example is different from the order in which they are executed.

[0040] When a user's computing device is powered on, BIOS 302 initializes 350 and is loaded into memory. BIOS 302 performs basic hardware checks and configures 352 basic components, such as SoC 304. SoC 304 configures 352 motherboard assembly 306, which connects 354 to webcam 308 and 356 to audio codec 310. Additionally, firmware drivers 312 and TPM 324 are initialized during BIOS 302. TPM is a hardware-based security feature that provides a secure environment for authentication. Firmware drivers 312 load 358 OS 314, which loads 360 OS drivers 316 that enable 362 recording application 318.

[0041] UDS 320 is a piece of confidential information or cryptographic key unique to a particular computing device. DICE 322 has exclusive access to UDS 320 and relies on this unique device secret to create a secret for each link in the DICE 322 chain. For example, each link can use a cryptographic one-way function to calculate the secret of the next link in the chain. During device attestation, BIOS 302 measures the integrity 366 of SoC 304, and this cryptographic one-way function calculates the secret for this link in the chain. Next, SoC 304 measures the integrity 368 of motherboard assembly 306, and a cryptographic one-way function calculates another secret for this next link and adds it to the chain. Thus, motherboard assembly 306 will then measure the integrity 370 of webcam 308; webcam 308 will then measure the integrity 372 of audio codec 310; audio codec 310 will then measure the integrity 374 of firmware driver 312; firmware driver 312 will measure the integrity 376 of OS 314; OS 314 will measure the integrity 378 of OS driver 316 and the integrity 380 of recording application 318. Each of these integrity measurements creates a link in the DICE 322 chain, and each of these links adds a unique secret to the chain. Therefore, if any one of these links in the chain is modified, the entire chain is modified. This modification constitutes evidence that someone has at least attempted to alter the device or circumvent its security.

[0042] Finally, in response to the entire DICE chain being verified, DICE 322 verifies 382 the integrity of the recording platform and communicates the verification to TPM 324. Thus, TPM 324 generates a device authentication key 384 in response to receiving the verification from DICE 322—or causes the device authentication key to be unlocked.

[0043] Watermarked media based on device attestation and continuous user biometric authentication during recording

[0044] Figure 4 A flow chart illustrating a method 400 for generating a cryptographic watermark for incorporation into video and / or audio content based on user biometric authentication and device attestation according to at least one embodiment is shown. Alternative embodiments may include Figure 4 There are more, fewer or different steps shown in the , and these steps can be followed in the same Figure 4 Method 400 begins at operation 402, where a user initiates a video or audio recording session, such as an avatar and / or live stream recording.

[0045] In operation 404, the device authentication engine 124 performs device attestation to verify the integrity of the content recording system of the user's computing device. In one embodiment, device attestation is performed using TCG DICE to verify the integrity of a set of hardware components of the computing device. In addition, in one or more embodiments, DICE is extended beyond hardware to verify the recording application that is recording video and / or audio content. In this example, device attestation is shown to be performed after starting the avatar and / or live stream recording. However, in various embodiments, device attestation can be performed at device startup, each time the user starts recording, or at various times during recording, such as every n seconds or every n minutes, every n frames.

[0046] In operation 406, the device authentication engine 124 determines whether the recording system (e.g., the user's computing device) is authenticated. If the user's computing device is authenticated, the device authentication engine 124 generates a device authentication key at operation 410 in response to successfully authenticating the user's computing device, as described herein. Conversely, in response to unsuccessful device authentication in decision operation 406, the device authentication engine 122 does not authenticate the user's computing device at operation 408, and therefore does not generate or unlock a device authentication key. As a result, the process ends, a watermark is not generated for the instance of the video or audio content, and in some embodiments, the video or audio recording session ends. Authentication failure results in automatic deletion or deactivation of the watermark.

[0047] Thus, at each of the plurality of instances 412 of the video or audio content recording, the user authentication engine 122 performs 412 one or more biometric authentication operations to authenticate the user. As described herein, the biometric authentication operations may be performed for each frame, each group of frames, or each second or every few seconds of the video or audio content to ensure continuous usage during the recording.

[0048] In decision operation 414, the user authentication engine 122 determines whether the user has been authenticated. If the user has been authenticated, the user authentication engine 122 generates a user authentication key for the instance in response to successfully authenticating the user in operation 418. Conversely, in response to unsuccessful user authentication in operation 414, the user authentication engine 122 does not generate a user authentication key in operation 416, and the process ends without a watermark being generated for the instance of video or audio content. Therefore, a failure at this stage of the authentication process may also result in the automatic deletion or invalidation of the watermark, even if the device authentication engine 124 has successfully authenticated the user's computing device in operation 406.

[0049] In operation 420, the watermark generation engine 126 generates a cryptographic watermark based on the device authentication key, the user authentication key, and the timestamp of the instance of the video or audio content. Thus, device attestation ensures that the hardware and recording application of the user's device are intact and have not been compromised, and the real-time biometric authentication performed each time ensures that the user interacting with or recording the content is an authorized individual, thereby preventing unauthorized users from misusing the digital avatar or live stream.

[0050] In operation 422, the watermark generation engine 126 embeds a cryptographic watermark into the recording of the instance. In one embodiment, the cryptographic watermark includes one or more of a timestamp, a frame number, a device authentication key, and a user authentication key. Thus, in one embodiment, after successful device authentication, the watermark is embedded or removed in real time based on the ongoing authentication state. This dynamic approach ensures that the watermark content is generated by an authenticated user, thereby effectively preventing attempts to misuse the digital representation. Compared to traditional watermarking solutions that do not bundle either device integrity proof or real-time user biometric authentication into the watermark, the cryptographic binding of the DICE proof together with the cryptographic watermark to form a composite watermark based on both device authentication and user biometric authentication is a significant feature.

[0051] In one embodiment, device attestation includes verifying the integrity of a set of hardware components and a set of software components of a computing device, such as Figure 3 As discussed, a set of software components includes an operating system and a recording application for capturing video or audio content, while a set of hardware components includes a camera or audio system of a computing device. A verification chain of verification certificates is generated, where each verification certificate corresponds to a component in the set of hardware components and the set of software components, and each component corresponds to a link in the verification chain. Thus, each link adds a unique secret to the verification chain, and a device authentication key is generated in response to verifying that the chain is complete.

[0052] Watermark Verification

[0053] Figure 5A process flow chart 500 for watermark verification according to at least one embodiment is shown. In various embodiments, the flow chart 500 includes, but is not limited to, a user A device, a user B device, a device information engine 502, a biometric verification engine 504, a signature algorithm 506, and a key exchange algorithm 508. Alternative embodiments may include, but are not limited to, Figure 5 There are more, fewer or different steps shown in the , and these steps can be followed in the same Figure 5 The order shown in the following example is different from the order in which they are executed.

[0054] The device information engine 502 verifies whether the received device information is correct and matches the expected device information. The biometric verification engine 504 verifies whether the received biometric data matches the expected biometric data. The signature algorithm 506 is used to sign the data and calculate its hash value. The message is then sent along with the hash value and the name or ID of the signature algorithm. The recipient can then calculate and compare the hash value to ensure that the message has not been altered during transmission. In addition, the signature algorithm 506 can be extended to a post-quantum cryptography (PQC) signature algorithm, such as the crystal dilithium algorithm, to make the process quantum-resistant. The key exchange algorithm 508 is a method for creating and exchanging secret keys between two or more parties. In addition, the key exchange algorithm 508 can also be extended to a post-quantum cryptography (PQC) key exchange algorithm, such as Kyber or NTRU, to make the process quantum-resistant.

[0055] Therefore, User A's device generates a key pair using a signature algorithm 502 and signs the message or key using the private key of the user operating User A's device. Furthermore, User A's device sends the biometric data and device information signed using the private key. User B's device receives the public key, biometric data, and device information. User B's device then verifies the signature, biometric data, and device information using the public key of the user operating User A's device. In one embodiment, User B's device uses a PQC algorithm—PQC Signature Algorithm 506 and PQC Key Exchange Algorithm 508—to verify the signature, biometric data, and device information using the public key of the user operating User A's device. User B's device then generates a key pair for secure communication.

[0056] Other variations are within the spirit and scope of the present disclosure. Therefore, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrative embodiments thereof have been shown in the drawings and described in detail herein. However, it should be understood that there is no intention to limit the disclosure to the particular forms disclosed, but on the contrary, the intent is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of the disclosure as defined in the appended claims.

[0057] In the context of describing the disclosed embodiments (especially in the context of the following claims), the use of the terms "a", "an", and "the", and similar referents, should be interpreted to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by the context, and are not intended to define the terms. The terms "comprising", "having", "including", and "containing" should be interpreted as open-ended terms (meaning "including, but not limited to"), unless otherwise indicated. "Connected", when unmodified and referring to a physical connection, should be interpreted as partially or completely contained within, attached to, or connected together, even if there is something in between. The recitation of ranges of values herein is intended merely as a shorthand method of individually referring to each individual value that falls within the range, unless otherwise indicated herein, and each individual value is incorporated into the specification as if it were individually recited herein. In at least one embodiment, the use of the term "set" (e.g., "set of items") or "subset" should be interpreted as a non-empty set containing one or more members, unless otherwise indicated or contradicted by the context. Furthermore, unless otherwise stated or contradicted by context, the term "subset" of a corresponding set does not necessarily mean a true subset of the corresponding set, but a subset and a corresponding set may be equivalent.

[0058] Unless expressly stated otherwise or clearly contradicted by context, connective language, such as phrases of the form "at least one of A, B, and C" or "at least one of A, B, and C," should be understood contextually to generally indicate that an item, term, or the like can be A or B or C, or any non-empty subset of the set of A, B, and C. For example, in the illustrative example of a set having three members, the connective phrases "at least one of A, B, and C" and "at least one of A, B, and C" refer to any one of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such connective language is generally not intended to imply that certain embodiments require that at least one of A, at least one of B, and at least one of C, respectively, be present. Furthermore, unless expressly stated otherwise or contradicted by context, the term "plurality" denotes plurality (e.g., "a plurality of items" means a plurality of items). In at least one embodiment, the number of items in the plurality is at least two, but may be greater when expressly or contextually indicated. Further, unless stated otherwise or clear from context, the phrase "based on" means "based at least in part on" rather than "based solely on."

[0059] Unless otherwise specified herein or the context clearly contradicts, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as the processes described herein (or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed on one or more processors, by hardware or a combination thereof. In at least one embodiment, the code is stored on a computer-readable storage medium, e.g., in the form of a computer program that includes multiple instructions that can be executed by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transient computer-readable storage medium that does not include transient signals (e.g., propagating transient electrical or electromagnetic transmissions), but includes non-transient data storage circuits (e.g., buffers, caches, and queues) within a transceiver for transient signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-transitory computer-readable storage media having executable instructions stored thereon (or other memory for storing executable instructions) that, when executed by one or more processors of a computer system (i.e., as a result of being executed), cause the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media includes a plurality of non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media of the plurality of non-transitory computer-readable storage media lack all code, while the plurality of non-transitory computer-readable storage media collectively store all code. In at least one embodiment, the executable instructions are executed in such a manner that different instructions are executed by different processors—for example, the non-transitory computer-readable storage medium stores instructions, and a main central processing unit ("CPU") executes some instructions, while a graphics processing unit ("GPU") executes other instructions. In at least one embodiment, different components of the computer system have separate processors, and the different processors execute different subsets of instructions.

[0060] In at least one embodiment, the ALU is a set of combinational logic circuits that take one or more inputs to produce a result. In at least one embodiment, the ALU is used by a processor to implement mathematical operations, such as addition, subtraction, or multiplication. In at least one embodiment, the ALU is used to implement logical operations, such as logical AND / OR or XOR. In at least one embodiment, the ALU is stateless and is made of physical switch components (e.g., semiconductor transistors arranged to form logic gates). In at least one embodiment, the ALU can operate internally as a stateful logic circuit with an associated clock. In at least one embodiment, the ALU can be constructed as an asynchronous logic circuit whose internal state is not maintained in an associated register set. In at least one embodiment, the ALU is used by a processor to combine operands stored in one or more registers of the processor and produce an output that can be stored by the processor in another register or memory location.

[0061] In at least one embodiment, as a result of processing an instruction retrieved by the processor, the processor presents one or more inputs or operands to the arithmetic logic unit, causing the arithmetic logic unit to generate a result based at least in part on the instruction code provided to the input of the arithmetic logic unit. In at least one embodiment, the instruction code provided by the processor to the ALU is based at least in part on the instruction executed by the processor. In at least one embodiment, the combinational logic in the ALU processes the inputs and generates outputs that are placed on a bus within the processor. In at least one embodiment, the processor selects a target register, memory location, output device, or output storage location on the output bus so that the processor is clocked so that the result generated by the ALU will be sent to the desired location.

[0062] Within the scope of this application, the term arithmetic logic unit or ALU is used to refer to any computational logic circuit that processes operands to produce a result. For example, in this document, the term ALU can refer to a floating point unit, a DSP, a tensor core, a shader core, a coprocessor, or a CPU.

[0063] Thus, in at least one embodiment, a computer system is configured to implement one or more services that individually or collectively perform the operations of the processes described herein, and such computer system is configured with suitable hardware and / or software to enable the execution of the operations. Furthermore, the computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, is a distributed computer system comprising multiple devices operating in different ways such that the distributed computer system performs the operations described herein and such that no single device performs all of the operations.

[0064] The use of any and all examples or exemplary language (e.g., "such as") provided herein is intended merely to better illuminate embodiments of the present disclosure and does not constitute a limitation on the scope of the present disclosure unless otherwise stated. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the present disclosure.

[0065] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0066] In the specification and claims, the terms "coupled" and "connected" and their derivatives may be used. It should be understood that these terms may not be synonymous with each other. Rather, in specific examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0067] Unless expressly stated otherwise, it is understood that throughout this specification, terms such as “process,” “compute,” “calculate,” “determine,” etc. refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulates and / or transforms data represented as physical quantities (e.g., electronic quantities) within the registers and / or memories of the computing system into other data similarly represented as physical quantities within the memories, registers, or other such information storage, transmission, or display devices of the computing system.

[0068] In a similar manner, the term "processor" may refer to any device or portion of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As non-limiting examples, a "processor" may be a CPU or a GPU. A "computing platform" may include one or more processors. As used herein, a "software" process may include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. In addition, each process may refer to multiple processes for executing instructions sequentially or in parallel, continuously or intermittently. In at least one embodiment, the terms "system" and "method" are used interchangeably herein, as long as a system can embody one or more methods and a method can be considered a system.

[0069] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished in a variety of ways, such as by receiving data as a parameter to a function call or a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be accomplished by transmitting data as an input or output parameter to a function call, a parameter to an application programming interface, or an interprocess communication mechanism.

[0070] Although the description herein sets forth example implementations of the technology, other architectures may also be used to implement the functionality and are intended to be within the scope of this disclosure. In addition, although the above may define specific allocations of responsibilities for descriptive purposes, the various functions and responsibilities may be allocated and divided in different ways depending on the circumstances.

[0071] 1. In some embodiments, a method includes: performing one or more biometric authentication operations to authenticate the user in each of a plurality of instances during recording of video or audio content by a user; and in response to successful authentication of the user, embedding a cryptographic watermark into the content for the instance, wherein the cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

[0072] 2. The method according to clause 1 further includes: performing device attestation to verify the integrity of at least one of the camera or audio system of the user's computing device; and generating a device authentication key in response to verifying the integrity of at least one of the camera or audio system, wherein the cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

[0073] 3. A method according to clause 1 or 2, wherein the cryptographic watermark is generated by cryptographically binding the user authentication key, the device authentication key, and the timestamp of the recorded frame of the video or audio content corresponding to the instance.

[0074] 4. A method according to any of clauses 1-3, wherein performing the device attestation includes: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content, and the set of hardware components includes the camera or the audio system of the computing device; and generating a verification chain of verification certificates, each verification certificate corresponding to a component in the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

[0075] 5. The method of any of clauses 1-4, wherein performing device attestation comprises verifying the integrity of an operating system of the computing device and an application that performs the recording of the video or audio content.

[0076] 6. A method according to any one of clauses 1 to 5, wherein the multiple instances at which the one or more biometric authentication operations are performed correspond to each frame of the video or audio content, a predetermined set of frames of the video or audio content, or a predetermined time between instances in a recording of the video or audio content.

[0077] 7. A method according to any one of clauses 1 to 6, wherein each of the multiple instances during the recording of video or audio content is determined based at least on: identifying that the user is speaking; and causing the one or more biometric authentication operations to be performed at each frame in which the user is speaking.

[0078] 8. A method according to any one of clauses 1 to 7, wherein each of the multiple instances during the recording of video or audio content is determined based at least on: analyzing audio to identify one or more predetermined trigger words or phrases spoken by the user during the recording of the video or audio content; and in response to identifying that the user has spoken a trigger word or phrase, causing the one or more biometric authentication operations to be performed when the trigger word or phrase is spoken.

[0079] 9. The method of any one of clauses 1-8, further comprising: in response to unsuccessful authentication of the user in an instance of the plurality of instances, causing the cryptographic watermark to be omitted from the video or audio content for the instance.

[0080] 10. In some embodiments, one or more processors include: processing circuitry for performing operations including: performing one or more biometric authentication operations to authenticate the user in each of multiple instances during recording of video or audio content by the user; and in response to successful authentication of the user, embedding a cryptographic watermark into the content for the instance, wherein the cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

[0081] 11. One or more processors according to claim 10, wherein the one or more processing units further perform operations comprising: performing device attestation to verify the integrity of at least one of a camera or an audio system of the user's computing device; and generating a device authentication key in response to verifying the integrity of at least one of the camera or the audio system, wherein the cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

[0082] 12. One or more processors according to clause 10 or 11, wherein the cryptographic watermark is generated by cryptographically binding the user authentication key, the device authentication key, and the timestamp of the recorded frame of the video or audio content corresponding to the instance.

[0083] 13. One or more processors according to any of clauses 10-12, wherein performing the device attestation includes: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content, and the set of hardware components includes the camera or audio system of the computing device; and generating a verification chain of verification certificates, each verification certificate corresponding to a component in the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

[0084] 14. The one or more processors of any of clauses 10-13, wherein performing the device attestation comprises verifying the integrity of an operating system of the computing device and an application that performs the recording of the video or audio content.

[0085] 15. One or more processors according to any of clauses 10-14, wherein the one or more processing units further perform operations comprising: in response to unsuccessful authentication of the user in an instance of the multiple instances, causing the cryptographic watermark to be omitted from the video or audio content for the instance.

[0086] 16. One or more processors according to any of clauses 10-15, wherein the one or more processors are included in at least one of: a system for performing simulation operations; a system for performing digital twin operations; a system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system implemented using a robot; a system for performing one or more conversational AI operations; a system implemented using one or more large language models (LLMs); a system for generating synthetic data; a system for performing one or more generative AI operations; a system comprising one or more virtual machines (VMs); a system implemented at least in part in a data center; or a system implemented at least in part using cloud computing resources.

[0087] 17. In some embodiments, a system includes: one or more processors for performing operations comprising: performing one or more biometric authentication operations to authenticate the user in each of a plurality of intervals during recording of video or audio content by a user; and in response to successful authentication of the user, embedding a cryptographic watermark into the content for the interval, wherein the cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

[0088] 18. A system according to clause 17, wherein the one or more processors further perform operations comprising: performing device attestation to verify the integrity of at least one of a camera or an audio system of the user's computing device; and generating a device authentication key in response to verifying the integrity of at least one of the camera or the audio system, wherein the cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

[0089] 19. A system according to clause 17 or 18, wherein performing the device attestation includes: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content, and the set of hardware components includes the camera or audio system of the computing device; and generating a verification chain of verification certificates, each verification certificate corresponding to a component in the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

[0090] 20. A system according to any of clauses 17 to 19, wherein the system is included in at least one of: a system for performing simulation operations; a system for performing digital twin operations; a system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; a system implemented using an edge device; a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content; a system implemented using a robot; a system for performing one or more conversational AI operations; a system implemented using one or more large language models (LLMs); a system for generating synthetic data; a system for performing one or more generative AI operations; a system comprising one or more virtual machines (VMs); a system implemented at least in part in a data center; or a system implemented at least in part using cloud computing resources.

[0091] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. A method comprising: At each of a plurality of instances during recording of the video or audio content by the user, performing one or more biometric authentication operations to authenticate the user; as well as In response to successful authentication of the user, embedding a cryptographic watermark into content for the instance, The cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

2. The method according to claim 1, further comprising: performing device attestation to verify the integrity of at least one of a camera or an audio system of the user's computing device; as well as generating a device authentication key in response to verifying the integrity of at least one of the camera or the audio system, The cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

3. The method according to claim 2, wherein: The cryptographic watermark is generated by cryptographically binding the user authentication key, the device authentication key, and a timestamp of a recorded frame of the video or audio content corresponding to the instance.

4. The method of claim 2, wherein performing the device attestation comprises: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content and the set of hardware components includes the camera or the audio system of the computing device; as well as A verification chain of verification certificates is generated, each verification certificate corresponding to one of the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

5. The method of claim 2, wherein performing device attestation comprises: The integrity of an operating system of the computing device and an application that performs the recording of the video or audio content is verified.

6. The method of claim 1 , wherein the multiple instances at which the one or more biometric authentication operations are performed correspond to each frame of the video or audio content, a predetermined set of frames of the video or audio content, or a predetermined time between instances in a recording of the video or audio content.

7. The method according to claim 1, wherein Each of the plurality of instances during recording of the video or audio content is determined based on at least: recognizing that the user is speaking; and The one or more biometric authentication operations are caused to be performed at each frame in which the user is speaking.

8. The method according to claim 1, wherein Each of the plurality of instances during recording of the video or audio content is determined based on at least: During recording of the video or audio content, analyzing the audio to identify one or more predetermined trigger words or phrases spoken by the user; as well as In response to identifying that the user has spoken a trigger word or phrase, causing the one or more biometric authentication operations to be performed while the trigger word or phrase is spoken.

9. The method according to claim 1, further comprising: In response to unsuccessful authentication of the user in an instance of the plurality of instances, the cryptographic watermark is omitted from the video or audio content for the instance.

10. One or more processors, including: A processing circuit configured to perform operations including: At each of a plurality of instances during recording of the video or audio content by the user, performing one or more biometric authentication operations to authenticate the user; and In response to successful authentication of the user, embedding a cryptographic watermark into content for the instance, The cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

11. The one or more processors of claim 10, wherein the one or more processing units further perform operations comprising: performing device attestation to verify the integrity of at least one of a camera or an audio system of the user's computing device; as well as generating a device authentication key in response to verifying the integrity of at least one of the camera or the audio system, The cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

12. The one or more processors of claim 11, wherein: The cryptographic watermark is generated by cryptographically binding the user authentication key, the device authentication key, and a timestamp of a recorded frame of the video or audio content corresponding to the instance.

13. The one or more processors of claim 11 , wherein performing the device attestation comprises: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content and the set of hardware components includes the camera or audio system of the computing device; as well as A verification chain of verification certificates is generated, each verification certificate corresponding to one of the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

14. The one or more processors of claim 11 , wherein performing the device attestation comprises: The integrity of an operating system of the computing device and an application that performs the recording of the video or audio content is verified.

15. The one or more processors of claim 10, wherein: The one or more processing units further perform operations including: In response to unsuccessful authentication of the user in an instance of the plurality of instances, the cryptographic watermark is omitted from the video or audio content for the instance.

16. The one or more processors of claim 10, wherein: The one or more processors are included in at least one of: a system for performing simulation operations; Systems for performing digital twin operations; A system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; Systems implemented using edge devices; a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content; Systems implemented using robots; A system for performing one or more conversational AI operations; A system implemented using one or more Large Language Models (LLMs); Systems for generating synthetic data; A system for performing one or more generative AI operations; A system comprising one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

17. A system comprising: One or more processors configured to perform operations comprising: During each of a plurality of intervals during recording of the video or audio content by the user, performing one or more biometric authentication operations to authenticate the user; and In response to successful authentication of the user, embedding a cryptographic watermark into the content for the interval, The cryptographic watermark is generated based on a user authentication key and represents the continued presence of the user during the recording.

18. The system of claim 17, wherein the one or more processors further perform operations comprising: performing device attestation to verify the integrity of at least one of a camera or an audio system of the user's computing device; as well as generating a device authentication key in response to verifying the integrity of at least one of the camera or the audio system, The cryptographic watermark is generated based on a combination of the user authentication key and the device authentication key to form a composite watermark.

19. The system of claim 18, wherein performing the device attestation comprises: verifying the integrity of a set of software components and a set of hardware components of the computing device, wherein the set of software components includes a recording application that records the video or audio content and the set of hardware components includes the camera or audio system of the computing device; as well as A verification chain of verification certificates is generated, each verification certificate corresponding to one of the set of hardware components and the set of software components, wherein each component corresponds to a link in the verification chain, and wherein each link adds a unique secret to the verification chain.

20. The system of claim 18, wherein: The system is included in at least one of the following: a system for performing simulation operations; Systems for performing digital twin operations; A system for performing collaborative content creation of 3D assets; a system for performing one or more deep learning operations; Systems implemented using edge devices; a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content; Systems implemented using robots; A system for performing one or more conversational AI operations; A system implemented using one or more Large Language Models (LLMs); Systems for generating synthetic data; A system for performing one or more generative AI operations; A system comprising one or more virtual machines VM; A system implemented at least in part in a data center; or A system implemented at least in part using cloud computing resources.

Images