User authentication via secure federated machine learning

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. A user verification (UV) machine learning model trained on at least one other computing device is accessed by a first computing device. A first identification signal corresponding to a first user is received and a first embedding is generated by processing the first identification signal using the UV machine learning model. A score is generated based on the first embedding and the first storage embedding, and an output is generated based on the score.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Greek Patent Application No. 20220100730 filed on September 7, 2022, the entire contents of which are incorporated herein by reference. Background Art

[0003] Aspects of the present disclosure relate to machine learning.

[0004] Machine learning architectures have been used to provide solutions to a wide variety of computing problems. Training machine learning models to perform accurately and reliably typically uses large amounts of data (and large amounts of computing resources), which are typically unavailable in common deployment systems (e.g., on end-user smartphones). In addition, in many solutions, some degree of user customization (e.g., using end-user-specific data to train models so that each user has a corresponding personalized model) is desirable for improved model performance. However, this customization relies on each user's personalized data, and conventional methods typically involve users providing a large amount of manual effort and incurring a large amount of computing costs to achieve model customization. Conventional efforts to alleviate this burden (such as via federated learning) cannot be used for common user verification problems (which typically rely on negative examples or data from other users to fully train the model) and / or introduce serious privacy and security issues (e.g., where such private data is sent to a central server for full training). Summary of the invention

[0005] Certain aspects provide a method comprising: accessing, by a first computing device, a user-verified (UV) machine learning model trained on at least one other computing device; receiving a first identification signal corresponding to a first user; generating a first embedding by processing the first identification signal using the UV machine learning model; generating a score based on the first embedding and a first stored embedding; and generating an output based on the score.

[0006] Certain aspects provide a method, including: at a central server, receiving multiple updates from multiple non-trusted computing devices using a federated learning operation, the multiple updates being generated based on identification data of multiple users; generating, by the central server, a user verification (UV) machine learning model based on the multiple updates; and deploying the UV machine learning model to an access-controlled computing device, wherein: the access-controlled computing device does not participate in the federated learning operation and is not controlled by any user of the multiple users, and the access-controlled computing device uses the UV machine learning model to verify the identity of at least one user of the multiple users.

[0007] Other aspects provide: a processing system configured to perform the aforementioned methods and those described herein; a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods and those described herein; a computer program product embodied on a computer-readable storage medium comprising code for performing the aforementioned methods and those further described herein; and a processing system comprising components for performing the aforementioned methods and those further described herein.

[0008] The following description and associated drawings set forth in detail certain illustrative features of the one or more aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The drawings depict certain aspects of the one or more aspects and therefore are not to be considered limiting of the scope of the present disclosure.

[0010] Figure 1 Depicts an example workflow for private and secure training and deployment of machine learning models.

[0011] Figure 2 Depicts an example workflow for private and secure inference using machine learning models.

[0012] Figure 3 Depicted is an example workflow for generating authorized embeddings to enable private and secure machine learning.

[0013] Figure 4 is a flowchart depicting an example method for private and secure training and deployment of machine learning models.

[0014] Figure 5 is a flowchart depicting an example method for private and secure inference using a machine learning model.

[0015] Figure 6 is a flowchart depicting an example method for securely using a machine learning model.

[0016] Figure 7 is a flowchart depicting an example method for securely training a machine learning model.

[0017] Figure 8 An example processing system configured to perform various aspects of the present disclosure is depicted.

[0018] Fig. 9 An example processing system configured to perform various aspects of the present disclosure is depicted.

[0019] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one aspect may be beneficially incorporated in other aspects without further recitation. DETAILED DESCRIPTION

[0020] Various aspects of the present disclosure provide apparatus, methods, processing systems, and non-transitory computer-readable media for improved customization and use of machine learning models with enhanced privacy and security. In some aspects, a federated learning approach that protects user privacy and data security is used to provide technology for multi-user and multi-device access.

[0021] Machine learning models have been increasingly used for user verification (UV) in various settings. For example, UV models can be used to selectively grant access (e.g., unlock a smartphone) based on a user's face, fingerprint, voice, etc. In general, UV involves granting access to a specific authorized user (or generating some other output) while denying access to unauthorized users (which may be unregistered users, or may be explicitly unauthorized users) (and in some aspects, triggering an alarm or other output). UV models can evaluate a variety of user-specific characteristics or modalities, such as fingerprints, voice commands, visual appearance (e.g., face, posture, walking gait), etc., depending on the specific implementation. In some aspects, these input characteristics may be referred to as biometric data. As used herein, the phrase "identification signal" means any user-specific characteristic or attribute (e.g., biometric data) that can be evaluated using a UV model to selectively grant access or otherwise identify an authorized user.

[0022] Training machine learning models (including UV models) typically involves using large amounts of training data that are typically unavailable to individual end users. For example, it is unlikely that any given user has thousands (or more) of samples, such as images of user faces, that are typically used for general training. In some conventional methods, the private data (e.g., images of user faces) is collected by a central system and used to train such models. In some methods, in order to help remedy privacy issues, a central model can be trained using data from a large number of users, and the model can be fine-tuned locally by each user using data for a specific user. In some aspects, this may be referred to as registration. However, in conventional systems, access security devices also participate in the training or refinement of the model registration method, and the model is subsequently deployed on the device that performs the registration. For example, in order to unlock a smartphone using a UV model, the smartphone itself typically performs a registration and refinement process. This significantly limits the applicability and use of such models.

[0023] Federated learning (FL) generally involves jointly training a model across multiple participating clients or systems. In some aspects, each participating device performs model training locally using a set of local training data (e.g., identification signals for the users of the participating devices), and sends model updates to a central host that aggregates model updates from multiple participants to generate a global model. The global model can then be redistributed to participating devices for a new round of training. In such aspects, the training can occur only on each user's trusted device (such as their smartphone), which allows their private data (e.g., identification signals) to be kept (e.g., not sent to or otherwise available on a central server).

[0024] However, FL cannot usually be used for accurate UV model training in conventional systems. Specifically, each participating device may only have access to the data of a single user (or a small group of users, such as in a mutually trusted group of users), which requires FL to handle the various non-independent and identically distributed (IID) properties of distributed datasets (whereas conventional FL usually relies on or at least benefits from data that is IID). In general, non-IID problems are particularly difficult for FL to solve. In some aspects of the present disclosure, each local model may be trained using only positive samples and positive loss terms (e.g., real images of the user), while conventional UV models also rely on negative samples and loss terms (e.g., images of other users) to ensure that the resulting model accurately distinguishes between users.

[0025] In various aspects of the present disclosure, techniques are provided that enable FL systems to train UV models in a manner that protects user privacy and security (e.g., where each user's identification signal is not available at a central server or on any other participating client device) while ensuring high model accuracy. These models can then be distributed to a variety of access-controlled systems for inference. Notably, using various aspects of the present disclosure, these access-controlled systems do not need to be trusted devices of any participating user and do not need to participate in the training or refinement process itself. In this way, users can use their private data to assist in training or refining a UV machine learning model once, and the model can be deployed for use across multiple devices and settings, thereby enabling the use of accurate UV machine learning in a more scalable and flexible manner (e.g., compared to conventional methods in which users re-register or retrain models for each access-controlled device and in which users typically have control or ownership of the access-controlled devices).

[0026] In some aspects, each user can register or participate in the training process once using a trusted device (such as a smart phone) (e.g., the user does not need to register or train the model for each access device separately), and the model can be deployed to a variety of access-controlled devices (such as smart door locks) (e.g., for short-term rentals, commercial building access, etc.). In order to enable access rights to spaces, in some aspects, users can register with specific access-controlled devices without sharing identification information, as discussed in more detail below. This allows a group of authorized users to be updated quickly and efficiently without compromising user privacy and security and without requiring users to participate in restarting training rounds.

[0027] Example workflow for private and secure training and deployment of machine learning models

[0028] Figure 1 An example workflow 100 for private and secure training and deployment of machine learning models is depicted.

[0029] In the illustrated example, a group of participating devices 105 participate in a federated learning process with a central server 120 to jointly train one or more UV models 125. Although a group of discrete participating devices 105 are depicted for conceptual clarity, in some aspects, one or more of the participating devices 105 may be implemented using hardware, software, or a combination of hardware and software, and may be implemented as part of one or more broader systems. Generally speaking, a participating device 105 corresponds to a trusted device or system of a user who wishes to enroll in the UV system.

[0030] As used herein, a device or system may be referred to as "trusted" with respect to a user or client to indicate that the system stores, processes, evaluates, uses, or otherwise maintains the user's private or secure information (e.g., identification and / or biometric data). In contrast, a device or system may be referred to as "untrusted" or "unverified" to indicate a lack of such trust. That is, the term "untrusted" does not necessarily mean that the device is untrustworthy or unsafe with respect to any given user. Instead, a device may be referred to as "untrusted" to indicate that the device has not been verified or evaluated by the user, is not controlled or owned by the user, or may or may not otherwise be a trustworthy device. For example, an "untrusted" device may be a device that is untrustworthy for a user to receive, process, or store such identification / biometric data.

[0031] For example, in the illustrated example, each participating device 105 is a trusted device of at least one corresponding user, and each participating device 105 uses a corresponding set of identification signals 110 (e.g., fingerprints, images of the user's face, etc.) to participate in the training process. However, as shown, these identification signals 110 are generally not available at the central server 120, the access device 130, or any other participating device 105. That is, the central server 120 and the access device 130 can be referred to as "untrusted" with respect to any given user. In addition, because each participating device 105 can use a corresponding set of identification signals 110 without accessing or receiving identification signals 110 from any other user or device, each participating device 105 can be referred to as "untrusted" with respect to other users, even when the participating device 105 is "trusted" with respect to the user who owns or uses the participating device 105 to participate in the training process. In general, each participating device 105 can avoid sending identification signals 110 to any other system or device in the workflow 100.

[0032] In some aspects, each user (also referred to as a client) can be associated with a single participating device 105. That is, each user can participate in the FL system using a single participating device 105. The user can then use the UV model 125 (as discussed in more detail below) to be verified or authenticated by a variety of access devices 130. Participating devices 105 can generally include a variety of computing devices, such as (but not limited to) smartphones, laptops, desktop computers, tablets, head-mounted displays (HMDs), smart watches, etc.

[0033] In some aspects, to jointly train the UV model 125, each respective participating device 105 uses its respective identification signal 110 to locally refine one or more parameters of the model. Generally speaking, the UV model 125 (and each local model trained by each participating device 105) includes a machine learning model (e.g., a neural network) that generates output embeddings based on the input identification signal 110. These embeddings can then be compared to a defined set of authorized embeddings for authorized users. If the distance or difference between the generated embeddings is within a defined or variable distance from the authorized embeddings, the FL system can determine that the input identification signal is from an authorized user.

[0034] Thus, in some aspects, to train the UV model 125 to generate reliable embeddings, the participating devices 105 can each use a local copy of the model to process one or more identification signals 110 to generate an embedding. This embedding can then be compared to the true embedding for the user, and the local model can be refined based on the difference. In this way, the local models each learn to generate more accurate embeddings for their specific users.

[0035] In some aspects, to enhance or protect privacy and security and prevent sharing of identification signals 110 between devices while still enabling federated training of UV models, participating devices 105 may each train a local model using a hidden or obfuscated identifier as a target output. For example, as discussed in more detail below, in some aspects, a target embedding may be generated using a unique user identifier (e.g., assigned by a central server 120 or other entity), a random tensor or value (e.g., selected by a participating device 105), and one or more parity bits, which may cause embeddings from different users (generated using different unique identifiers and random values) to be distinct (and in some cases, maximally distinct), thereby increasing the minimum distance between clusters in the embedding space and facilitating training of the UV model without using negative samples during training.

[0036] In some aspects, as discussed in more detail below, each participating device 105 may further combine or aggregate the personal identifier with a learned set of parameters (e.g., multiplying the personal identifier by a weight matrix having values ​​learned during a training process) to further obfuscate the personal identifier. The generated embedding (created by the machine learning model processing the input data) may then be compared to the obfuscated identifier embedding, thereby enabling the model to learn to generate an embedding based on the identification signal 110 that approximates or approaches a target embedding (as discussed in more detail below, the target embedding may be generated by combining data such as the assigned user identifier, the selected random value, the parity bits, and the learned values).

[0037] In some aspects, depending on the FL system, each participating device 105 can use any number of identification signals 110 to perform local training of the model. As shown, for each round of FL processing, each participating device 105 sends a corresponding set of model updates 115 to the central server 120. In some aspects, the model updates 115 can generally include updated model parameters and / or parameter gradients. For example, if the UV model 125 is a neural network, the model updates 115 may include a set of updated weights (generated based on the corresponding identification signals 110) from each participating device 105, a set of weight gradients, etc. In some aspects, as discussed in more detail below, the model updates 115 also include updated values ​​or gradients of weight tensors used to generate the target embedding.

[0038] As shown, the central server 120 may then use the model updates 115 from each (or at least some) of the participating devices 105 to generate an aggregate or global UV model 125. For example, the central server 120 may sum the updates, average the updates, find the median of the updates, generate a weighted average of the updates, etc.

[0039] Although a single round of training is depicted (where the central server 120 aggregates a single set of model updates 115 to generate a UV model 125), in various aspects, there may be multiple rounds of such training. That is, the central server 120 may aggregate updates to generate an updated global UV model 125, and the central server 120 may send the updated global model 125 to each participating device 105. Each participating device 105 may then use the corresponding identification signal 110 to refine the updated global model (thereby creating a new updated local model), send new model updates 115 to the central server 120 for aggregation, and so on until training is complete.

[0040] Although the illustrated example shows that all participating devices 105 participate in all rounds of FL operation, in some aspects, a subset of devices may be used in any given round. For example, the central server 120 may select a subset (e.g., ten percent) of the participating devices to participate in each round using any suitable technique (including randomly or pseudo-randomly).

[0041] As discussed above, using aspects of the present disclosure, the central server 120 does not need to receive or access the user's identification signal 110. However, the UV model 125 learns to generate an embedding based on the input identification signal, which can be reliably used to authenticate or verify whether the corresponding user (associated with the identification signal) is included in a set of authorized users. As used herein, "accessing" data may generally include reading, receiving, retrieving, storing, extracting, obtaining, processing, modifying, or otherwise interacting with the data or obtaining access rights to the data. In some aspects, although the central server 120 itself does not receive or access the identification signal 110, the central server 120 does receive the results of the calculations on the data (e.g., model updates 115), which can be used to infer or reconstruct some or all of the identification signals in the identification signal 110. Therefore, in some aspects, various security enhancement techniques (such as differential privacy, secure aggregation, etc.) can be used, which can be used to further protect the client's private data.

[0042] In the illustrated workflow 100, once the training process is complete, the central server 120 may deploy or send the trained UV model 125 to one or more access devices 130. As discussed above, each access device 130 may be untrusted with respect to one or more users. For example, the access device 130 may include a shared system, such as a security system or smart lock in a commercial or residential space (e.g., office space, short-term housing or hotel rental, private part of a park, etc.).

[0043] In the illustrated workflow 100, the access device 130 need not participate in the training process (e.g., the access device 130 does not use the identification signal 110 to generate the model update 115). However, in some aspects, one or more of the access devices 130 may also serve as a trusted participating device 105 for one or more users. That is, while some access devices 130 (e.g., smart locks in an office space) may receive UV models 125 without participating in training or being owned or controlled by any participating user, other access devices 130 (e.g., smart phones with biometric unlocking capabilities) may both participate in the training process and receive the final UV model 125 for inference.

[0044] As indicated by arrow 135, one or more of the participating devices 105 may register or otherwise authenticate with one or more access devices 130 to enable user verification using the UV model 125. For example, a participating device 105 (or another system) may register with a given access device 130 using a provided link or other technique, thereby sharing their (obfuscated) identifier / embedding to be stored as an authenticated embedding. To authenticate a user, each access device 130 may receive an identification signal (e.g., a fingerprint) from the user, process the identification signal using the UV model 125 to generate an embedding, and compare the generated embedding to a set of authorized or authenticated embeddings to determine whether to grant access (or perform some other output). In this way, a variety of access devices 130 may provide user verification and authentication using a UV model 125 trained on data specific to each user without exposing the user's security data, without requiring the user to register or train the model specifically for or on an access device 130.

[0045] In some aspects, the authenticated set of embeddings can be easily updated (adding and removing embeddings as appropriate) to register (and deregister) users without retraining or refining the UV model 125. For example, in the case of an access device 130 corresponding to a smart door lock (e.g., a lock on a hotel door), a newly authorized user can be added to the system by simply registering or adding the user's identifier embedding to the access device 130. If the user should be removed later, the user's identifier embedding can be deregistered or removed from the authorized list. In this way, the access device 130 does not need to be owned or controlled by the user, does not need to receive the user's identification information during registration, and does not need to participate in model training, but can still use the UV model 125 trained for the user's individual.

[0046] In some aspects, the workflow 100 may be repeated periodically or when a defined event occurs. For example, when a new user wishes to participate in the workflow 100, the new user may participate in a new round (or multiple rounds) of federated learning (using the user's trusted participating device 105 to generate and send model updates 115). In some aspects, the central server 120 may push a new global UV model 125 to the access device 130 each time a new user successfully participates or at a certain time (e.g., every hour, every day, etc.). In at least one aspect, each time a new identification signal or authentication request is received, the access device 130 may determine (e.g., via a query or request to the central server 120) whether an updated UV model 125 exists. This allows each access device 130 to stay updated with any new users and models.

[0047] Example workflow for private and secure inference with machine learning models

[0048] Figure 2 An example workflow 200 for private and secure inference using a machine learning model is depicted.

[0049] In the illustrated example, the access device 130 receives or accesses one or more UV models 125 from the central server 120. For example, as discussed above, the central server 120 can use a secure or privacy-enhanced federated learning operation to allow users to jointly train the UV models 125 to generate embeddings based on input data, where these embeddings can be used to verify or authenticate the user and / or input data. As discussed above, the access device 130 does not need to be a trusted device (although in some aspects, the access device 130 can be a trusted device of one or more users) and does not need to participate or participate in the training of the UV model 125 (although in some aspects, the access device 130 can participate in the training).

[0050] As shown, the access device 130 may additionally receive, directly or indirectly, one or more authorized embeddings 210 from one or more trusted devices 205. As discussed above and discussed in more detail below, the authorized embeddings 210 generally correspond to users who are authorized to access or use the access device 130 (or to access or use resources protected by the access device 130). For example, if the access device 130 corresponds to a security system in an office, the authorized embeddings 210 may correspond to employees working in the office.

[0051] The trusted devices 205 generally correspond to computing devices used, owned, maintained, controlled, or otherwise trusted by one or more users participating in or enrolled in the training of the UV model 125. For example, the trusted devices 205 may include, but are not limited to, each individual user's smartphone, laptop, etc. In at least one aspect, some or all of the trusted devices 205 may correspond to participating devices 105 participating in the training process of the UV model 125. In the illustrated example, since each trusted device 205 can directly or indirectly provide the authorized embedding 210 to the access device 130, the central server 120 does not need to receive or access the authorized embedding 210. This enhances security and privacy.

[0052] Generally, authorized embeddings 210 may be provided to or accessed by access device 130 using a variety of techniques. For example, in some aspects, a user may register with access device 130 in any suitable manner (e.g., by using a link provided by an entity controlling access device 130) and provide their identifier embedding (e.g., as discussed above and below with reference to Figure 3 The embedding may then be stored as an authorized embedding (by access device 130 or by another system or device).

[0053] In some aspects, access device 130 uses a trusted and / or encrypted local or personal network (e.g., a wireless local area network (WLAN) such as a WiFi network, Bluetooth, etc.) to access authorized embeddings 210. For example, each authorized user may use a personal device (e.g., trusted device 205) to provide their identifier embedding directly to access device 130 via a secure local network.

[0054] In some aspects, the access device 130 may access the authorized embed 210 from a secure network location. For example, a user may upload their identifier embed to a secure network location (e.g., in the cloud) using a trusted device using a secure, encrypted, or otherwise trusted connection. The access device 130 may then access or retrieve the embed (e.g., using a username and password) via a trusted or encrypted network. In this manner, the authorized embed 210 is securely provided.

[0055] In some aspects, the access device 130 (or another device storing the authorized embedding 210 on behalf of the access device 130) uses encryption or other security mechanisms to store or maintain the authorized embedding 210. For example, upon receiving the authorized embedding 210, the access device 130 may encrypt the embedding and store the encrypted embedding in a secure location. When a verification request is subsequently received, the access device 130 may decrypt the stored authorized embedding 210 to compare with the verification request, and then re-encrypt the authorized embedding 210 (or delete the decrypted embedding) for security. In some aspects, the decryption and comparison process may be performed by the access device 130 within a trusted execution environment (TEE) (e.g., one or more separate hardware components) so that the embedding cannot be read, decrypted, or otherwise accessed or extracted from the access device 130.

[0056] In the illustrated workflow 200, the access device 130 receives or accesses an identification signal 215 for verification. The identification signal 215 can generally be received or accessed in a variety of ways. For example, the identification signal 215 can be provided by another device or system (e.g., sent by the trusted device 205), or the identification signal 215 can be directly input to the access device 130 (e.g., the access device 130 may include one or more sensors, such as imaging sensors, audio sensors, fingerprint sensors, etc. to capture the identification signal 215). In some aspects, when the identification signal 215 is received, the identification signal 215 can be provided without an indication or identification of a corresponding user associated with the signal.

[0057] As discussed above, the access device 130 can process the received identification signal 215 using the UV model 125 to generate an embedding and compare the embedding to the stored authorized embeddings 210 to determine whether to verify or authenticate the identification signal 215 (and thus authenticate or authenticate the user providing the signal). In some aspects, the access device 130 verifies the identification signal 215 by calculating or determining the distance (in embedding space) between the generated embedding and each authorized embedding 210 and comparing the distance to one or more thresholds.

[0058] In some aspects, the access device 130 uses a fixed and / or global threshold distance (e.g., defined by the central server 120, defined by the entity that owns or controls the access device 130, or defined by another entity) to determine whether the generated embedding is close enough to the authorized embedding 210. This can allow the owner of the access device 130 to configure the security level of the system (e.g., by setting a small threshold so that more inputs are rejected). In some aspects, the threshold distance can be a user-configurable value. For example, when providing an authorized embedding 210 during registration, the trusted device 205 can further specify the threshold distance that should be used when comparing the generated embedding to the provided authorized embedding 210. In this way, each user can selectively set a higher or lower threshold, where a lower threshold distance (corresponding to a higher threshold similarity) will tend to reduce false positives (e.g., the probability that an unauthorized user will be mistaken for a user of the trusted device 205 is reduced), but may result in more false negatives (e.g., where authorized users are incorrectly rejected or denied access).

[0059] In at least one aspect, each user sets their respective threshold distance based on their own identification signal used to train or validate the UV model 125. For example, the trusted device 205 or participating device may set the threshold so that a fixed percentage (e.g., 90%) of the user's test samples result in a pass or validation, while the remaining samples (e.g., the remaining 10%) result in a rejection.

[0060] In the illustrated example, if the access device 130 determines that the embedding generated using the identification signal 215 is sufficiently close to an authorized embedding 210 (e.g., within a global or user-specific distance), the access device 130 may grant or provide access 220 to the user. As used herein, access 220 may generally include a variety of outputs or actions, including providing secure or protected data to the user, providing access to the access device 130 itself, providing physical access to a space or object (e.g., unlocking a door), etc. In some aspects, if the access device 130 determines that the embedding is not sufficiently close to any of the authorized embeddings 210, a variety of actions may be taken depending on the particular implementation, including dismissing or denying access 220, generating a warning or alarm, etc.

[0061] In some aspects, whenever a new UV model 125 is available (e.g., trained for a new user), the central server 120 may push the updated model to the access device 130 or otherwise notify the access device 130 that the updated UV model 125 is ready for deployment. In some aspects, the access device 130 may request or check for updated UV models 125 periodically or when certain criteria or events occur. For example, in some aspects, when the identification signal 215 is received, the access device 130 may determine whether a new model is available before processing the identification signal 215. In some aspects, if the access device 130 fails to find an authorized embedding 210 that is close enough, the access device 130 may check to see if there is a new UV model 125 that should be used.

[0062] Similarly, in some aspects, the access device 130 may determine whether an updated set of authorized embeddings 210 are available periodically (e.g., daily) or when particular events or criteria occur (e.g., when an identification signal 215 is received, when the access device 130 fails to verify the identification signal 215, etc.).

[0063] In this way, the access device 130 can use the trained UV model 125 (which is trained using specific private and secure data for a specific authorized user without exposing the private or secure data to other devices) to dynamically verify or authenticate a user who may not have ownership or control of the access device 130. Similarly, although a user can use a trusted device for a single registration, the UV model 125 can be deployed to a variety of access devices 130, enabling rapid and easy deployment of enhanced user verification techniques. In addition, since authorization can be updated by simply updating a set of authorized embeddings 210 (without retraining or refining the UV model 125), the system can quickly respond to changing conditions or events.

[0064] Example workflow for generating authorized embeddings for private and secure machine learning

[0065] Figure 3 An example workflow 300 for generating authorized embeddings to enable private and secure machine learning is depicted. In some aspects, the workflow 300 can be executed by a user device (such as a Figure 1 In some aspects, the workflow 300 may be performed by a user device (such as a user device) that wishes to register with or use an access device that uses the trained UV model. Figure 2 trusted device 205) to execute.

[0066] Generally speaking, workflow 300 is used to generate a target embedding 335. As discussed above and discussed in more detail below, target embedding 335 can be used for a variety of purposes, including training of a UV model (e.g., where target embedding 335 is used as a target or truth label during training), registration with an access-controlled device (e.g., where target embedding 335 is provided or stored as an authorized embedding to which a newly generated embedding is compared), etc.

[0067] In the illustrated example, a user-specific tensor 305 is generated. In some aspects, the user-specific tensor 305 can be based on a user identifier 310, a random tensor 315, and a set of parity bits 320. In some aspects, the user identifier 310 is a unique identifier (e.g., a vector or tensor with a unique set of values) that can be used to uniquely identify a specific user. In at least one aspect, the user identifier 310 is generated by a central system (such as a centralized system) that facilitates federated learning of the UV model. Figure 1 For example, when a new user registers or begins participating in the training process, the central server 120 may assign a unique user identifier 310 to the user and / or the user's participating device. The user identifier 310 ensures that the user-specific tensor 305 is globally unique.

[0068] In at least one aspect, the random tensor 315 can be selected by the user (or by the user's participating device) using a variety of techniques (including random or pseudo-random). The user's device can select a set of values ​​of a vector or tensor of variable or defined length to use as the random tensor 315. The random tensor 315 can thus obfuscate the unique user identifier 310 and / or the user-specific tensor 305 because the random tensor 315 may be known to the user or the user's device, but unknown to other devices (e.g., other participating systems or a central server) (e.g., not shared or accessed by other devices).

[0069] In the illustrated example, the parity bits 320 are generated using one or more error correction code (ECC) algorithms or operations. In general, ECC techniques enable data sequences to be recovered from noise and are designed to maximize the distance between different sequences (e.g., between user identifiers 310 and / or random tensors 315). That is, one or more ECC techniques (e.g., using Bose-Chaudhuri-Hocquenghem codes (BCH codes)) can be used to generate parity bits 320 based on user identifiers 310 and / or random tensors 315, so that the parity bits 320, when added to the user identifiers 310 and / or random tensors 315, play the following role: maximize the distance between the user-specific tensors 305 associated with one user and the user-specific tensors 305 of all other users.

[0070] In this way, as discussed above, the parity bits 320 enable more accurate and efficient training of the UV model even when only positive samples are available on each participating device, because the user-specific tensors 305 (and thus the resulting embeddings) from different users can be substantially (or even maximally) separated from each other. As shown, the user identifier 310, the random tensor 315, and the parity bits 320 are aggregated to form the user-specific tensor 305. For example, the user identifier 310, the random tensor 315, and the parity bits 320 can be concatenated to form the user-specific tensor 305.

[0071] In some aspects, the user-specific tensor 305 may be referred to as hidden, secret, or obfuscated, at least in part due to the random tensor 315. In some aspects, to further obfuscate the user-specific tensor 305 (thereby protecting the identity of the user) and / or to achieve improved model training, the user-specific tensor 305 is aggregated with a weight tensor 325 (e.g., a matrix of weights or values) learned during the training process using operation 330. As discussed above, the weight tensor 325 generally corresponds to a set of weights learned during training of the UV model (e.g., using gradient descent). In this way, during each round of training, the weight tensor 325 may change (and therefore, the target embedding 335 may change). For example, the parameters of the weight tensor 325 may be updated locally by each participating device, and the central server may aggregate these updates and distribute the updated weight tensor (along with the updated UV model) for the next round of training. In the illustrated example, operation 330 is a multiplication operation (eg, where user-specific tensor 305 is combined with weight tensor 325 using matrix multiplication).

[0072] In at least one aspect, the user identifier 310, the random tensor 315, and the parity bit 320 can each be a binary vector. That is, the user identifier 310, the random tensor 315, and the parity bit 320 can each be a vector in which each value is either 0 or 1. In some aspects, before applying operation 330, the system can replace all zero values ​​in the user-specific tensor 305 with a defined value (such as -1). The new user-specific tensor 305 (where all values ​​are -1 or 1) can then be processed (e.g., using operation 330) to generate a target embedding 335. In this way, the expected embedding value of the target embedding 335 is zero. That is, if the initial user-specific tensor 305 (where the value is zero or one) is used, the expected embedding will be positive (greater than zero) and the learning problem may become asymmetric. By replacing the zero value with a -1 value, the problem can become more symmetric.

[0073] As discussed above, the target embedding 335 can then be used for a variety of operations. For example, during training, the target embedding 335 can be used as a target label or true value for the user's identification signal. In this way, the UV model learns to cluster the embeddings based on the identity of the user providing the identification signal. That is, the machine learning model learns to generate embeddings based on the identification signal, where the embeddings corresponding to the first user will tend to cluster or be relatively close to each other and close to the target embedding 335 in the embedding space, while the embeddings from other users will tend to be away from the first user's embedding or separate from the first user's embedding. However, it should be noted that even if each user's embeddings are separated from each other, the embeddings of each other user will tend to remain clustered close to each other and close to the corresponding target embeddings of the other users.

[0074] As discussed above, once the model is trained, the target embedding 335 can be further used to register with the access device to enable user authentication. In some aspects, the user's device (e.g., Figure 2 The trusted device 205 of the user stores the target embed 335 in a secure location (e.g., in encrypted storage) for use with any future registrations. For example, each time a user desires to register with a new access device, the target embed 335 can be decrypted and sent or otherwise provided to the access device, where the target embed can be stored as an authorized embed, as discussed above.

[0075] Example methods for private and secure training and deployment of machine learning models

[0076] Figure 4 400 is a flowchart depicting an example method 400 for private and secure training and deployment of machine learning models. In some aspects, the method 400 is performed by a central server (such as Figure 1 In one aspect, the method 400 is Figure 1 The workflow 100 provides additional details.

[0077] At block 405, the central server initiates training of the UV model using federated learning. For example, the central server may identify a group of participating users (e.g., Figure 1 In some aspects, initiating training may be performed in response to a variety of occurrences or criteria (such as periodically, upon determining that a new user wishes to enroll, etc.). In some aspects, initiating training may include sending data to each device, such as a current version of the UV model (which may include a weight tensor for obfuscating an identity tensor, as discussed above), hyperparameters or information related to training (e.g., indicating a learning rate), a trigger to initiate local training on each participating device, a unique identifier for each user (e.g., Figure 3 user identifier 310), etc.

[0078] In some aspects, in response to receiving the initiation, each participating device may train a local version of the UV model, as discussed above. For example, each participating device may generate a local target embedding (e.g., as discussed above with reference to Figure 3 ), and accessing or collecting a set of identification signals (e.g., Figure 1 Each participating device may then use the model to process the identification data to generate an embedding and generate a loss based on the difference between the embedding and the target embedding. The loss may then be used to update the UV model parameters (e.g., using backpropagation).

[0079] At block 410, the central server receives UV model updates (e.g., Figure 1 As discussed above, these model updates may typically include updated parameters of the UV model (which may include updated parameters for generating a weight tensor for the target embedding), gradients indicating how the parameters should be updated, and the like.

[0080] At block 415, the central server updates the global UV model based on the model updates received from each participating device. That is, the central server may aggregate the model updates from each individual participant to update or generate a global version of the UV model. For example, the central server may calculate the average, sum, weighted average, mean, median, etc. of each parameter or gradient and use these values ​​to update or generate an updated global model.

[0081] At block 420, the central server determines whether one or more training termination criteria are met. In some aspects, the central server may determine whether training is complete based on a variety of termination criteria, including the number of defined training rounds or epochs, the amount of time or resources spent on training, the expected model accuracy (e.g., determined using test data), etc. If the central server determines at block 420 that the termination criteria are not met, the method 400 continues to block 425, where the central server sends the updated global UV model (and in some aspects, the updated weight tensor) to each participating client. In this way, the federated learning operation begins a new round of training, and the method 400 then returns to block 410.

[0082] If the central server determines at block 420 that the termination criteria are met, the method 400 proceeds to block 430 where the central server distributes or deploys the updated global UV model to one or more access devices (e.g., Figure 1 and / or Figure 2For example, the central server may identify any access device using the UV model and distribute the updated version to each access device.

[0083] In some aspects, this distribution of the UV model may include notifying one or more of the access devices that a new version is available and allowing the access devices to request the updated model. In some aspects, the central server may distribute the UV model by making the model available for request (e.g., where the access devices request updated models periodically or according to other criteria) rather than immediately providing the UV model to each access device. As discussed above, the access devices may then use the updated model to perform reliable user authentication with enhanced security and privacy protection for the user.

[0084] At block 435, the central server determines whether there are one or more new clients or users that are desired to be registered in the system (e.g., to assist in training the UV model so that the UV model can be used to verify or authenticate the client). If not, the method 400 iterates at block 435. If there is at least one new client (or other criteria are met, such as periodic retraining), the method 400 returns to block 405 to initiate a new round of UV model training.

[0085] In this way, a central server can use secure and private federated learning techniques to train a UV model using user-specific data on trusted devices, and distribute the model to various (non-trusted) devices for runtime inference.

[0086] Example method for private and secure inference using machine learning models

[0087] Figure 5 is a flowchart depicting an example method 500 for performing private and secure inference using a machine learning model. In some aspects, the method 500 is performed by an access device, such as Figure 2 In some aspects, the method 500 is Figure 2 Workflow 200 provides additional details.

[0088] At block 505, the access device accesses the trained UV model. For example, as discussed above, the access device may access the trained UV model from a central server such as Figure 1 and Figure 2The UV model is retrieved or received from the central server 120 of the access device. In some aspects, as discussed above, the UV model is trained using a secure federated learning operation that allows each participating user to securely maintain their identification information (e.g., identification signals and / or user-specific tensors) on their local devices while still enabling the training of the UV model. As discussed above, the access device can typically access the UV model based on a variety of criteria, including periodically, upon determining that a new model is available, upon determining that a new user has registered with the access device (e.g., in response to receiving an authorized embedding from a new user who was not previously registered), etc.

[0089] At block 510, the access device accesses a set of authorized embeddings (e.g., Figure 2 For example, as discussed above, the access device may retrieve or receive the authorized embedding from the trusted user device during the registration process. In some aspects, as discussed above, the authorized embedding corresponds to the target embedding, such as using Figure 3 The workflow 300 generates a target embedding 335 .

[0090] In some aspects, as discussed above, an access device may access a target embedding from a trusted device directly (e.g., using a secure or encrypted local network), indirectly (e.g., via a trusted or encrypted cloud location), etc. In some aspects, after receiving an authorized embedding, the access device may store the authorized embedding in a secure (e.g., encrypted) location for future use. In at least one aspect, accessing an authorized embedding may include determining whether any previously stored embedding is no longer authorized. For example, the access device may determine whether a user has been deregistered, whether another entity (e.g., an entity that controls, owns, or operates the access device) has delisted or deregistered any embedding or user, etc.

[0091] In some aspects, the access device accesses the authorized embeddings periodically or in response to other criteria. For example, the access device may request an updated set of authorized embeddings in response to receiving a new UV model, in response to receiving an identification signal that cannot be verified, in response to receiving any identification signal (e.g., before evaluating a signal using a UV model), etc.

[0092] At block 515, the access device receives an identification signal of the user. As discussed above, the identification signal may generally include any information, data, or characteristic that can be used to identify the user, such as biometric data (e.g., fingerprint or eye scan), facial image or scan, audio (e.g., voice or speech recording), gesture or other physical movement, etc.

[0093] Generally speaking, the access device may use a variety of techniques to receive the identification signal depending on the specific implementation. For example, the access device may directly receive the identification signal (eg, via one or more sensors of the access device), or may receive the signal from another device or system.

[0094] At block 520, the access device generates an embedding by processing the identification signal using the accessed UV model. As discussed above, the embedding is typically a multi-dimensional vector or tensor. If each value in the embedding is used as a coordinate of a corresponding dimension in a multi-dimensional embedding space, the access device can then identify similar embeddings (e.g., those that are within a defined distance in the embedding space).

[0095] At block 525, the access device generates a score for the generated embedding based on the stored set of authorized embeddings. In one such aspect, the access device scores the generated embedding by calculating a distance (e.g., a Euclidean distance) between the generated embedding and each stored embedding, where the score may indicate, for example, a distance to at least the nearest authorized embedding in the embedding space.

[0096] At block 530, the access device may determine whether the generated score satisfies one or more defined criteria. For example, the access device may determine whether the score (e.g., distance) satisfies one or more defined minimum distance thresholds. In some aspects, as discussed above, the distance threshold is defined globally (e.g., where data from all users are compared to the same threshold). In at least one aspect, the threshold is user-specific and / or user-defined (e.g., where each stored authorized embedding is associated with a corresponding distance criterion or threshold). For example, a user (or user device) may set a threshold to ensure that a certain percentage of their training or test samples are accurately classified, or a user (or user device) may set a threshold based on personal or institutional preferences relative to false positives and false negatives.

[0097] If the access device determines at block 530 that the criteria are not met, the method 500 proceeds to block 535, where the access device denies access. That is, the access device determines that the user is not authorized (or the identification signal is not verified), and therefore denies or refuses access. As discussed above, denying access may generally include a variety of actions, depending on the particular implementation. For example, the access device may avoid opening a physical space or lock, avoid providing access rights to secure data or a device, generate a warning or alarm, etc.

[0098] If the access device determines at block 530 that the criteria are satisfied, the method 500 proceeds to block 540 where the access device provides the defined access to the user. As discussed above, providing or granting access may generally include various operations depending on the particular implementation. For example, the access device may open or unlock a physical device or object (e.g., a smart lock), provide defined data, provide access rights to the access device, etc.

[0099] Example methods for safely using machine learning models

[0100] Figure 6 is a flowchart depicting an example method 600 for securely using a machine learning model. In some aspects, the method 600 is performed by an access device, such as Figure 2 The access device 130 is executed.

[0101] At box 605, a user-verified (UV) machine learning model trained on at least one other computing device is accessed.

[0102] At block 610, a first identification signal corresponding to a first user is received.

[0103] At block 625 , a first embedding is generated by processing the first identification signal using a UV machine learning model.

[0104] At block 620 , a score is generated based on the first embedding and the first stored embedding.

[0105] At block 625, an output is generated based on the score.

[0106] In some aspects, at least one other computing device is a non-trusted device relative to the first computing device.

[0107] In some aspects, the UV machine learning model is trained at least in part using a federated learning operation performed by a trusted computing device of the first user, at least one other computing device, and a central server, the federated learning operation being performed based on at least one identification signal corresponding to the first user, and the at least one identification signal being unavailable at the central server.

[0108] In some aspects, the federated learning operation includes: generating a training embedding by processing a training identification signal corresponding to a first user using a version of the UV machine learning model; generating a target embedding by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during the federated learning operation; and generating a loss based on the training embedding and the target embedding.

[0109] In some aspects, the UV machine learning model is further distributed to multiple computing devices to perform user verification for multiple users.

[0110] In some aspects, method 600 further includes receiving the first storage embedding from a trusted computing device of the first user via the trusted communication link.

[0111] In some aspects, method 600 further includes receiving the encrypted first embedding from the secure network location, wherein the encrypted first embedding is sent to the secure network location by a trusted computing device of the first user, and decrypting the encrypted first embedding.

[0112] In some aspects, method 600 also includes: receiving an updated UV machine learning model, which is trained using a federated learning operation performed in part by a trusted computing device of a second user; receiving a second identification signal corresponding to the second user; and generating a second embedding by processing the second identification signal using the updated UV machine learning model.

[0113] In some aspects, the first stored embedding is generated by a trusted device of the first user by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during training of the UV machine learning model.

[0114] In some aspects, generating the score includes: calculating a distance between the first embedding and the first stored embedding; and determining whether the distance satisfies a defined distance threshold.

[0115] In some aspects, generating an output based on the score includes, in response to determining that the score satisfies the one or more defined criteria, granting the first user access to the secure entity.

[0116] In some aspects, the secure entity is a first computing device or a physical space.

[0117] In some aspects, the first storage embedding is one storage embedding of a plurality of storage embeddings.The plurality of storage embeddings (including the first storage embedding) may be stored on the first computing device.

[0118] In some aspects, the first identification signal includes biometric data of the first user.

[0119] Example method for safely training machine learning models

[0120] Figure 7 is a flowchart depicting an example method 700 for securely training a machine learning model. In some aspects, the method 700 is performed by a central server (such as Figure 1 The central server 120) is executed.

[0121] At block 705 , a plurality of updates are received from a plurality of non-trusted computing devices using a federated learning operation, the plurality of updates being generated based on identification data of a plurality of users.

[0122] At block 710, a user verification (UV) machine learning model is generated based on the plurality of updates.

[0123] At box 715, the UV machine learning model is deployed to an access-controlled computing device, where the access-controlled computing device does not participate in the federated learning operation and is not controlled by any user of the multiple users, and the access-controlled computing device uses the UV machine learning model to verify the identity of at least one user of the multiple users.

[0124] In some aspects, method 700 further includes: deploying the UV machine learning model to a plurality of access-controlled computing devices.

[0125] In some aspects, identification data for multiple users is not available on the central server.

[0126] Example processing system for training and deploying machine learning models

[0127] In some respects, reference Figures 1 to 7 The described workflows, techniques, and methods may be implemented on one or more devices or systems. Figure 8 Depicted are various aspects configured to implement the present disclosure including, for example, reference Figures 1 to 7 An example processing system 800 for describing the techniques and methods. In one aspect, the processing system 800 may correspond to a central server, such as Figure 1 and Figure 2 Central server 120. Although depicted as a single system for conceptual clarity, in at least some aspects, as discussed above, the operations described below with respect to processing system 800 may be distributed across any number of devices.

[0128] Processing system 800 includes a central processing unit (CPU) 802, which may be a multi-core CPU in some examples. Instructions executed at CPU 802 may be loaded, for example, from a program memory associated with CPU 802, or may be loaded from a partition of memory 824.

[0129] The processing system 800 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 804 , a digital signal processor (DSP) 806 , a neural processing unit (NPU) 808 , a multimedia processing unit 810 , and wireless connectivity components 812 .

[0130] An NPU, such as 808, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0131] The NPU, such as 808, is configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, the NPU may be part of a dedicated neural network accelerator.

[0132] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can usually still be performed independently.

[0133] NPUs designed to accelerate training are typically configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing dataset (usually labeled or tagged), iterating on the dataset, and then adjusting model parameters (such as weights and biases) to improve model performance. Generally speaking, optimization based on error predictions involves passing back through the layers of the model and determining the gradient to reduce the prediction error.

[0134] NPUs designed to accelerate inference are typically configured to operate on complete models. Such NPUs can thus be configured to input new data slices and quickly process the new data through an already trained model to generate model output (e.g., inference).

[0135] In one specific implementation, the NPU 808 is part of one or more of the CPU 802 , the GPU 804 , and / or the DSP 806 .

[0136] In some examples, the wireless connectivity component 812 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. The wireless connectivity component 812 is further connected to one or more antennas 814.

[0137] The processing system 800 may also include one or more sensor processing units 816 associated with any manner of sensors, one or more image signal processors (ISPs) 818 associated with any manner of image sensors, and / or a navigation component 820, which may include a satellite-based positioning system component (e.g., GPS or GLONASS) and an inertial positioning system component.

[0138] The processing system 800 may also include one or more input and / or output devices 822, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, speakers, microphones, and the like.

[0139] In some examples, one or more of the processors of processing system 800 may be based on the ARM or RISC-V instruction set.

[0140] The processing system 800 also includes a memory 824, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 824 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 800.

[0141] In particular, in this example, memory 824 includes update component 824A and deployment component 824B as well as model parameters 824C. Although for conceptual clarity, Figure 8 Components illustrated (and other components not depicted) are depicted as discrete components, but the illustrated components (and other components not depicted) in various aspects may be implemented jointly or separately.

[0142] As discussed above, model parameters 824C may generally correspond to one or more UV models (e.g., Figure 1 and Figure 2 The UV model 125 of FIG. 824C may include parameters of the target embedding. For example, the model parameters 824C may include weights of a neural network, weights of a weight tensor used to generate a target embedding, and the like.

[0143] Processing system 800 also includes update circuitry 826 and deployment circuitry 827. The depicted circuitry and other circuitry not depicted may be configured to perform various aspects of the techniques described herein.

[0144] For example, update component 824A and update circuit 826 can be used to facilitate federated learning and aggregate model updates (from participating devices) to generate an updated global UV model, as discussed above. Deployment component 824B and deployment circuit 827 can be used to deploy or distribute the updated global UV model to access devices, as discussed above.

[0145] Although for clarity in Figure 8 Although depicted as separate components and circuits, the update circuit 826 and the deployment circuit 827 may be implemented together or separately in other processing devices of the processing system 800, such as within the CPU 802, GPU 804, DSP 806, NPU 808, etc.

[0146] Generally speaking, the processing system 800 and / or its components may be configured to perform the methods described herein.

[0147] It is worth noting that in other aspects, aspects of the processing system 800 may be omitted, such as where the processing system 800 is a server computer, etc. For example, in other aspects, the multimedia processing unit 810, the wireless connectivity component 812, the sensor processing unit 816, the ISP 818, and / or the navigation component 820 may be omitted. Furthermore, aspects of the processing system 800 may be distributed among multiple devices.

[0148] Example processing system for performing inference using machine learning models

[0149] In some respects, reference Figures 1 to 7 The described workflows, techniques, and methods may be implemented on one or more devices or systems. Fig. 9 Depicted are various aspects configured to implement the present disclosure including, for example, reference Figures 1 to 7 An example processing system 900 for describing techniques and methods. In one aspect, the processing system 900 may correspond to an access device such as Figure 1 and Figure 2 Access device 130. Although depicted as a single system for conceptual clarity, in at least some aspects, as discussed above, the operations described below with respect to processing system 900 may be distributed across any number of devices.

[0150] The processing system 900 includes a central processing unit (CPU) 902, which may be a multi-core CPU in some examples. Instructions executed at the CPU 902 may be loaded, for example, from a program memory associated with the CPU 902, or may be loaded from a partition of the memory 924.

[0151] The processing system 900 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 904 , a digital signal processor (DSP) 906 , a neural processing unit (NPU) 908 , a multimedia processing unit 910 , and wireless connectivity components 912 .

[0152] An NPU, such as 908, is typically a dedicated circuit configured to implement control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU is sometimes alternatively referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graphics processing unit.

[0153] The NPU, such as 908, is configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system on a chip (SoC), while in other examples, the NPU may be part of a dedicated neural network accelerator.

[0154] The NPU can be optimized for training or inference, or in some cases configured to balance performance between the two. For NPUs that can perform both training and inference, the two tasks can usually still be performed independently.

[0155] NPUs designed to accelerate training are typically configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing dataset (usually labeled or tagged), iterating on the dataset, and then adjusting model parameters (such as weights and biases) to improve model performance. Generally speaking, optimization based on error predictions involves passing back through the layers of the model and determining the gradient to reduce the prediction error.

[0156] NPUs designed to accelerate inference are typically configured to operate on complete models. Such NPUs can thus be configured to input new data slices and quickly process the new data through an already trained model to generate model output (e.g., inference).

[0157] In one implementation, the NPU 908 is part of one or more of the CPU 902 , the GPU 904 , and / or the DSP 906 .

[0158] In some examples, wireless connectivity component 912 may include, for example, subcomponents for third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. Wireless connectivity component 912 is further connected to one or more antennas 914.

[0159] The processing system 900 may also include one or more sensor processing units 916 associated with any manner of sensors, one or more image signal processors (ISPs) 918 associated with any manner of image sensors, and / or a navigation component 920, which may include a satellite-based positioning system component (e.g., GPS or GLONASS) and an inertial positioning system component.

[0160] The processing system 900 may also include one or more input and / or output devices 922, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, speakers, microphones, and the like.

[0161] In some examples, one or more of the processors of processing system 900 may be based on the ARM or RISC-V instruction set.

[0162] The processing system 900 also includes a memory 924, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 924 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 900.

[0163] In particular, in this example, memory 924 includes embedding component 924A, scoring component 924B, and access component 924C, as well as model parameters 924D and authorized embedding 924E. Although for conceptual clarity, Fig. 9 Components illustrated (and other components not depicted) are depicted as discrete components, but the illustrated components (and other components not depicted) in various aspects may be implemented jointly or separately.

[0164] As discussed above, model parameters 924D may generally correspond to one or more UV models (e.g., Figure 1 and Figure 2 The UV model 125 of FIG. 124B may be a parameter of a target model 125 of FIG. 124C. For example, the model parameters 924D may include weights of a neural network, weights of a weight tensor used to generate a target embedding, etc. The authorized embeddings 924E generally correspond to a set of target embeddings (e.g., obtained during a registration process) for users who are authorized to access the device, as discussed above.

[0165] Processing system 900 also includes embedding circuitry 926, scoring circuitry 927, and access circuitry 928. The depicted circuitry and other circuitry not depicted may be configured to perform various aspects of the techniques described herein.

[0166] For example, embedding component 924A and embedding circuit 926 can be used to generate an embedding by processing a received identification signal or data using a trained UV model, as discussed above. Scoring component 924B and scoring circuit 927 can be used to generate a score (e.g., distance) for the generated embedding based on a stored set of authorized embeddings 924E, as discussed above. Access component 924C and access circuit 928 can be used to selectively grant or deny access rights to a user (e.g., based on the generated score and one or more thresholds), as discussed above.

[0167] Although for clarity in Fig. 9 900 , but the embedded circuit 926 , the scoring circuit 927 , and the access circuit 928 may be implemented together or separately in other processing devices of the processing system 900 , such as in the CPU 902 , the GPU 904 , the DSP 906 , the NPU 908 , etc.

[0168] Generally speaking, the processing system 900 and / or its components may be configured to perform the methods described herein.

[0169] It is worth noting that in other aspects, aspects of the processing system 900 may be omitted, such as where the processing system 900 is a server computer, etc. For example, in other aspects, the multimedia processing unit 910, the wireless connectivity component 912, the sensor processing unit 916, the ISP 918, and / or the navigation component 920 may be omitted. Furthermore, aspects of the processing system 900 may be distributed among multiple devices.

[0170] Sample Clauses

[0171] Specific implementation examples are described in the following numbered clauses:

[0172] Item 1: A method comprising: accessing, by a first computing device, a user-verified (UV) machine learning model trained on at least one other computing device; receiving a first identification signal corresponding to a first user; generating a first embedding by processing the first identification signal using the UV machine learning model; generating a score based on the first embedding and a first stored embedding; and generating an output based on the score.

[0173] Clause 2: The method of clause 1, wherein the at least one other computing device is an untrusted device relative to the first computing device.

[0174] Clause 3: A method according to clause 1 or 2, wherein: the UV machine learning model is trained at least in part using a federated learning operation performed by a trusted computing device of the first user, the at least one other computing device, and a central server, the federated learning operation being performed based on at least one identification signal corresponding to the first user, and the at least one identification signal is not available at the central server.

[0175] Clause 4: A method according to any one of clauses 1-3, wherein the federated learning operation comprises: generating a training embedding by processing a training identification signal corresponding to the first user using a version of the UV machine learning model; generating a target embedding by aggregating user-specific tensors and weight tensors, wherein the weight tensor is learned during the federated learning operation; and generating a loss based on the training embedding and the target embedding.

[0176] Clause 5: The method of any one of clauses 1-4, wherein the UV machine learning model is further distributed to a plurality of computing devices to perform user authentication for a plurality of users.

[0177] Clause 6: The method of any of clauses 1-5, further comprising: receiving the first storage embedding from a trusted computing device of the first user via a trusted communication link.

[0178] Clause 7: The method of any one of clauses 1-6, further comprising: receiving an encrypted first embed from a secure network location, wherein the encrypted first embed is sent to the secure network location by a trusted computing device of the first user; and decrypting the encrypted first embed.

[0179] Clause 8: The method according to any one of clauses 1-7 further includes: receiving an updated UV machine learning model, wherein the updated UV machine learning model is trained using a federated learning operation performed in part by a trusted computing device of a second user; receiving a second identification signal corresponding to the second user; and generating a second embedding by processing the second identification signal using the updated UV machine learning model.

[0180] Clause 9: A method according to any of clauses 1-8, wherein the first stored embedding is generated by a trusted device of the first user by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during training of the UV machine learning model.

[0181] Clause 10: A method according to any of clauses 1-9, wherein generating the score comprises: calculating a distance between the first embedding and the first stored embedding; and determining whether the distance satisfies a defined distance threshold.

[0182] Clause 11: The method of any of clauses 1-10, wherein generating the output based on the score comprises: in response to determining that the score satisfies one or more defined criteria, granting the first user access rights to a secure entity.

[0183] Clause 12: The method of any of clauses 1-11, wherein the security entity is the first computing device.

[0184] Clause 13: A method as described in any of clauses 1-11, wherein the security entity is a physical space.

[0185] Clause 14: The method of any of clauses 1-13, wherein the first storage embed is stored on the first computing device.

[0186] Clause 15: A method as described in any of clauses 1-14, wherein the first identification signal includes biometric data of the first user.

[0187] Clause 16: A method comprising: at a central server, receiving multiple updates from multiple non-trusted computing devices using a federated learning operation, the multiple updates being generated based on identification data of multiple users; generating a user verification (UV) machine learning model by the central server based on the multiple updates; and deploying the UV machine learning model to an access-controlled computing device, wherein: the access-controlled computing device does not participate in the federated learning operation and is not controlled by any of the multiple users, and the access-controlled computing device uses the UV machine learning model to verify the identity of at least one of the multiple users.

[0188] Clause 17: The method according to Clause 16 also includes: deploying the UV machine learning model to multiple access-controlled computing devices.

[0189] Clause 18: The method of clause 16 or 17, wherein the identification data of the plurality of users is not available on the central server.

[0190] Clause 19: A processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method according to any one of clauses 1-18.

[0191] Clause 20: A processing system comprising: means for performing the method according to any one of clauses 1-18.

[0192] Clause 21: A non-transitory computer-readable medium comprising: computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any of clauses 1-18.

[0193] Clause 22: A computer program product embodied on a computer readable storage medium, the computer program product comprising: code for performing the method according to any of clauses 1-18.

[0194] Additional considerations

[0195] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, the functions and arrangements of the elements discussed may be changed without departing from the scope of the present disclosure. Various examples may omit, replace, or add various processes or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, the features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods practiced using other structures, functionality, or structures and functionality in addition to or different from the various aspects of the present disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of the claims.

[0196] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0197] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items (including single members). As an example, "at least one of a, b, or c" is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination with multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0198] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include computing, calculating, processing, deriving, investigating, searching (e.g., searching in a table, a database, or another data structure), ascertaining, and the like. Additionally, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Additionally, "determining" may include resolving, selecting, choosing, establishing, and the like.

[0199] The method disclosed herein includes one or more steps or actions for implementing the method. The method steps and / or actions may be interchangeable with each other without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. In addition, the various operations of the above-mentioned method may be performed by any suitable component capable of performing the corresponding function. The component may include various hardware and / or software components and / or modules, including but not limited to circuits, application specific integrated circuits (ASICs) or processors. Generally speaking, in the case of operations illustrated in the drawings, those operations may have corresponding corresponding components plus functional components with similar numbers.

[0200] The following claims are not intended to be limited to the various aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless explicitly stated, reference to an element in the singular form is not intended to mean "one and only one", but "one or more". Unless specifically stated, the term "some" refers to one or more. No claim element should be interpreted according to the provisions of 35 U.S.C. § 112 (f) unless the element is explicitly recorded using the phrase "a component for...", or in the case of a method claim, the element is recorded using the phrase "a step for...". All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or will be known later to a person of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recorded in the claims.

Claims

1. A computer-implemented method comprising: accessing, by the first computing device, a user-verified (UV) machine learning model trained on at least one other computing device; receiving a first identification signal corresponding to a first user; generating a first embedding by processing the first identification signal using the UV machine learning model; generating a score based on the first embedding and the first stored embedding; as well as An output is generated based on the score. 2 . The computer-implemented method of claim 1 , wherein the at least one other computing device is an untrusted device relative to the first computing device.

3. The computer-implemented method of claim 2, wherein: The UV machine learning model is trained at least in part using a federated learning operation performed by a trusted computing device of the first user, the at least one other computing device, and a central server, The federated learning operation is performed based on at least one identification signal corresponding to the first user, and The at least one identification signal is unavailable at the central server.

4. The computer-implemented method of claim 3, wherein the federated learning operation comprises: generating a training embedding by processing a training identification signal corresponding to the first user using a version of the UV machine learning model; generating a target embedding by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during the federated learning operation; and A loss is generated based on the training embeddings and the target embeddings.

5. The computer-implemented method of claim 1 , wherein the UV machine learning model is further distributed to a plurality of computing devices to perform user verification for a plurality of users.

6. The computer-implemented method of claim 1 , further comprising: The first storage embed is received from a trusted computing device of the first user via a trusted communication link.

7. The computer-implemented method of claim 1 , further comprising: receiving an encrypted first embedding from a secure network location, wherein the encrypted first embedding is sent to the secure network location by a trusted computing device of the first user, and The encrypted first embedding is decrypted.

8. The computer-implemented method of claim 1 , further comprising: receiving an updated UV machine learning model, the updated UV machine learning model being trained using federated learning operations performed in part by a trusted computing device of a second user; receiving a second identification signal corresponding to the second user; as well as A second embedding is generated by processing the second identification signal using the updated UV machine learning model.

9. The computer-implemented method of claim 1 , wherein the first storage embedding is generated by a trusted device of the first user by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during training of the UV machine learning model.

10. The computer-implemented method of claim 1 , wherein generating the score comprises: calculating a distance between the first embedding and the first stored embedding; as well as A determination is made as to whether the distance satisfies a defined distance threshold.

11. The computer-implemented method of claim 1 , wherein generating the output based on the score comprises: In response to determining that the score satisfies one or more defined criteria, the first user is granted access rights to a secure entity.

12. The computer-implemented method of claim 11, wherein the security entity is the first computing device or a physical space.

13. The computer-implemented method of claim 1, wherein the first storage embedding is stored on the first computing device.

14. The computer-implemented method of claim 1, wherein the first identification signal comprises biometric data of the first user.

15. A computer-implemented method comprising: receiving, at a central server, a plurality of updates from a plurality of non-trusted computing devices using a federated learning operation, the plurality of updates being generated based on identification data of a plurality of users; generating, by the central server, a user verification (UV) machine learning model based on the plurality of updates; as well as Deploy the UV machine learning model to an access-controlled computing device, wherein: The access-controlled computing device does not participate in the federated learning operation and is not controlled by any user of the plurality of users, and The access-controlled computing device uses the UV machine learning model to verify the identity of at least one user among the plurality of users.

16. The computer-implemented method of claim 15, further comprising: Deploy the UV machine learning model to multiple access-controlled computing devices.

17. The computer-implemented method of claim 15, wherein the identification data of the plurality of users is not available on the central server.

18. A processing system comprising: a memory including computer executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform operations including: accessing, by the first computing device, a user-verified (UV) machine learning model trained on at least one other computing device; receiving a first identification signal corresponding to a first user; generating a first embedding by processing the first identification signal using the UV machine learning model; generating a score based on the first embedding and the first stored embedding; as well as An output is generated based on the score.

19. The processing system of claim 18, wherein the at least one other computing device is an untrusted device relative to the first computing device.

20. The processing system of claim 19, wherein: The UV machine learning model is trained at least in part using a federated learning operation performed by a trusted computing device of the first user, the at least one other computing device, and a central server, The federated learning operation is performed based on at least one identification signal corresponding to the first user, and The at least one identification signal is unavailable at the central server.

21. The processing system of claim 20, wherein the federated learning operation comprises: generating a training embedding by processing a training identification signal corresponding to the first user using a version of the UV machine learning model; generating a target embedding by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during the federated learning operation; and A loss is generated based on the training embeddings and the target embeddings.

22. The processing system of claim 18, wherein the UV machine learning model is further distributed to a plurality of computing devices to perform user authentication for a plurality of users.

23. The processing system of claim 18, the operations further comprising: The first storage embed is received from a trusted computing device of the first user via a trusted communication link.

24. The processing system of claim 18, the operations further comprising: receiving an encrypted first embedding from a secure network location, wherein the encrypted first embedding is sent to the secure network location by a trusted computing device of the first user, and The encrypted first embedding is decrypted.

25. The processing system of claim 18, the operations further comprising: receiving an updated UV machine learning model, the updated UV machine learning model being trained using federated learning operations performed in part by a trusted computing device of a second user; receiving a second identification signal corresponding to the second user; as well as A second embedding is generated by processing the second identification signal using the updated UV machine learning model.

26. The processing system of claim 18, wherein the first storage embedding is generated by a trusted device of the first user by aggregating a user-specific tensor and a weight tensor, wherein the weight tensor is learned during training of the UV machine learning model.

27. The processing system of claim 18, wherein generating the score comprises: calculating a distance between the first embedding and the first stored embedding; as well as A determination is made as to whether the distance satisfies a defined distance threshold.

28. The processing system of claim 18, wherein generating the output based on the score comprises: In response to determining that the score satisfies one or more defined criteria, the first user is granted access rights to a secure entity.

29. The processing system of claim 28, wherein the security entity is the first computing device or a physical space.

30. The processing system of claim 18, wherein the first memory embed is stored on the first computing device.

31. The processing system of claim 18, wherein the first identification signal comprises biometric data of the first user.

32. A processing system comprising: a memory including computer executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform operations including: receiving, at a central server, a plurality of updates from a plurality of non-trusted computing devices using a federated learning operation, the plurality of updates being generated based on identification data of a plurality of users; generating, by the central server, a user verification (UV) machine learning model based on the plurality of updates; as well as Deploy the UV machine learning model to an access-controlled computing device, wherein: The access-controlled computing device does not participate in the federated learning operation and is not controlled by any user of the plurality of users, and The access-controlled computing device uses the UV machine learning model to verify the identity of at least one user among the plurality of users.

33. The processing system of claim 32, the operations further comprising: Deploy the UV machine learning model to multiple access-controlled computing devices.

34. The processing system of claim 32, wherein the identification data of the plurality of users is not available on the central server.

35. A non-transitory computer-readable medium comprising: Computer executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform operations including: accessing, by the first computing device, a user-verified (UV) machine learning model trained on at least one other computing device; receiving a first identification signal corresponding to a first user; generating a first embedding by processing the first identification signal using the UV machine learning model; generating a score based on the first embedding and the first stored embedding; as well as An output is generated based on the score.

Images