Embedded facial recognition

Abstract

Provides embedded facial recognition. [Solution] An embedded face recognition system that receives captured images from an image acquisition device. Face images are extracted from the captured images. The extracted face images are aligned to a reference face model. A machine learning model is used to generate face embeddings based on the aligned extracted face images. Individuals associated with the face embeddings are identified based on the generated face embeddings and a database of existing face embeddings.

Description

Technical Field

[0001] Aspects and implementations of the present disclosure relate to embedded face recognition.

[0002] Background Art A face recognition (face ID) system uses computer vision technology and machine learning technology to recognize or verify a person's identity based on the person's face features. Embedded face ID systems (i.e., face ID systems implemented on embedded devices) have gained significant traction in various applications due to their portability and convenience. These systems integrate advanced face recognition algorithms into compact, often mobile devices, enabling real-time identification in diverse environments ranging from unlocking smartphones to security checkpoints.

[0003] Aspects and implementations of the present disclosure will be more fully understood from the following detailed description and the accompanying drawings of various aspects and implementations of the present disclosure, provided that those aspects and implementations are not to be construed as limiting the present disclosure to specific aspects or implementations, but are for purposes of explanation and understanding only.

Brief Description of the Drawings

[0004] [Figure 1] It is a diagram of an exemplary computer system according to an implementation of the present disclosure. [Figure 2] It is a diagram of an exemplary continuous face detection component of the embedded face recognition system of FIG. 1 according to an implementation of the present disclosure. [Figure 3] It is a diagram of an exemplary on-demand face recognition component of the embedded face recognition system of FIG. 1 according to an implementation of the present disclosure. [Figure 4] It is a flowchart of an exemplary method for embedded face recognition according to an implementation of the present disclosure. [Figure 5] It is a block diagram showing an exemplary computer system according to an implementation of the present disclosure.

[0005] Modes for carrying out the invention Some aspects of this disclosure relate to embedded facial ID systems. Due to the strong traction in various applications of embedded facial ID systems, embedded facial ID systems incorporate a pipeline that includes face detection, landmark estimation, alignment, and identification steps. Face alignment normalizes the detected image using similarity transformation based on five facial landmarks, and identification compares the newly captured face embedding with a stored database of embeddings.

[0006] However, two main technical challenges hinder the efficiency and accuracy of embedded facial ID systems on devices with limited computational resources. First, current alignment methods assume coplanarity and uniform scaling of landmarks, proving insufficient for non-frontal captures and resulting in suboptimal alignment for oblique views. Second, as the number of stored embeddings increases, the system encounters a linear increase in high-dimensional vector comparisons, leading to increased processing time and resource consumption. This computational complexity negatively impacts real-time performance. Consequently, improved alignment techniques capable of handling non-frontal images and more efficient embedding comparison methods are needed to improve the overall robustness, accuracy, and efficiency of these systems.

[0007] Aspects and embodiments of this disclosure address these and other limitations of existing technologies by providing embedded face recognition that enables more efficient face recognition by aligning face images of detected faces using homogeneous transformations and then preprocessing embedded images of detected faces. In particular, the system for embedded face recognition receives images sequentially and detects whether faces are present in the images (e.g., detected faces). The system aligns the detected faces by using homogeneous transformations and one or more features of the detected faces (e.g., left eye, right eye, and center of lip) to identify a set of parameters that align the detected faces to a reference face. For the aligned detected faces, the system generates embeddings (e.g., detected face embeddings) that represent unique numerical representations of the face features. The system further performs dimensionality reduction on the embeddings for faster comparison with known user (or registered user) embeddings. For each cluster in the known user (or registered user) embeddings, the system calculates a similarity score between the detected face embedding and one or more representative embeddings. The system compares each calculated similarity score to a predetermined similarity score threshold assigned to each known user (or registered user). If the calculated similarity score associated with a known user meets (for example, exceeds) a predetermined similarity score assigned to that known user, the system determines that the detected face belongs to a known user.

[0008] Aspects of this disclosure overcome these shortcomings and others by improving the accuracy of face recognition and the efficiency of embedding comparison in non-frontal face images.

[0009] Figure 1 is a simplified diagram of a computer system 10 according to the implementation of the present disclosure. The computing system 10 may be a server, workstation, personal computer (PC), mobile phone, personal digital assistant (PDA), or any other suitable computing device. The computing system 10 includes a computing processing device (also called a processing device) 50, an input device 170, and a storage device 180.

[0010] The processing device 50 includes various components capable of executing instructions that encode arithmetic operations, logical operations, or input / output operations. The processing device 50 may be a single-core processor, a multi-core processor capable of executing multiple instructions simultaneously, or a single integrated circuit, a multi-chip module. The storage device 180 may include volatile and / or non-volatile memory such as RAM, ROM, EEPROM, or any other device capable of storing data.

[0011] The computing device 50 includes an embedded face recognition system 100. The face recognition system 100 includes a continuous face detection component 110 and an on-demand face recognition component 150 that jointly provide face recognition.

[0012] The continuous face detection component 110 continuously receives input (e.g., captured images) from an image acquisition device 170 (e.g., a camera) coupled to the computing device 50, and performs detection preprocessing, bounding box, and landmark detection to make a preliminary determination of whether faces are present in the captured images. When a face is detected, the on-demand face recognition component 150 aligns the detected face and performs face ID preprocessing to prepare the aligned face image for identification. The on-demand face recognition component 150 generates embeddings that represent unique numerical representations of face features and performs postprocessing to improve these representations. The on-demand face recognition component 150 compares the embedded faces of the detected faces (e.g., detected face embeddings) with known face embeddings in a database 185 of registered faces stored in a storage device 180.

[0013] The registered face database 185 includes embeddings (e.g., face recognition) for each user registered for face recognition or detection. During registration, each user provides multiple images, each with different orientations of their face (e.g., yaw, pitch, and roll of the face). For each image of a user, the face in each image is detected and aligned, high-dimensional features are extracted, and compressed into a compact, fixed-length vector (e.g., an embedding) that serves as a unique numerical representation of the face. Each user is assigned a unique similarity threshold (e.g., a per-user similarity threshold) to distinguish them from other similar registered users (e.g., siblings). In some embodiments, the per-user similarity threshold is a threshold that distinguishes a user from all other users using the cosine distance between the user and each of the other users.

[0014] Figure 2 is a simplified diagram of a continuous face detection component 200, similar to the continuous face detection component 110 in Figure 1, according to the implementation of the present disclosure. The continuous face detection component 200 may include a detection preprocessing module 210, a face detection model 220, and a landmark detection model 230. The preprocessing module 210, and similar modules, include executable instructions that are processed and executed by one or more processing devices.

[0015] The detection preprocessing module 210 resizes the image to match the input dimensions required by a neural network model trained for face detection (e.g., face detection model 220), while maintaining image integrity and avoiding distortion. After resizing, the detection preprocessing module 210 performs image normalization to adjust the pixel values ​​of the image to fall within a specified input range based on the image's bit depth or precision. For example, a 32-bit floating-point (fp32) image is typically normalized to a range of -1 to 1. Image normalization may include zero-centering or the use of different mean and standard deviation values. The detection preprocessing module 210 converts the (resized and normalized) image to the desired precision to ensure that the output bit depth matches the requirements of the face detection model 220.

[0016] The face detection model 220 scans the entire image to generate multiple potential bounding boxes of varying sizes and positions using a sliding window and / or a predetermined anchor box. Each of these bounding boxes is assigned a confidence score indicating the likelihood that the bounding box contains a face. In some embodiments, one or more techniques, such as non-maximal suppression (NMS), systematically select high-confidence bounding boxes while removing significant overlaps according to a predetermined confidence threshold, ensuring that each face is represented by only one bounding box to eliminate redundancy. Each bounding box, defined by its coordinates in the original image, serves as a guide for extracting the sub-region containing the detected face. The face detection model 220 uses the coordinates of each bounding box (typically the x and y coordinates of the upper-left corner, as well as the width and height; other coordinate systems or reference points may be used) to isolate (e.g., crop) the corresponding portion of the original image that generates individual face images containing only the detected faces.

[0017] The landmark detection model 230 identifies important facial landmarks within a face image (e.g., facial landmarks for each eye, each corner of the mouth, and the nose). The landmark detection model 230 processes the face image through multiple layers, extracting relevant facial features and patterns used to predict the precise location of important facial landmarks within the face image. Important facial landmarks are represented as coordinate points within the face image. The output of the landmark detection model 230 consists of coordinate values ​​that effectively map important facial features of the face within the face image.

[0018] In some embodiments, the face detection model 220 and the landmark detection model 230 can be integrated into a single unified model that simultaneously detects faces and facial landmarks. In some embodiments, the face detection model 220 and the landmark detection model 230 can operate as separate sequential components as follows: the face detection model 220 first identifies faces and generates bounding boxes, and then the landmark detection model 230 processes these cropped face regions (associated with the bounding boxes) to identify specific facial landmarks.

[0019] Figure 3 is a simplified diagram of an on-demand face recognition component 300, similar to the on-demand face recognition component 150 in Figure 1, according to the implementation of this disclosure. The on-demand face recognition component 300 includes a face alignment module 310, a face ID preprocessing module 320, an embedding generation module 330, and an embedding postprocessing module 340. The face alignment module 310, the face ID preprocessing module 320, the embedding generation module 330, the embedding postprocessing module 340, and similar modules include executable instructions that are processed and executed by one or more processing devices.

[0020] The face alignment module 310 aligns faces in a face image (e.g., detected faces) using similarity transformation. Similarity transformation refers to a method of preserving the shape of an object (e.g., detected faces) during rotation, translation, and uniform scaling. Rotation refers to rotating an object by a specific angle. Translation refers to shifting or moving an object along the x and y axes. Uniform scaling refers to scaling an object by the same coefficient in both the x and y directions while maintaining the aspect ratio. These operations (rotation, translation, and scaling) can be combined into a single transformation, typically two-dimensional (2D), that affects a set of points or an image. Transformations can be expressed as follows:

number

[0021] This transformation formula involves rotation (θ), uniform scaling (s), and x-axis translation (t). x ), and y-axis translation (t yIt includes four degrees of freedom. In face alignment, these four parameters are important for converting the detected face to match the reference face. To determine these parameters, two pairs of corresponding points, typically important face features of both the detected face and the reference face, are used. Two pairs are needed because a single pair can only provide information about translation, while two pairs provide sufficient data to calculate all four transformation parameters. The first pair establishes the baseline for translation and rotation, and the second pair enables scale determination and improves rotation calculation. By comparing the positions, angles, and distances between these point pairs on both faces, the face alignment module 310 can calculate the exact rotation, scaling, and translation necessary to align the detected face with the reference face. Therefore, the face alignment module 310 can efficiently solve for all four parameters in a similarity transformation formula (e.g., Equation (1)), enabling precise face alignment that adapts to the geometric shape of individual faces while normalizing the position, orientation, and scale for subsequent recognition tasks.

[0022] Since there are at least five features in the detected face, least squares estimation can be used in face alignment to determine the optimal parameters for minimizing the overall difference between points in the detected face and points in the reference face, compensating for detection inaccuracies and face variations. By leveraging all available data, a more robust and accurate alignment is provided compared to using only the two pairs required by the degrees of freedom. However, in an oblique view, unlike the eyes and mouth corners, which are typically located on one plane, the tip of the nose protrudes on a different plane, so the position of the tip of the nose has a different scale compared to other features. Least squares estimation, which treats all points equally, does not account for this parallax that results in inaccurate alignment in non-frontal views.

[0023] Instead, when using homogeneous coordinates, the transformation can be represented as a 3×3 matrix (e.g., homogeneous transformation). The homogeneous transformation can be represented as follows. [Mathematics] (2)

[0024] The same transformation involves parameters [Mathematics] [Mathematics] [Mathematics] [Mathematics] Translation parallel to the x-axis (t x ), and translation parallel to the y-axis (t y ), encompassing six degrees of freedom. Similarly, to determine these parameters, three pairs of corresponding points, typically the important facial features of both the detected face and the reference face, are used. The important facial features of both the detected face and the reference face are typically the center of the left eye, the right eye, and the lips on the same plane. [Mathematics] (3)

[0025] where x0, y0 refer to the original point (before transformation) of the left eye, x'0, y'0 refer to the transformed point (after transformation) of the left eye, x1, y1 refer to the original point (before transformation) of the right eye, x'1, y'1 refer to the transformed point (after transformation) of the right eye, x2, y2 refer to the original point (before transformation) of the center of the lips, and x'2, y'2 refer to the transformed point (after transformation) of the center of the lips.

[0026] Solving for the parameters of a homogeneous transformation involves forming a system of linear equations that are overdetermined when multiple pairs of points are considered, for each pair of points (for example, establishing two equations). Then, the least squares method is used to find the best-fitting parameters that minimize the overall error across all pairs of points. Techniques such as singular value decomposition (SVD) and normal equations can be used to obtain the values ​​of the parameters of a homogeneous transformation.

[0027] The face alignment module 310 constructs a 3x3 matrix using the acquired parameters as follows:

number

[0028] This matrix is ​​then applied to each pixel of the face image. The resulting non-integer pixel coordinates are interpolated, typically using bilinear or bicubic methods, to generate a new aligned face image. This aligned image features standardized size, orientation, and facial feature positions that conform to the reference image.

[0029] The face ID preprocessing module 320, like the detection preprocessing module 210, normalizes the aligned face image and adjusts the pixel values, transforming the aligned face image to the desired accuracy, data type, channel order, contrast enhancement, and / or noise reduction.

[0030] The embedding generation module 330 receives a face image and passes it through a pre-trained neural network model trained to process the input image, outputting an embedding (e.g., detected face embedding). In some embodiments, the pre-trained neural network model may be optimized using a special loss function such as a contrast loss, triplet loss, quadruplet loss, or additive angular margin loss to enhance the model's ability to distinguish between different individuals. Ideally, embeddings of different images of the same person would cluster together in a high-dimensional space, while embeddings of different individuals would remain clearly separated, which is crucial for accurate face recognition and identification.

[0031] The embedding post-processing module 340 reduces the dimensionality of the embeddings received from the face ID pre-processing module 320. The embedding post-processing module 340 can perform dimensionality reduction using various techniques such as principal component analysis, t-distribution stochastic neighbor embedding (t-SNE), and homogeneous manifold approximation and projection (UMAP).

[0032] Once the dimensionality of the embeddings is reduced, the embedding post-processing module 340 calculates a similarity score between the detected face embeddings and each embedding in the database of registered faces (for example, the database of registered faces 185 in Figure 1). In some embodiments, instead of calculating a similarity score between the detected face embeddings and each embedding in the database of registered faces, the embedding post-processing module 340 calculates a similarity score between the detected face embeddings and one or more representative embeddings from each cluster of embeddings in the database of registered faces. The embedding post-processing module 340 uses various techniques, such as nearest neighbor, clustering centroid, K nearest neighbor (KNN), and support vector machines (SVM), to identify one or more representative embeddings for each cluster of embeddings in the database of registered faces.

[0033] The embedded post-processing module 340 compares each similarity score to a user-specific similarity threshold associated with the registered user. If the similarity score exceeds the user-specific similarity threshold associated with the registered user, the embedded post-processing module 340 indicates that the detected face is likely to be the registered user. Otherwise, the detected face is likely not to be the registered user. In some embodiments, once a match is determined, other techniques, such as the OpenCV library, may be used to track the detected face and identity until the detected face leaves the frame, providing faster and more efficient tracking of the detected face.

[0034] Figure 4 is a flowchart of Method 400 for embedded facial recognition in an implementation of the present disclosure. Method 400 can be carried out by processing logic that may include hardware (e.g., processing devices, circuits, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions that operate on or are executed on the processing device), or a combination thereof. In some embodiments, Method 400 is carried out by an embedded facial recognition system 100.

[0035] In operation 410, the processing logic performs sequential face detection on a series of images from the image acquisition device. As previously described, the processing logic receives images sequentially as input from the image acquisition device. In some embodiments, the processing logic resizes the images to match the input dimensions of a face detection model that scans the input to generate potential bounding boxes. Each potential bounding box is assigned a confidence score indicating the likelihood of a bounding box containing a face. If the confidence score exceeds a predetermined confidence threshold (i.e., a face is detected), the coordinates of the bounding box are used to generate a face image containing the detected face. The processing logic identifies key face landmarks within the face image.

[0036] In operation 420, in response to detecting a face in a series of images, the processing logic aligns the face in the images. In some embodiments, the faces in the face images are not aligned with a reference face. Therefore, the processing logic uses key face landmarks in the face images and key face landmarks in the reference face to determine the parameters of a homogeneous transformation. After determining the parameters of the homogeneous transformation, the processing logic applies the homogeneous transformation to each pixel of the face images with the determined parameters to align the faces with the reference face. In other words, the faces in the face images are made to fit the size, orientation, and position of the reference face.

[0037] In operation 430, the processing logic generates face embeddings using the aligned faces. As previously mentioned, the face images of the aligned faces are passed through a pre-trained neural network model trained to process input images and output face embeddings. The trained neural network model ideally generates embeddings of different images of the same person that cluster together in a high-dimensional space, while also generating embeddings of clearly separated different individuals. In some embodiments, the processing logic may reduce the dimensionality of the face embeddings.

[0038] In operation 440, the processing logic identifies the individual belonging to the aligned face. For each face embedding in the registered face database, the processing logic calculates a similarity score between the generated face embedding and the respective face embedding, and then compares the similarity score to the similarity threshold of the respective face embedding to determine whether the registered user associated with the respective face embedding matches the generated face embedding. If so, the individual associated with the generated face embedding is the registered user associated with the respective face embedding. Otherwise, the individual associated with the generated face embedding is not the registered user associated with the respective face embedding.

[0039] Depending on the embodiment, the processing logic may identify a representative face embedding for each cluster of face embeddings in the database of registered faces. For each representative face embedding, the processing logic calculates a similarity score between the generated face embedding and the respective representative face embedding, and then compares the similarity score to the similarity threshold of the representative face embedding to determine whether the registered user associated with the respective representative face embedding matches the generated face embedding. If so, the individual associated with the generated face embedding is the registered user associated with the respective representative face embedding. Otherwise, the individual associated with the generated face embedding is not the registered user associated with the respective representative face embedding.

[0040] The similarity threshold for each face embedding or representative face embedding in the registered face database is assigned based on the user corresponding to the face embedding or representative face embedding of the registered face.

[0041] Figure 5 is a block diagram showing an exemplary computer system 500 in an implementation of the present disclosure. The computer system 500 can operate as a server or endpoint machine in an endpoint server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a television, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) that specifies the actions that the machine should take. Furthermore, although only a single machine is shown, the term “machine” shall also be interpreted to include any set of machines individually or collectively executing one or more sets of instructions to perform any one or more of the actions described herein.

[0042] The computer system 500 includes a processing device (processor) 502, main memory 504 (for example, read-only memory (ROM), flash memory, synchronous DRAM (SDRAM), double data-rate SDRAM (DDR SDRAM), and RAM bus DRAM (RDRAM), which are dynamic random access memories (DRAM)), static memory 506 (for example, flash memory, static random access memory (SRAM), etc.), and a data storage device 518, which communicate with each other via a bus 540.

[0043] The processor (processing device) 502 represents one or more general-purpose processing devices, such as a microprocessor or a central processing unit. More specifically, the processor 502 may be a composite instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets or combinations of instruction sets. The processor 502 may also be one or more dedicated processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. The processor 502 may include processing logic 522 used to perform the operations described herein. The processor 502 is configured to execute instructions 505 for performing the operations described herein.

[0044] The computer system 500 may further include a network interface device 508. The computer system 500 may also include a video display unit 510 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an input device 512 (e.g., a keyboard, and an alphanumeric keyboard, a motion-sensing input device, a touchscreen), a cursor control device 514 (e.g., a mouse), and a signal generating device 520 (e.g., a speaker).

[0045] The data storage device 518 may include a non-temporary machine-readable storage medium 524 (which is also a computer-readable storage medium) storing one or more instruction sets 526 that embody any one or more of the methods or functions described herein. The instructions may also reside, all or at least partially, in the main memory 504 and / or processor 502 while the computer system 500 is executing the instructions, and the main memory 504 and processor 502 also constitute machine-readable storage media. The instructions may also be transmitted or received over the network 530 via the network interface device 508.

[0046] Although computer-readable storage medium 524 (machine-readable storage medium) is shown as a single medium in exemplary implementations, the terms “computer-readable storage medium” and “machine-readable storage medium” should be interpreted to include a single or multiple mediums that store one or more instruction sets (e.g., a centralized or distributed database and / or associated caches and servers). The terms “computer-readable storage medium” and “machine-readable storage medium” should also be interpreted to include any medium that can store, encode, or carry instruction sets for machine execution, causing a machine to perform any one or more of the methods of this disclosure. Accordingly, the terms “computer-readable storage medium” and “machine-readable storage medium” should be interpreted to include, but are not limited to, solid-state memory, optical media, and magnetic media.

[0047] Throughout this specification, when we refer to “one implementation,” “one embodiment,” “implementation,” or “embodiment,” we mean that the particular features, structure, or characteristics described in relation to that implementation and / or embodiment are included in at least one implementation and / or embodiment. Thus, when the phrases “in one implementation” or “in an implementation” are used in various places throughout this specification, they may, though not necessarily, refer to the same implementation depending on the context. Furthermore, particular features, structure, or characteristics can be combined in any suitable way in one or more implementations.

[0048] To the extent that the terms “comprises,” “comprising,” “includes,” “including,” “has,” and “contains,” variations thereof, and other similar terms are used in either the detailed description or the claims, these terms are intended to be as comprehensive as the term “comprising” as open transitional terms that do not exclude additions or other elements.

[0049] As used in this application, terms such as “block,” “layer,” “component,” “module,” and “system” are intended to generally refer to computer-related entities that relate to any of the following: hardware (e.g., circuits), software, a combination of hardware and software, or an entity relating to an operating machine having one or more specific functions. For example, components may be processes, processors, objects, executable files, execution threads, programs, and / or computers running on a processor (e.g., a digital signal processor), but are not limited to these. Exemplarily, both an application running on a controller and a controller may be components. One or more components may reside within a process and / or execution thread, and components may be localized on one computer and / or distributed across two or more computers. Furthermore, “device” may take the form of specially designed hardware, general-purpose hardware specialized by the execution of software that enables the hardware to perform a specific function (e.g., generate points of interest and / or descriptors), software on a computer-readable medium, or a combination thereof.

[0050] The aforementioned systems, circuits, modules, etc., are described in relation to the interactions between several components and / or blocks. It can be understood that such systems, circuits, components, blocks, etc., may include those components or designated subcomponents, a portion of a designated component or subcomponent, and / or additional components, according to the various sortings and combinations described herein. Subcomponents may also be implemented as components that are communicatively coupled to other components, rather than being contained within (hierarchically) a parent component. Furthermore, it should be noted that one or more components may be combined into a single component that provides an aggregation function, or they may be divided into several separate subcomponents, and one or more intermediate layers, such as a management layer, may be provided to communicatively couple such subcomponents in order to provide integrated functionality. Any component described herein may also interact with one or more other components that are not specifically described herein but are known to those skilled in the art.

[0051] Furthermore, the terms “example” or “exemplary” are used herein to mean that an example, instance, or illustration is provided. An embodiment or design described herein as “exemplary” should not necessarily be interpreted as being preferable or advantageous to other embodiments or designs. Rather, the use of the terms “example” or “exemplary” is intended to provide a concrete representation of the concept. Where used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or evident from the context, “X uses A or B” is intended to mean any of the natural inclusive sortings. That is, if X uses A, if X uses B, or if X uses both A and B, then “X uses A or B” is satisfied under any of the aforementioned examples. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be interpreted as meaning “one or more” unless otherwise specified or evident from the context.

[0052] Finally, the implementations described herein include a collection of data describing the user and / or the user's activities. In one implementation, such data is collected only when the user consents to its collection. In some implementations, the user is prompted to explicitly authorize data collection. Furthermore, the user can opt in or opt out of participating in such data collection activities. In one implementation, the collected data is anonymized before any analysis is performed to obtain any statistical patterns so that the user's identity cannot be determined from the collected data.

Claims

1. Receiving captured images from an image acquisition device, Extracting facial images from the aforementioned captured images, Aligning the extracted facial image to a reference facial model, Using a machine learning model, generate face embeddings based on the aligned extracted face images, Based on the generated face embeddings and the existing face embedding database, the individual associated with the face embedding is identified. A method that includes this.

2. Aligning the extracted facial image to the reference facial model is, Identifying a landmark set from the extracted facial image, Obtaining a subset of the aforementioned landmark set from the aforementioned landmark set, Using the subset, the reference face model, and the homogeneous transformation, the parameters of the homogeneous transformation are determined. The homogeneous transformation is applied to each pixel of the extracted face image along with the determined parameters. The method according to claim 1, including the method described in claim 1.

3. The method according to claim 2, wherein the subset includes the left eye of the face image, the right eye of the face image, and the center of the lips of the face image.

4. Generating the aforementioned face embedding means The machine learning model is input to the aligned extracted face image in order to output the face embedding, To reduce the dimension of the aforementioned face implantation and The method according to claim 1, including the method described in claim 1.

5. Determining the individual associated with the face embedding based on the generated face embedding and the existing face embedding database is: For each existing face embedding in the aforementioned existing face embedding database, a similarity score is calculated between the generated face embedding and the respective existing face embedding. The calculated similarity score between the generated face embedding and each of the existing face embeddings is compared with the similarity score threshold of each of the existing face embeddings. The method according to claim 1, including the method described in claim 1.

6. Determining the individual associated with the face embedding based on the generated face embedding and the existing face embedding database is: For each cluster of existing face embeddings in the aforementioned database of existing face embeddings, obtain a representative face embedding from each cluster of existing face embeddings. For each representative face embedding, a similarity score is calculated between the generated face embedding and the representative face embedding. The calculated similarity score between the generated face embedding and the representative face embedding is compared with the similarity score threshold of the representative face embedding. The method according to claim 1, including the method described in claim 1.

7. The method according to claim 5, wherein a unique similarity threshold is assigned to each cluster of existing face embeddings in the database of existing face embeddings corresponding to an individual.

8. The method according to claim 1, wherein the existing face embedding database includes embeddings of each user in various orientations.

9. In response to execution by the processing device, the processing device is instructed to: The process of receiving captured images from an image acquisition device, The operation of extracting a face image from the aforementioned captured image, The operation of aligning the extracted facial image with a reference facial model, The operation involves using a machine learning model to generate a face embedding based on the aligned extracted face image, Based on the generated face embeddings and the existing face embedding database, an action is taken to identify the individual associated with the face embedding. A non-temporary computer-readable medium containing instructions that cause an action to be performed.

10. The operation of aligning the extracted facial image with the reference facial model is as follows: The operation of identifying a landmark set from the extracted facial image, The operation of obtaining a subset of the landmark set from the aforementioned landmark set, The operation of determining the parameters of the homogeneous transformation using the subset, the reference face model, and the homogeneous transformation, The operation of applying the homogeneous transformation along with the determined parameters to each pixel of the extracted face image. A non-temporary computer-readable medium according to claim 9, including the following:

11. The non-temporary computer-readable medium according to claim 10, wherein the subset includes the left eye of the face image, the right eye of the face image, and the center of the lips of the face image.

12. The operation to generate the aforementioned face embedding is, The operation of inputting the aligned extracted face image to the machine learning model in order to output the face embedding, The operation to reduce the dimension of the aforementioned face embedding and A non-temporary computer-readable medium according to claim 9, including the following:

13. The operation to determine the individual associated with the face embedding, based on the generated face embedding and the existing face embedding database, For each existing face embedding in the aforementioned database of existing face embeddings, the operation of calculating a similarity score between the generated face embedding and each existing face embedding, The operation involves comparing the calculated similarity score between the generated face embedding and each of the existing face embeddings with the similarity score threshold of each of the existing face embeddings. A non-temporary computer-readable medium according to claim 9, including the following:

14. The operation to determine the individual associated with the face embedding, based on the generated face embedding and the existing face embedding database, For each cluster of existing face embeddings in the aforementioned database of existing face embeddings, the operation involves obtaining a representative face embedding from each cluster of existing face embeddings. For each representative face embedding, the operation involves calculating a similarity score between the generated face embedding and the representative face embedding, The operation involves comparing the calculated similarity score between the generated face embedding and the representative face embedding with the similarity score threshold of the representative face embedding. A non-temporary computer-readable medium according to claim 9, including the following:

15. The non-temporary computer-readable medium according to claim 13, wherein each cluster of existing face embeddings in the database of existing face embeddings corresponding to an individual is assigned a unique similarity threshold.

16. The non-temporary computer-readable medium according to claim 9, wherein the existing database of face embeddings includes embeddings of each user in various orientations.

17. Image acquisition device and A processing device coupled to the image acquisition device, wherein the processing device is The operation of receiving the captured image from the aforementioned image acquisition device, The operation of extracting a face image from the aforementioned captured image, The operation of aligning the extracted facial image with a reference facial model, The operation involves using a machine learning model to generate a face embedding based on the aligned extracted face image, Based on the generated face embeddings and the existing face embedding database, an action is taken to identify the individual associated with the face embedding. A processing device that performs operations including A system that includes this.

18. The operation of aligning the extracted facial image with the reference facial model is as follows: The operation of identifying a landmark set from the extracted facial image, The operation of obtaining a subset of the landmark set from the aforementioned landmark set, The operation of determining the parameters of the homogeneous transformation using the subset, the reference face model, and the homogeneous transformation, The operation of applying the homogeneous transformation along with the determined parameters to each pixel of the extracted face image. The system according to claim 17, including the system described in claim 17.

19. The operation to generate the aforementioned face embedding is, The operation of inputting the aligned extracted face image to the machine learning model in order to output the face embedding, The operation to reduce the dimension of the aforementioned face embedding and The system according to claim 17, including the system described in claim 17.

20. The operation to determine the individual associated with the face embedding, based on the generated face embedding and the existing face embedding database, For each cluster of existing face embeddings in the aforementioned database of existing face embeddings, the operation involves obtaining a representative face embedding from each cluster of existing face embeddings. For each representative face embedding, the operation involves calculating a similarity score between the generated face embedding and the representative face embedding, The operation involves comparing the calculated similarity score between the generated face embedding and the representative face embedding with the similarity score threshold of the representative face embedding. The system according to claim 17, including the system described in claim 17.

Images