Method and system for dynamic and incremental face recognition

Abstract

In one embodiment, a method includes identifying a facial image from an image of a scene. The method then determines a context associated with the facial image based on a context model. The method then identifies a person from a database based on the context and a facial recognition model associated with the facial image.

Description

Technical Field

[0001] The present disclosure generally relates to facial recognition. Background Art

[0002] Existing facial recognition systems may typically require initial annotation of the user's face, and may suffer from cold start problems without such initial annotation. Furthermore, the user's facial appearance may vary depending on different situations, such as lighting changes, makeup, wearing / not wearing glasses, a new hairstyle, or simply aging. As a result, the initial annotation may quickly become insufficient to reliably identify the user, and existing users may be incorrectly identified as new faces. Furthermore, for scenarios such as implementation within a home security application, the application may need to generate alerts for unfamiliar faces, and insufficient identification of existing users may result in false alerts. Summary of the invention BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 An example facial recognition system is shown.

[0004] Figure 2 An example flow chart for dynamic facial model collection is shown.

[0005] Figure 3 An example flow chart of dynamic facial model construction is shown.

[0006] Figure 4 An example flow diagram for comparing facial samples to identify landmarks corresponding to the facial samples is shown.

[0007] Figure 5 An example flow chart for querying users with face samples is shown.

[0008] Figure 6 An example user interaction with a facial sample is shown.

[0009] Figure 7 An example method of identifying a person through facial recognition and context is shown.

[0010] Figure 8 An example computer system is shown. DETAILED DESCRIPTION

[0011] The electronic device of the present disclosure provides the ability to capture and organize personal identities associated with facial images. For example, the electronic device may include a camera that uses computer vision to analyze streaming data from a video captured by the camera. Using a camera and computer vision, the device may be able to dynamically collect facial images and gradually build a facial recognition model. The device can build a facial recognition model without any initial facial image samples and / or data. The device can be connected to a server (e.g., a cloud) and can dynamically organize facial recognition models and associate them across multiple devices, locations, users, or other contexts. The device can implement active learning to reduce the need for initial and incremental user tagging of facial data. Users interfacing with the device can use multimodal interactions, such as voice, touch, or gestures.

[0012] In certain embodiments, a facial recognition model may associate a particular face with an identity of that face. A facial recognition model may be based on a set of facial samples, such as facial images of a particular face (e.g., from different angles or in different settings). A facial recognition model may be based on user-provided information, such as a user-provided tag for an identity associated with a particular face (e.g., "Jim" or "Postman"). As described below, the identity and / or facial recognition model may be temporary or persistent data in a database. An existing facial recognition model may be updated with additional information, such as with additional facial samples of an individual, additional user-provided or machine-generated identity tags (e.g., last name), and / or contextual information.

[0013] When building a facial recognition model, the electronic device can determine the context associated with the facial image samples captured by the camera. In addition, the device can identify various semantic levels of information about scenes and elements within the scenes. For example, in a captured sequence of images, the device can determine that two people captured in the image are playing with a ball, the ball is a soccer ball, the image was captured in the afternoon, and so on. The various semantic levels of information about the scene captured within the image can provide further information corresponding to the context in which the image was captured. In addition, the information can also provide the context in which the facial image was captured. In a particular embodiment, the scene includes scene parameters, wherein the scene parameters include one or more of the following: time, date, one or more elements or activities captured within the image.

[0014] The electronic device can establish a character model and / or a user context model for the identified person in the scene. The device can build an overall representative structure of data representing a specific person in the scene and across multiple scenes. The device can establish a time-varying co-occurrence graph based on these character models / user context models representing a single character / individual. The time-varying co-occurrence graph can indicate the relationship between two or more individuals. The device can use these user context models to more accurately identify the people in the scene. For example, if two children usually play in the living room area in the afternoon, if certain items are detected, such as toys previously detected in previous images captured with the two children, the device can more accurately mark the two people identified in the image captured in the afternoon as two children. Although the present disclosure describes an electronic device that performs the above functions, the present disclosure contemplates that in some embodiments, such functions can be performed by multiple devices, such as devices in a server-client architecture.

[0015] Figure 1 An example facial recognition system 100 is shown. The facial recognition system 100 may include a camera device 102, a facial backend service 104, a user context database 106, an assistant service 108, and a natural language understanding (NLU) backend service 114. The camera device 102 may include a face detector 110 and a gesture detector 112. The camera device 102 may be connected to a network to wirelessly communicate with the facial backend service 104, the user context database 106, and / or the assistant service 108. In particular embodiments, one or more of the components 102, 104, 106, 108, 114 of the facial recognition system 100 may be combined with other components 102, 104, 106, 108, 114 of the facial recognition system 100. By way of example and not limitation, the user context database 106 may be stored on the camera device 102. Although only a certain number of components 102, 104, 106, 108, 114 of the facial recognition system 100 are shown, there may be any number and / or combination of components 102, 104, 106, 108, 114. By way of example and not limitation, there may be two camera devices 102 in the facial recognition system 100. In certain embodiments, each of the components 102, 104, 106, 108, 114 may be wirelessly coupled to each other via a network.

[0016] In certain embodiments, the camera device 102 may be an electronic device including hardware, software, or embedded logic components or a combination of two or more such components, and capable of performing appropriate functions implemented or supported by the camera device 102. By way of example and not limitation, the camera device 102 may include a computer system, such as a desktop computer, a notebook or laptop computer, a netbook, a tablet computer, an e-book reader, a GPS device, a camera, a personal digital assistant (PDA), a handheld electronic device, a cellular phone, a smart phone, a smart speaker, other suitable electronic devices, a camera system, or any suitable combination thereof. In certain embodiments, the camera device 102 may be a computer system coupled to a camera. The camera device 102 may capture streaming data. By way of example and not limitation, the camera device 102 may capture multiple images in a sequence, such as a video. The camera device 102 may use a facial detector 110 to identify a facial image within a captured image. In certain embodiments, the camera device 102 may access a facial image from the streaming data. In certain embodiments, the camera device 102 may use a facial recognition model (which may be embodied by the facial detector 110) to identify a facial image from an image of a scene.

[0017] The camera device 102 may use a pose detector 112 to determine a pose of a facial image in a captured image. By way of example and not limitation, the camera device 102 may capture an image and identify a face in the image at a particular position and orientation. In certain embodiments, the camera device 102 may not include a pose detector 112, and may use another device to detect poses or not detect poses at all. A user may interface with the camera device 102. In certain embodiments, the camera device 102 may include a user interface, such as a touch screen display. The camera device 102 may send a facial image to the facial backend service 104. The camera device 102 may determine a context associated with the captured image as described herein. By way of example and not limitation, the camera device 102 may analyze the image to identify an activity being performed by a person identified within the image. The camera device 102 may upload context and user data to a user context database 106. The camera device 102 may also upload interaction data to the user context database 106. The interaction data may indicate a determination of two or more persons identified in a captured image. By way of example and not limitation, two persons live together in a house and typically watch television together. To accurately link the facial image to the context, the camera device 102 may associate the facial image with the facial identifier received from the facial backend service 104. The camera device 102 may then upload the context with the facial identifier to the user context database to create an association between the person identified by the facial identifier and the specific context (e.g., the user typically goes home at 5 p.m. every weekday).

[0018] The camera device 102 may request user data from the user context database 106. The user data may include one or more contexts associated with a particular individual. By way of example and not limitation, in all images captured by the camera device 102, there may be only one person playing a musical instrument. The camera device 102 may send the user data to the facial backend service 104. The user data may be associated with a corresponding facial recognition model. In a particular embodiment, the camera device 102 may use the user data and facial image received from the user context database 106 to identify the person in the captured image. By way of example and not limitation, if the only musician in the family is Sam identified from the user data, then based on the facial image (which may be determined by the facial recognition model to be Sam with a 70% confidence / match score) and the user data, the person playing the musical instrument in the captured image may be identified as Sam. The difference of the incomplete match with the facial recognition model may be due to one or more objects blocking the person's face and / or changes in the person's face. The camera device 102 may send a query to the assistant service 108. The assistant service 108 may return a response to the query.

[0019] In certain embodiments, the facial backend service 104 may include one or more types of servers, one or more data stores, one or more interfaces (including but not limited to APIs), one or more web services, one or more content sources, one or more networks, or any other suitable components with which, for example, a server may communicate. The facial backend service 104 may be used to identify a facial identifier corresponding to a facial image captured by the camera device 102. In certain embodiments, the facial backend service 104 may include a plurality of facial recognition models. Each facial recognition model may be used to identify a particular person based on facial image data. In certain embodiments, the facial backend service 104 may determine a facial match score by applying each of the plurality of facial recognition models to the facial image.

[0020] In a particular embodiment, the facial backend service 104 may determine the minimum distance between the facial image and one or more facial recognition models and select the facial recognition model with the minimum distance for use. After determining the facial match score between the facial image captured by the camera device 102 and the one or more facial recognition models, the facial backend service 104 may select one of the facial recognition models based on the facial match score. The facial backend service may identify a facial identifier associated with the selected facial recognition model. The facial backend service 104 may send back a facial identifier corresponding to the selected facial recognition model and the facial image received from the camera device 102. The facial backend service 104 may also request user confirmation from the camera device 102. The user confirmation may be confirmation of the accurate marking of the facial image. As an example and not limitation, the user confirmation may be confirmation that an unfamiliar person is captured in the image. As another example and not limitation, the user confirmation may be confirmation that a familiar person is captured in the image. The facial backend service 104 may receive facial confirmation data from the user via the camera device 102 in response to the request.

[0021] In certain embodiments, user context database 106 may include one or more types of servers, one or more data storage devices, one or more interfaces (including but not limited to APIs), one or more network services, one or more content sources, one or more networks, or any other suitable components with which a server can communicate. User context database 106 may include multiple user data for each identified individual. User data may include context information associated with an individual as described herein. User context database 106 may be updated by camera device 102 and / or multiple camera devices 102 through multiple uploads.

[0022] In certain embodiments, the assistant service 108 may include one or more types of servers, one or more data stores, one or more interfaces (including but not limited to APIs), one or more network services, one or more content sources, one or more networks, or any other suitable components with which a server, for example, can communicate. The assistant service 108 may communicate with the NLU backend service 114. When the assistant service 108 receives a query such as a user input from the camera device 102, the assistant service 108 may send the user input to the NLU backend service 114 to identify the query associated with the user input. The assistant service 108 may query other databases to generate a response to the query and return the response to the camera device 102. The camera device may present information associated with the response to the user. As an example and not limitation, if the user camera device 102 sends a user input to the assistant service 108, "What's the weather like today?", then the assistant service 108 may identify the query associated with the user input through the NLU backend service 114 and generate a response to send to the camera device 102 for display to the user.

[0023] In a particular embodiment, the facial recognition system 100 may receive input from a streaming source, such as receiving video frames in a continuous manner. The video frames may be received by the camera device 102. In a particular embodiment, a facial sample pool is maintained by the facial recognition system 100 to find a match with an input facial image. A classifier may be trained based on the facial sample pool to classify faces. The facial recognition system 100 may initially start with an empty facial sample pool. The facial recognition system 100 may determine which new facial image to select from the video frame to add to the facial sample pool. To make the determination, the facial recognition system 100 may determine that the matching error exceeds a threshold relative to an existing facial recognition model and / or that the existing facial recognition model fails to recognize an individual associated with the facial image. In a particular embodiment, the facial recognition system 100 may generate a new facial recognition model using a new facial image that has a matching error exceeding a threshold relative to an existing facial recognition model. In a particular embodiment, the facial recognition system 100 may use a clustering algorithm that may construct a facial sample pool into multiple groups of facial samples for multiple facial identities. Each group may correspond to a different facial identifier. As an example and not limitation, for a facial sample of a man with a beard, the facial recognition system 100 may generate a group or cluster of facial samples of men with a beard under a first facial identifier. The facial recognition system 100 may then obtain a facial sample of a woman with long hair and generate a group or cluster of facial samples of women with long hair under a second facial identifier.

[0024] In certain embodiments, the facial recognition system 100 may be coupled or wirelessly coupled to a home security system. The home security system and / or the facial recognition system 100 may be coupled to a plurality of devices to capture streaming data. As described herein, the streaming data may be sent to the facial recognition system 100 for analysis. The home security system may send an unfamiliar face notification to the user in response to identifying a person as an unfamiliar person. To this end, the facial recognition system 100 may access facial images from the streaming data and apply a plurality of facial recognition models to the facial images to determine whether there is a match. In response to determining that there is no match, the facial recognition system 100 may send an alert to the home security system, which will send an unfamiliar face notification to the user. In certain embodiments, the facial recognition system 100 and / or the home security system may identify several common occurrences of unfamiliar faces that may need to be excluded. As an example, and not by way of limitation, a mailman may come every day and may not need to be labeled as an unfamiliar face. In certain embodiments, the facial recognition system 100 may incorporate detection frequency as another triggering mechanism. As an example and not by way of limitation, the identification of unfamiliar people only within a threshold time period may trigger an unfamiliar face notification to the user. As another example, but not by way of limitation, if a mail carrier frequently passes by the front door within a short period of time, then the mail carrier may not trigger an unfamiliar face notification.

[0025] Figure 2 An example flow chart of dynamic facial model collection is shown. Process 200 may start at step 202, where a facial recognition system may collect facial images and encode the collected facial images. To collect facial images, the facial recognition system may use a camera to capture multiple images, and apply a facial detector on the multiple images to identify facial images within the multiple images. These facial images may be added to a facial sample pool. At step 204, the facial recognition system may incrementally add facial models for each facial image. To do this, the facial recognition system may first match each facial image with an existing facial sample pool. The facial recognition system may use the distance to the facial sample cluster to determine whether each new facial image belongs to one of the existing facial identifiers. The facial recognition system may assign a temporary facial identifier to each facial image that does not belong to one of the facial sample clusters with an existing facial identifier.

[0026] At step 206, the facial recognition system may cluster together the facial images with the temporary facial identifiers and add the facial identifiers to the facial image clusters. As an example and not limitation, the facial recognition system may identify a woman with short hair among multiple facial images in the facial sample pool, cluster the facial images together, and add the facial identifiers to the clusters. At step 208, the facial recognition system may match facial images from other devices. As an example and not limitation, if a bearded person is identified by one device in the living room area of ​​his home, and a bearded person is identified by another device in the kitchen area of ​​his home, the facial recognition system may determine that the two instances of the bearded person identified by the two independent devices may be the same person, and match the facial images from each device. For example, by using a backend service and / or database, facial recognition models from multiple devices may be cross-checked to obtain the best results, such as accurately identifying a person. At step 210, the facial recognition system may establish a device-specific face list, a location-specific face list, and a user-specific face list. A device-specific face list may be a list of facial identifiers associated with a device. A location-specific face list may be associated with multiple devices and is a list of facial identifiers associated with a location. That is, the facial identifier is associated with a facial image captured by a device at the location within the location.A user-specific face list may include facial images associated with a particular user or a particular facial identifier.

[0027] Figure 3 An example flow chart for dynamic facial model construction is shown. Process 300 may begin at step 302, where a facial recognition system may collect facial images. At step 304, the facial recognition system may incrementally add facial markers to the facial images. At step 306, the facial recognition system may encode each facial image. At step 308, the facial recognition system may match the encoded facial images. The facial recognition system may match each detected facial image to an existing pool of facial samples. The facial recognition system may use the distance to the facial sample clusters to determine whether each encoded facial image belongs to one of the existing facial identifiers. The facial recognition system may assign a temporary facial identifier to each facial image that does not belong to one of the facial sample clusters with an existing facial identifier. At step 310, the facial recognition system may cross-correlate the facial images collected across multiple devices, locations, and users. Through cross-correlation, the facial identifiers may be combined to identify a specific individual through the facial image.

[0028] In a particular embodiment, an active learning algorithm that can be implemented by a facial recognition system can select multiple facial samples from a facial sample pool to query a user based on a distance from a facial sample cluster. As an example and not limitation, if a new facial image is received and the active learning algorithm determines that the distance between the new facial image and the facial sample cluster is greater than a threshold distance, the active learning algorithm can select the new facial image to query the user. The query to the user can be a request to confirm whether the new facial image belongs to the same facial sample cluster. The user can interact with the query to confirm that the new facial image belongs to the same cluster, or that it is separated from the cluster. In a particular embodiment, if the user confirms that the new facial image does not belong to an existing facial sample cluster, the new facial image becomes a sample of an unfamiliar facial cluster. The detection of unfamiliar faces can be based on a match with an existing facial recognition model, or depends on user feedback. In a particular embodiment, a facial recognition model can be associated with a set of facial samples.

[0029] Figure 4An example flow chart for comparing facial samples to identify tags corresponding to the facial samples is shown. Process 400 may begin by receiving an input facial image 402. As described herein, a camera device may capture an image and detect facial image 402. In certain embodiments, an active learning algorithm may be applied to facial image 402. Each input facial image 402 undergoes a comparison process 403 with a pool of facial samples 404a-404n. As an example and not limitation, the input facial image 402 may be a user John with a beard. Facial sample tag 2 404b, facial sample tag 3 404c, and facial sample tag N 404n may be John with a beard and wearing different clothing. The three facial samples 404b, 404c, and 404n may have the same facial identifier K 406k corresponding to John with a beard. The facial sample marker 1404a may be John without a beard associated with a different facial identifier 1406a, because the facial sample marker 1404a may exceed the threshold distance of the cluster of images 404b-404n corresponding to the facial identifier K 406k. In certain embodiments, the facial recognition system may ask the user to perform a merge function 408 to combine the two facial sample clusters. Since each image 404 is of John, the user may approve the request and confirm that the images 404 are of the same person. In certain embodiments, the facial recognition system may incrementally build a facial recognition model for the same user. This may be based on matching a new facial image with an existing facial recognition model, or based on user feedback. If the user confirms that the new facial image does belong to an existing facial sample cluster, the new facial image becomes an additional facial sample to the existing facial sample cluster. This may help improve the recognition of the same person, because as facial appearance changes over time, more facial samples are added to the facial sample cluster associated with the person, thereby improving the correct recognition of the person as the person's appearance changes.

[0030] Figure 5 An example flow chart of querying a user with a facial sample is shown. In certain embodiments, the facial recognition system may use an active learning algorithm for process 500 to begin matching an input facial image with existing facial samples in an internal facial sample pool 502. In certain embodiments, facial sample pool 502 may include a plurality of facial samples from previous images captured by the facial recognition system with their respective facial landmarks (e.g., Figure 4As an example and not limitation, in a home setting, the facial sample pool 502 may include facial images of family members and guests visiting the home. However, if the facial sample pool 502 is empty, for example, there are no known facial recognition models or facial samples initially, then the input facial image becomes a facial sample. In a particular embodiment, the active learning algorithm can learn the correspondence between the input facial image and the existing facial samples in the internal facial sample pool 502 by modeling the context associated with the facial image. Each facial sample 502 of the pool may be associated with a context. The context may include multiple separate presence modes. An individual presence mode may include visual consistency, where an individual instance in the input facial image should be visually similar to an individual instance with the same individual label l j The individual presence mode includes co-presence, where individual l i and individual l j The individual presence pattern may include a sequential pattern, in which the individual l i Appearing after the previous frame a, individual l i As an example and not limitation, the sequential order may identify when an individual appears in an image and another individual typically appears after the individual. For example, when a boy walks home from school, his sister typically follows the boy in a series of images. An individual presence pattern may include presence in a scene, where in scene S, individual l i Appearance. In certain embodiments, the active learning algorithm may use a deep learning module to combine scenes to better predict the presence pattern of individuals. In certain embodiments, the visual consistency pattern may be associated with a matching distance 506. The matching distance 506 may be associated with the original visual match of each facial image in the facial sample pool 502 and the individual in the database (e.g., a database of individuals associated with context). In certain embodiments, the social connection 508 may be associated with the co-existence pattern. The social connection 508 may be associated with a recognition and matching process that recognizes and matches the context associated with the facial image from the facial sample pool 502 with the context in the database (e.g., a user context database). In certain embodiments, the facial tracking 504 may be associated with a sequential pattern. As an example and not limitation, the facial tracking 504 may analyze a series of images and recognize that an individual within one image may be the same individual in consecutive images (if the facial image is in the same or similar position within the image). In certain embodiments, the active learning algorithm may use one or more of the facial tracking 504, the matching distance 506, the social connection 508, or the random sampling 510 to determine whether to query the user for a facial sample 516.

[0031] In a particular embodiment, if the minimum matching distance 506 between the input facial image and the existing facial samples exceeds a threshold, the input facial image may become a new facial sample in the facial sample pool 502 and be assigned a new temporary facial identifier. However, if the matching distance 506 between the input facial image and the existing facial samples 502 is within a distance, the input facial image may be placed in a candidate pool of facial samples for active learning. The active learning algorithm may query the user based on feature selection facial samples 516, including but not limited to: the matching distance 506 between the facial samples in the candidate pool and the facial samples in the internal facial recognition model sample pool 502 and the consistency of the facial tracking 504 associated with some facial samples. Each facial tracking 504 may be a continuous tracking of a single detected facial bounding box through time and space. In a particular embodiment, social connections 508 between socially connected people or people appearing in social events together influence the weight of sampling. By way of example and not limitation, if a married couple that has previously been identified by the facial recognition system is attending an event together, the active learning algorithm may increase the weight of the sampled facial images corresponding to the married couple because the facial recognition system may be able to accurately identify the two people as corresponding to the first facial identifier and the second facial identifier that have previously been socially connected 508. In certain embodiments, the active learning algorithm may randomly sample 510 from different clusters of facial samples corresponding to two different facial identifiers. The random sampling 510 may be used to identify clusters to be merged. By way of example and not limitation, if a person has a beard and is now identified as a different individual in the facial sample pool 502 than if the person did not have a beard, the random sampling 510 will increase the likelihood of combining the two clusters of facial samples 502 to represent one individual. In certain embodiments, the facial recognition system may set a certain percentage of facial images to randomly sample users as query samples 516. The facial images to be randomly sampled may be from a user that has been identified using J Comb 4, and the facial images were not scored high enough to query the user. By way of example and not limitation, the facial recognition system may determine to send the user 5% (or another set percentage) of random facial images that may not score high enough to send the user a query. In certain embodiments, in some scenarios, if the user identifies a random facial image from the database that was originally labeled as User 4 as User 1, the facial recognition system may combine facial images associated with User 1 and User 4.

[0032] In certain embodiments, the active learning algorithm may focus on two criteria when selecting a query sample 516 to send to a user. In certain embodiments, the criteria may include an information content measure 512 and a representativeness measure 514. In certain embodiments, the context may be determined based on an assessment of the information content and representativeness of the facial image. After determining which query sample 516 to send to the user, the facial recognition system may send the query sample 516, requesting confirmation of whether the query sample 516 corresponds to a facial recognition model. In certain embodiments, the information content measure 512 may measure the ability of the instance sample to reduce the uncertainty of the statistical model. The information content measure 512 may be associated with a threshold information content to determine whether the facial image in the image can be identified from the database. In one implementation, the residual-based algorithm may select the unlabeled instance x that is closest to the decision boundary. s , which means that the objective function has a relatively high score regardless of its class label l s How. For example, face instance x j The margin can be calculated as:

[0033]

[0034] Assumptions:

[0035]

[0036] in:

[0037] J Comb (L i |x j , H i , S i )=J Vis (L i |x j )+J Context (t i |H i , S i )

[0038] J Comb J can be a combined visual appearance model Vis Cost Function and Context Model J Context The residual equation can be used to present a query sample 516 to a user that may be ambiguous, such as two individuals that may look similar. If the scores of the facial images for two individuals in the database are fairly close, the facial recognition system can send the query sample 516 to determine which of the two individuals (or the other user if the facial recognition system is incorrect) the query sample 516 belongs to. In a particular embodiment, the most informative query sample 516 can be selected as:

[0039]

[0040] Where N u It can be the total number of unlabeled instances up to the current frame (e.g., all unlabeled instances so far). This function can ensure a small margin and a good matching score J at the same time.

[0041] In a particular embodiment, the informativeness metric 512 may reward confusing facial samples and send these facial samples to the user to query the user. In a particular embodiment, the cost function may have facial tracking 504, matching distance 506, and social connection 508 components. The social connection component 508 may extract user data information from a user context database. In a particular embodiment, using the same matching distance, the active learning algorithm may tend to group facial samples that appear in the same facial tracking 504 and / or are compatible with social relationships together, and assign relevant facial samples with higher weights to query the user. In a particular embodiment, the representativeness measure 514 may measure whether the sampled instances represent the overall distribution of the input unlabeled data, so that the query sample 516 will cover the unlabeled data space. In a particular embodiment, a label density term may be added to the query sampling function above, and the final sample selection combining the informativeness and representativeness metrics is:

[0042]

[0043] in Can be a category tag The probability of appearing in the video frame, from J Comb Calculate and normalize between class labels. In a specific embodiment, A frequency term may be associated with the facial recognition system to determine how often a person appears in multiple images. As an example and not limitation, if a person (face image) appears in multiple images at once, the facial recognition system may not send a query sample 516 or determine not to send a query sample 516 of an image to ask the user who the person may be. However, if the person appears more frequently, the frequency term may force the facial recognition system to send a query sample 516 to the user to request the user's identification and / or notify the user of an unknown person identified.

[0044] In particular embodiments, when proposing query samples 516 to a user, the representative metric 514 may reward simpler or fewer groups. The representative metric 514 may measure face tracking 504, matching distance 506, and social connections 508 at the group level. The representative metric 514 may also randomly sample face samples from different face sample clusters to propose merging two clusters together under the same face identifier. In particular embodiments, the data may be a stream as a result of the sequential nature of video clips. In particular embodiments, each new video may have limited or no initial labeled face recognition model. The process may be a cold start process to build a face sample library for the face recognition model. Individual identification may be based on visual matching and individual presence patterns. The objective function for face visual matching may be

[0045]

[0046] Among them G i It can be marked as l i A collection of gallery instances. j can be individual instances in a video frame, y i The example of the gallery may be the sample face pool 502. Vis It can be used to find the best visual match score between the facial image identified in the image and each facial image in the sample face pool 502. The gallery instances can be labeled by pre-clustering and learning. The objective function can be based on the context, J Context , which can exploit the order and co-occurrence relationship between each individual and the individuals in the scene. In particular, the co-occurrence objective function can rely on a set of individual labels estimated from the frame, and their co-occurrence probabilities. Let:

[0047]

[0048] Among them l j Can be individual instance x j Individual markers, I i can be the set of individual instance indices of frame i. In addition, suppose h j Is an individual j the frame number when it appeared in recent history, and

[0049]

[0050] The objective function of individual recognition can be defined based on the context of frame i as:

[0051] J Context (L i |H i , S i )=αJ Co-Presence (L i)+βJ Seq (L i |H i )+γJ Scene (L i |S i )|

[0052] in

[0053] J Co-Presence (L i )∝p CP (L i ),

[0054] J Seq (L i |H i )∝P Seq (L i |H i ).

[0055] α, β and γ can be weights. CP (L i ) and P Seq (L i |H i ) may be an individual marker j ∈L i The probability of co-occurring in a frame and reappearing respectively based on the history of their appearance in the past. They can be estimated based on observations from the video clip.

[0056] In a particular embodiment, a facial recognition system can extract individuals from a video and construct contexts to represent individual presence patterns so as to match time-varying co-occurrence graphs. As an example and not limitation, based on co-occurrence graphs involving main characters in a story, main characters can be matched with individual presence patterns in a video. These main characters can be identified individuals that often appear in multiple images (e.g., videos, movies, etc.). As an example and not limitation, main characters can be people who appear in multiple images within a threshold time period. A facial recognition system can construct a time-varying co-occurrence graph by processing stream data. A time-varying co-occurrence graph can be a representation of individual social connections that take into account evolution in a continuous order. In a particular embodiment, a facial recognition system can generate a character basis to accurately model the characters in a video, thereby having enhanced video understanding. A facial recognition system can accurately identify characters in a video, regardless of appearance changes (e.g., changes in clothing, blocked faces, blurred faces, heavily made-up faces, etc.) throughout the video. To this end, a facial recognition system can generate a character basis using contexts associated with individual characters. This can even be done by developing a deep context model for an individual character in the absence of a facial image by combining other individual characters and / or scenes. In certain embodiments, this can be applied to other settings and used to identify "primary persons" in different environments. By way of example and not limitation, an employee may be identified as a primary person in a workplace if the employee frequently appears in images captured in the workplace, while other people appear or disappear from images captured in the workplace (e.g., temporary employees, delivery persons, etc.).

[0057] In a particular embodiment, a facial recognition system may establish and trigger a family context graph. The facial recognition system may initially access multiple facial images from one or more devices and send a query sample to the user to confirm and / or mark each facial sample cluster from multiple facial images. The facial recognition system may identify the background of each member of the family. As an example and not limitation, the husband usually leaves in the early morning and the wife usually leaves in the late morning. In a particular embodiment, the facial recognition system may send a facial identifier corresponding to a user identifier to another device. Another device may perform an action in response to receiving the user identifier. As an example and not limitation, when a user identified by facial identifier 1 stands in front of a display or mirror coupled to a device that has received facial identifier 1 (or a user identifier corresponding to facial identifier 1), the device may retrieve one or more information and / or notifications corresponding to the user identified by facial identifier 1. For example, if the user is a husband, the estimated time of his morning commute on the route to work may be presented to the user. As another example and not limitation, when a user identified by facial identifier 2 stands in front of a display or mirror coupled to a device that received facial identifier 2 (or a user identifier corresponding to facial identifier 2), then the device may retrieve one or more information and / or notifications corresponding to the user identified by facial identifier 2. In addition, the display or mirror may interact with the user to actively learn new faces through new facial images. This may be to establish facial context associations.

[0058] In certain embodiments, the facial recognition system may define an occurrence frequency target based on person-scene co-occurrence patterns or person-person co-occurrence patterns, such that the facial recognition system may be optimized for detecting unfamiliar faces, in the sense that at any moment in time, the system goal may be to minimize false positives of unfamiliar faces and optimize the accuracy of detected unfamiliar faces:

[0059] J facecontext =w 1 J SceneContext +w 2 J PeopleContext

[0060] Among them, J SceneContext can represent the location and scene parameters associated with an individual, and J PeopleContext The first weight and the second weight may be applied to each J SceneContext and J PeopleContext .

[0061] Figure 6An example user interaction with a facial sample is shown. Process 600 may begin by displaying known and unlabeled facial images 602 to a user. The known and unlabeled facial images 602 may be presented to the user on a user device such as a smartphone. In certain embodiments, the user may have a multimodal user interaction to view known and unlabeled faces 602. In certain embodiments, a facial recognition system may determine to send a query sample to the user, as described herein. The query sample may be a known and unlabeled facial image 602. In certain embodiments, the facial recognition system may send a facial image to the user in response to determining to send the query sample to query the user. The facial recognition system may display known faces 602 and their exact context, as well as unlabeled faces 602, on an interface (e.g., a user device). Providing context with known faces 602 may increase the available information for the user to make decisions based on context. In certain embodiments, the user may perform a rename of a facial image or remove a facial image 604, add a facial identifier or add an identifier to a group 606, or request to display a user's facial image collection 608. The user may rename 604 known and / or unlabeled facial images 602 to correct known and / or unlabeled facial images 602. For situations where facial images 602 are of unfamiliar individuals, the user may remove 604 known and / or unlabeled facial images 602. The user may add facial identifiers 606 to unlabeled facial images 602 where the user knows the individual associated with the unlabeled facial images 602. The user may add identifiers 606 to a set of known and / or unlabeled facial images 602 to create a new cluster or clusters of labeled facial samples. In certain embodiments, the user may request that a collection of the user's facial images 608 be displayed.

[0062] In certain embodiments, a known facial image 602 may be presented to a user and a user identifier associated with the known facial image 602 may be requested. In certain embodiments, a user may send user input to a facial recognition system via a user device. In certain embodiments, the user device may be a camera device as described herein. The user input may be a text input, a touch input, a gesture, or an audio input. In certain embodiments, a user may use a voice service to tag a user identifier and / or process a user identifier via an assistant service. As an example and not limitation, an untagged facial image 602 may be presented to a user and the user may say “This is John” to apply a user identifier associated with “John” to the untagged facial image 602. In certain embodiments, a user may confirm or reject a known and / or untagged facial image 602 being presented to the user.

[0063] In a particular embodiment, the facial recognition system may request access to audio input from another device based on the privacy settings of a user associated with the facial recognition system. The facial recognition system may use the audio input to generate a voice profile that matches the facial identifier. The voice profile may be added to the context model associated with the facial identifier by enabling the facial recognition system to more accurately identify people in images. To do this, the facial recognition system may access facial images of people within a scene and determine a 60% facial match score with the nearest facial sample cluster. In addition, the facial recognition system may access the voice profile associated with the facial identifier and / or separately determine the facial identifier associated with the retrieved voice profile to identify the person associated with the voice profile and compare whether the facial identifier associated with the retrieved voice profile matches the facial identifier corresponding to the facial sample cluster.

[0064] In certain embodiments, a facial recognition system may use cloud computing to synchronize facial recognition models acquired from mobile devices, workplaces, and home environments, so that facial recognition models in different contexts and locations may be shared. Synchronization of facial recognition models may improve the accuracy with which a facial recognition system is able to identify individuals associated with corresponding facial recognition models. In certain embodiments, synchronization and sharing of facial recognition models may be based on privacy settings. Privacy settings may be associated with a user corresponding to a facial recognition model, a user of a user device, and / or both. In certain embodiments, in response to determining that two or more facial recognition models correspond to the same individual, two or more facial recognition models on the same device and / or different devices may be merged.

[0065] In certain embodiments, the facial recognition system may define an objective function to pay more attention to the scene context or the person context by adjusting or learning weights. 1 ,w 2 :J facecontext =w 1 J scene +w 2 J people In certain embodiments, a user may specify an objective function to reflect their preferred scenarios for accurate facial recognition and fewer false alarms for unfamiliar faces. As an example and not limitation, a user may turn off alarms sent to the user. As another example and not limitation, a user may increase the sensitivity of the detection alarm and request that more query samples be sent.

[0066] Figure 7An example method for identifying a person through facial recognition and context is shown. At step 702, the method may include identifying a facial image from a scene image using a facial recognition model. At step 704, the method may include determining a context associated with the facial image based on the context model. At step 706, the method may include identifying a person from a database based on the context and the facial image.

[0067] Where appropriate, specific examples may be repeated Figure 7 Although the present disclosure will Figure 7 The specific steps of the method are described and shown as occurring in a specific order, but the present disclosure contemplates Figure 7 Any suitable steps of the method may occur in any suitable order. Furthermore, although this disclosure describes and illustrates example methods for mapping user input to content, this disclosure contemplates any suitable method for mapping user input to content, which may include, where appropriate, Figure 7 In addition, although the present disclosure describes and illustrates the implementation of Figure 7 specific components, devices or systems for specific steps of the method, but the present disclosure contemplates performing Figure 7 Any suitable combination of any suitable components, devices, or systems of any suitable steps of the method.

[0068] In certain embodiments, the facial recognition system may generate a plurality of context models, each corresponding to a facial identifier. As an example and not limitation, the facial recognition system may associate a facial identifier with an individual and determine that the individual has been identified as being associated with one or more contexts. For example, if the person has previously been identified by a facial image in an image of a person playing a musical instrument, the person is associated with a context that includes a musical instrument. In certain embodiments, the facial recognition model may be incrementally trained by presenting facial images to be tagged. The facial image may also be presented with a context (e.g., the person is associated with a context that includes a musical instrument). As another example, but not by way of limitation, the person may be a younger brother who typically comes in through the front door after his older sister after school. Thus, the facial recognition system may associate a co-presence pattern with a person who is around his older sister after school. In certain embodiments, when a new individual presence pattern associated with a context (e.g., visual consistency, co-presence, presence in a continuous or scene) is detected, the facial recognition system may associate the new individual presence pattern with the one or more individuals identified within the image. In certain embodiments, the facial recognition system may incrementally train a facial recognition model associated with an individual by presenting a facial image with information corresponding to a pattern of the person (co-presence pattern). In certain embodiments, contexts may include co-presence patterns associated with a person or a person and / or a scene. By way of example and not limitation, the context may be the same individual in the same place, such as two people routinely going to a restaurant for lunch on a weekday. By way of another example, and not limitation, the context may be the user arriving at a location at the same time or similar times during the work week. By way of another example and not limitation, the context may be the user appearing at a series of locations in the same or similar order, such as going to work, then to a child's school, and then returning home.

[0069] In certain embodiments, a facial recognition system may be used to identify individuals during an album search. As an example and not limitation, a facial recognition system may be able to identify context associated with an image scene and identify facial images of people within an image. For example, if two brothers often appear together in an album, a facial recognition system may be able to identify the two brothers from a database based on context and facial images. As an example and not limitation, for Halloween photos, if one brother is wearing a mask, but the other brother is wearing a different costume that reveals his face, the facial recognition system may identify the two people as the correct brothers by using the context associated with the image and the people in the scene. In certain embodiments, a facial recognition system may associate a specific location with an individual. As an example and not limitation, if an individual is identified in a work environment, the facial recognition system may associate the individual with the work environment (e.g., as an employee). In certain embodiments, a facial recognition system may generate a spatial common appearance map that associates a specific individual with a specific location. In certain embodiments, when a specific face is identified in an album, the facial recognition system may suggest finding related photos with other people (e.g., two brothers who often appear together in photos) based on context modeling.

[0070] Figure 8 An example computer system 800 is shown. In a particular embodiment, one or more computer systems 800 perform one or more steps of one or more methods described or shown herein. In a particular embodiment, one or more computer systems 800 provide functionality described or shown herein. In a particular embodiment, software running on one or more computer systems 800 performs one or more steps of one or more methods described or shown herein, or provides functionality described or shown herein. A particular embodiment includes one or more portions of one or more computer systems 800. Here, where appropriate, references to a computer system may include a computing device, and vice versa. Furthermore, where appropriate, references to a computer system may include one or more computer systems.

[0071] The present disclosure contemplates any suitable number of computer systems 800. The present disclosure contemplates that computer system 800 takes any suitable physical form. By way of example and not limitation, computer system 800 may be an embedded computer system, a system on a chip (SOC), a single board computer system (SBC) (e.g., a computer on a module (COM) or a system on a module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a computer system grid, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or a combination of two or more of these. Where appropriate, computer system 800 may include one or more computer systems 800; separate or distributed; across multiple locations; across multiple machines; across multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 800 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. By way of example and not limitation, one or more computer systems 800 may perform one or more steps of one or more methods described or illustrated herein in real time or in batch mode. Where appropriate, one or more computer systems 800 may perform at different times or different locations one or more steps of one or more methods described or illustrated herein.

[0072] In a particular embodiment, computer system 800 includes a processor 802, memory 804, storage 806, input / output (I / O) interface 808, communication interface 810, and bus 812. Although this disclosure describes and illustrates a particular computer system having particular numbers of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable numbers of any suitable components in any suitable arrangement.

[0073] In a particular embodiment, the processor 802 includes hardware for executing instructions, such as instructions that constitute a computer program. By way of example and not limitation, to execute instructions, the processor 802 may retrieve (or fetch) instructions from an internal register, an internal cache, a memory 804, or a storage 806; decode and execute them; and then write one or more results to an internal register, an internal cache, a memory 804, or a storage 806. In a particular embodiment, the processor 802 may include one or more internal caches for data, instructions, or addresses. The present disclosure contemplates that the processor 802 includes any suitable number of any suitable internal caches, where appropriate. By way of example and not limitation, the processor 802 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). The instructions in the instruction cache may be copies of instructions in the memory 804 or the storage 806, and the instruction cache may speed up the retrieval of those instructions by the processor 802. The data in the data cache may be a copy of data in the memory 804 or storage 806 for operation by instructions executed on the processor 802; the result of a previous instruction executed at the processor 802 for access or writing to the memory 804 or storage 806 by subsequent instructions executed at the processor 802; or other suitable data. The data cache may speed up read or write operations of the processor 802. The TLB may speed up virtual address translation of the processor 802. In a particular embodiment, the processor 802 may include one or more internal registers for data, instructions, or addresses. The present disclosure contemplates that the processor 802 includes any suitable number of any suitable internal registers, where appropriate. Where appropriate, the processor 802 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 802. Although the present disclosure describes and illustrates a particular processor, the present disclosure contemplates any suitable processor.

[0074] In certain embodiments, memory 804 includes main memory for storing instructions for processor 802 to execute or data for processor 802 to operate on. By way of example and not limitation, computer system 800 may load instructions from memory 806 or another source (e.g., another computer system 800) to memory 804. Processor 802 may then load instructions from memory 804 to internal registers or internal caches. To execute the instructions, processor 802 may retrieve the instructions from the internal registers or internal caches and decode them. During or after the execution of the instructions, processor 802 may write one or more results (which may be intermediate or final results) to the internal registers or internal caches. Processor 802 may then write one or more of these results to memory 804. In certain embodiments, processor 802 executes only instructions in one or more internal registers or internal caches or memory 804 (as opposed to memory 806 or elsewhere), and operates only on data in one or more internal registers or internal caches or memory 804 (as opposed to memory 806 or elsewhere). One or more memory buses (each of which may include an address bus and a data bus) may couple the processor 802 to the memory 804. The bus 812 may include one or more memory buses, as described below. In a particular embodiment, one or more memory management units (MMUs) reside between the processor 802 and the memory 804 and facilitate access to the memory 804 requested by the processor 802. In a particular embodiment, the memory 804 includes a random access memory (RAM). Where appropriate, the RAM may be a volatile memory. Where appropriate, the RAM may be a dynamic random access RAM (DRAM) or a static RAM (SRAM). In addition, where appropriate, the RAM may be a single-port or multi-port RAM. The present disclosure contemplates any suitable RAM. Where appropriate, the memory 804 may include one or more memories 804. Although the present disclosure describes and illustrates specific memories, the present disclosure contemplates any suitable memory.

[0075] In a particular embodiment, the memory 806 includes a large capacity memory for data or instructions. As an example and not limitation, the memory 806 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 806 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 806 may be inside or outside the computer system 800. In a particular embodiment, the memory 806 is a non-volatile solid-state memory. In a particular embodiment, the memory 806 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. The present disclosure contemplates that the large capacity memory 806 takes any suitable physical form. Where appropriate, the memory 806 may include one or more storage control units that facilitate communication between the processor 802 and the memory 806. Where appropriate, the memory 806 may include one or more memories 806. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

[0076] In a particular embodiment, the I / O interface 808 includes hardware, software, or both, providing one or more interfaces for communication between the computer system 800 and one or more I / O devices. Where appropriate, the computer system 800 may include one or more of these I / O devices. One or more of these I / O devices can implement communication between a person and the computer system 800. As an example and not limitation, the I / O device may include a keyboard, a keypad, a microphone, a monitor, a mouse, a printer, a scanner, a speaker, a static camera, a stylus, a tablet computer, a touch screen, a trackball, a camera, another suitable I / O device, or a combination of two or more of these. The I / O device may include one or more sensors. The present disclosure contemplates any suitable I / O device and any suitable I / O interface 808 for them. Where appropriate, the I / O interface 808 may include one or more device or software drivers so that the processor 802 can drive one or more of these I / O devices. Where appropriate, the I / O interface 808 may include one or more I / O interfaces 808. Although the present disclosure describes and illustrates a specific I / O interface, the present disclosure contemplates any suitable I / O interface.

[0077] In a particular embodiment, the communication interface 810 includes hardware, software, or both, providing one or more interfaces for communication (e.g., packet-based communication) between the computer system 800 and one or more other computer systems 800 or one or more networks. As an example and not limitation, the communication interface 810 may include a network interface controller (NIC) or a network adapter for communicating with an Ethernet or other wired network, or a wireless NIC (WNIC) or a wireless adapter for communicating with a wireless network (e.g., a WI-FI network). The present disclosure contemplates any suitable network and any suitable communication interface 810 thereof. As an example and not limitation, the computer system 800 may communicate with one or more parts of a self-organizing network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more parts of one or more of these networks may be wired or wireless. As an example, the computer system 800 may communicate with a wireless PAN (WPAN) (e.g., a Bluetooth WPAN), a WI-FI network, a WI-MAX network, a cellular phone network (e.g., a global system for mobile communications (GSM) network), or other suitable wireless networks, or a combination of two or more of these. Computer system 800 may include any suitable communication interface 810 for any of these networks, where appropriate. Communication interface 810 may include one or more communication interfaces 810, where appropriate. Although this disclosure describes and illustrates particular communication interfaces, this disclosure contemplates any suitable communication interfaces.

[0078] In certain embodiments, bus 812 includes hardware, software, or both that couple the components of computer system 800 to each other. By way of example and not limitation, bus 812 may include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front side bus (FSB), a hypertransport (HT) interconnect, an industry standard architecture (ISA) bus, an INFINIBAND interconnect, a low pin count (LPC) bus, a memory bus, a micro channel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus, or another suitable bus, or a combination of two or more of these. Where appropriate, bus 812 may include one or more buses 812. Although the present disclosure describes and illustrates a particular bus, the present disclosure contemplates any suitable bus or interconnect.

[0079] Here, where appropriate, the computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical disk drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, the computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0080] Here, "or" is inclusive and not exclusive, unless expressly stated otherwise or the context indicates otherwise. Thus, as used herein, "A or B" means "A, B, or both," unless expressly stated otherwise or the context indicates otherwise. Furthermore, "and" is both joint and separate, unless expressly stated otherwise or the context indicates otherwise. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly stated otherwise or the context indicates otherwise.

[0081] As used herein, “automatically” and its derivatives mean “without human intervention” unless expressly stated otherwise or the context indicates otherwise.

[0082] The scope of the present disclosure includes all changes, substitutions, variations, alterations and modifications to the exemplary embodiments described or shown herein that will be understood by those of ordinary skill in the art. The scope of the present disclosure is not limited to the exemplary embodiments described or shown herein. In addition, although the present disclosure describes and illustrates various embodiments including specific components, elements, features, functions, operations or steps, any of these embodiments may include any combination or arrangement of any components, elements, features, functions, operations or steps described or shown anywhere herein, which is understandable to those of ordinary skill in the art. In addition, in the appended claims, references to devices or systems or components of devices or systems suitable for, arranged to, capable of, configured to, capable of, operable or operable to perform specific functions include the device, system, component, whether or not it or the specific function is activated, turned on or unlocked, as long as the device, system or component is so suitable for, arranged to, capable of, configured to, capable of, operable or operable. In addition, although the present disclosure describes or illustrates specific embodiments as providing specific advantages, specific embodiments may not provide, provide some or all of these advantages.

Claims

1. A method performed by a computer device, comprising: identifying a first facial image from the scene image; determining a context associated with the first facial image based on the context model; identifying a person from a database based on the context and a facial recognition model associated with the first facial image; determining that the person is an unfamiliar person based on the context and a failure to match the first facial image to one of the plurality of facial recognition models; assigning a temporary facial identifier to the first facial image; forming a cluster including a first facial image having the temporary facial identifier and a second facial image having a second temporary facial identifier; wherein the first facial image and the second facial image are both identified as being associated with the unfamiliar person, and wherein the first facial image is obtained using a first device at a first location, and the second facial image is obtained using a second device at a second location; and The facial images in the cluster are cross-checked using a plurality of facial recognition models to identify the unfamiliar person.

2. The method according to claim 1, further comprising: Determining the informativeness and representativeness of facial images; and Present facial images to be labeled for training a facial recognition model. The method of claim 2 , wherein presenting the facial image comprises presenting a context associated with the facial image. 4 . The method of claim 1 , wherein the context comprises at least one of: visual consistency, sequence, presence associated with a scene, or a co-presence pattern associated with a person or scene.

5. The method of claim 4, wherein the co-presence pattern associated with the person identifies a first person and a second person that were previously identified together, and wherein the co-presence pattern for the scene includes the first person identified in the scene and a set of scene parameters.

6. The method according to claim 1, wherein: A scene includes one or more scene parameters; and The scene parameters include one or more of: a time associated with the scene, a date associated with the scene, one or more elements captured within the scene, or an activity associated with the scene.

7. A device comprising: Camera; and One or more non-transitory storage media containing instructions and one or more processors operable to execute the instructions to: identifying a first facial image from the scene image; determining a context associated with the first facial image based on the context model; identifying a person from a database based on the context and a facial recognition model associated with the first facial image; determining that the person is an unfamiliar person based on the context and a failure to match the first facial image to one of a plurality of facial recognition models; assigning a temporary facial identifier to the first facial image; forming a cluster including a first facial image having the temporary facial identifier and a second facial image having a second temporary facial identifier; wherein the first facial image and the second facial image are both identified as being associated with the unfamiliar person, and wherein the first facial image is obtained using a first device at a first location, and the second facial image is obtained using a second device at a second location; and The facial images in the cluster are cross-checked using a plurality of facial recognition models to identify the unfamiliar person.

8. The apparatus of claim 7, wherein the one or more processors are further operable to execute instructions to: Determining the informativeness and representativeness of facial images; and Present facial images to be labeled for training a facial recognition model.

9. The device according to claim 8, wherein: The one or more processors are further operable to execute the instructions to present a context associated with the facial image.

10. The apparatus of claim 7, wherein the context comprises at least one of: visual consistency, sequence, presence associated with a scene, or a co-presence pattern associated with a person or scene.

11. The apparatus of claim 10, wherein the copresence pattern associated with the person identifies a first person and a second person that were previously identified together, and wherein the copresence pattern for the scene includes the first person identified in the scene and a set of scene parameters.

12. The device according to claim 7, wherein: A scene includes one or more scene parameters; and The scene parameters include one or more of: a time associated with the scene, a date associated with the scene, one or more elements captured within the scene, or an activity associated with the scene.

13. One or more computer-readable storage media containing instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 6.

Images