Face recognition method and device

Abstract

Embodiments of the present application disclose a face recognition method and device. In the embodiments of the present application, after obtaining a target image pair to be recognized, the target image pair is input into a feature extraction model to determine target face features of each target face image in the target image pair through the feature extraction model, determine an initial similarity of the target image pair according to the target face features, determine scene compensation values of each target face image matched with a current scene, and then determine a target similarity of the target image pair according to the initial similarity and the scene compensation values, and further generate a face recognition result according to the target similarity. The scene compensation values are used for scene compensation of the initial similarity, and the face recognition result is used for representing whether each target face image comes from a same target object. Thus, the embodiments of the present application can improve the robustness and accuracy of a face recognition system in various complex scenes.

Description

Technical Field

[0001] This invention relates to the field of facial image recognition technology, specifically to a facial recognition method and apparatus. Background Technology

[0002] With the rapid development of artificial intelligence and computer vision technologies, facial recognition, as a crucial biometric identification technology, has been widely applied in numerous fields. However, due to the inability to adapt to different scenarios, current facial recognition solutions are easily affected by the environment, resulting in poor robustness and accuracy in various complex situations. Summary of the Invention

[0003] In view of this, the present invention aims to provide a face recognition method and apparatus to improve the robustness and accuracy of face recognition systems in various complex scenarios.

[0004] In a first aspect, embodiments of the present invention aim to provide a face recognition method, the method comprising:

[0005] Obtain the target image pair to be identified;

[0006] The target image pair is input into the feature extraction model to determine the target face features of each target face image in the target image pair through the feature extraction model;

[0007] The initial similarity of the target image pairs is determined based on the target facial features of each target;

[0008] Determine a scene compensation value for each target face image to match the current scene, and the scene compensation value is used to perform scene compensation on the initial similarity;

[0009] The target similarity of the target image pair is determined based on the initial similarity and each of the scene compensation values;

[0010] A face recognition result is generated based on the target similarity, and the face recognition result is used to characterize whether each of the target face images comes from the same target object.

[0011] Secondly, embodiments of the present invention aim to provide a face recognition device, the device comprising:

[0012] The image acquisition unit is used to acquire the target image pair to be identified.

[0013] The model calling unit is used to input the target image pair into the feature extraction model, so as to determine the target face features of each target face image in the target image pair through the feature extraction model;

[0014] An initial similarity determination unit is used to determine the initial similarity of the target image pairs based on the target facial features of each target;

[0015] A similarity compensation value determination unit is used to determine a scene compensation value for each target face image and the current scene, wherein the scene compensation value is used to perform scene compensation on the initial similarity.

[0016] A target similarity determination unit is used to determine the target similarity of the target image pair based on the initial similarity and each of the scene compensation values;

[0017] The recognition result generation unit is used to generate a face recognition result based on the target similarity, wherein the face recognition result is used to characterize whether each of the target face images comes from the same target object.

[0018] Thirdly, embodiments of the present invention aim to provide a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the method described in the first aspect.

[0019] Fourthly, embodiments of the present invention aim to provide an electronic device, the device comprising:

[0020] Memory is used to store one or more computer program instructions;

[0021] A processor, wherein the one or more computer program instructions are executed by the processor to implement the method as described in the first aspect.

[0022] Fifthly, embodiments of the present invention aim to provide a computer program product that, when run on a computer, causes the computer to perform the method described in the first aspect.

[0023] In this embodiment of the invention, after acquiring the target image pair to be identified, the target image pair is input into a feature extraction model to determine the target facial features of each target face image in the target image pair. Based on these features, an initial similarity of the target image pair is determined, along with a scene compensation value for matching each target face image with the current scene. Then, based on the initial similarity and the scene compensation values, the target similarity of the target image pair is determined, and finally, a face recognition result is generated based on the target similarity. The scene compensation value is used to compensate for the initial similarity, and the face recognition result is used to characterize whether the target face images belong to the same target object. Therefore, by introducing an adaptive similarity post-processing step in the face recognition service process, this embodiment of the invention can improve the robustness and accuracy of the face recognition system in various complex scenarios. Attached Figure Description

[0024] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0025] Figure 1 This is a flowchart of a face recognition method according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of the scene compensation value determination method according to an embodiment of the present invention;

[0027] Figure 3 This is a flowchart of the feature extraction model training method according to an embodiment of the present invention;

[0028] Figure 4 This is a flowchart of the sample image group determination method according to an embodiment of the present invention;

[0029] Figure 5 This is a flowchart of the method for determining difficult-to-correct sample image pairs according to an embodiment of the present invention;

[0030] Figure 6 This is a flowchart of the method for determining difficult-to-bear sample image pairs according to an embodiment of the present invention;

[0031] Figure 7 This is a flowchart illustrating the training of the initial model based on a contrastive learning method according to an embodiment of the present invention.

[0032] Figure 8 This is a schematic diagram illustrating the training process of the feature extraction model in an embodiment of the present invention;

[0033] Figure 9 This is a schematic diagram illustrating the process of generating face recognition results according to an embodiment of the present invention;

[0034] Figure 10 This is a schematic diagram of a face recognition device according to an embodiment of the present invention;

[0035] Figure 11 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0036] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0037] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0038] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0039] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0040] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only under the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0041] It should be noted that the facial recognition scheme in this invention can be applied to any scenario involving comparing two facial images to achieve facial recognition. Examples include ride-hailing order verification, online payment verification, commuting check-in, access control identity recognition, and tourism identity verification, etc., which will not be elaborated further here.

[0042] Figure 1 This is a flowchart of a face recognition method according to an embodiment of the present invention. It is intended to be noted that... Figure 1 The entity executing the facial recognition method shown can be a corresponding facial recognition device. By executing... Figure 1 The illustrated face recognition method allows the face recognition device to incorporate an adaptive similarity post-processing step during the face recognition service provision process. Therefore, embodiments of the present invention can improve the robustness and accuracy of the face recognition system in various complex scenarios. Optionally, the face recognition device can be a terminal device (e.g., a desktop computer, laptop computer, smartphone, smart speaker, smart wearable device, tablet computer, or in-vehicle terminal, etc.) or a server (which can be a single computer, a cluster of multiple computers, or a cloud server that can flexibly adjust computing resources through cloud technology). This application does not impose any limitations on this. Figure 1 As shown, the face recognition method may specifically include the following steps:

[0043] Step S100: Obtain the target image pair to be identified.

[0044] Specifically, a face recognition device can acquire a pair of target images to be recognized. These target images may include two target face images that need to be compared. It should be understood that, in this embodiment, by comparing the two target face images, the face recognition device can determine whether the two target face images come from the same target object, thereby achieving face recognition.

[0045] Optionally, the target face image in the target image pair can be obtained by the face recognition device through receiving input from a requesting object (the requesting object can be a user, a corresponding terminal device, or a server), or by the face recognition device through self-collection, or by the face recognition device through querying a database. This application does not impose any limitations on this. It should be understood that the target face image acquisition methods given above are merely illustrative. In actual applications, based on the design and instructions of relevant personnel, the face recognition device can specifically adopt a corresponding method matching the application scenario to acquire two target face images, thereby obtaining a target image pair. It should be understood that the two target face images in the target image pair can be acquired by the face recognition device through the same method, or by the face recognition device through different methods. This application does not impose any limitations on this. For example, one target face image in the target image pair can be acquired by the face recognition device through receiving input from a requesting object, while the other target face image can be acquired by the face recognition device through self-collection. Furthermore, when a facial recognition device acquires a target face image through its own acquisition method, if the facial recognition device is a terminal device, it can acquire the target face image using its own onboard sensor components. If the facial recognition device is a server, it can acquire the target face image through communication with corresponding offline deployed terminal devices.

[0046] Optionally, the target face image in the target image pair can be any type of face image, such as a selfie, artistic photo, a photo of a personal ID document, or a scanned image. The specific type can depend on the application scenario, and this application does not impose any restrictions on it.

[0047] Step S200: Input the target image pair into the feature extraction model to determine the target face features of each target face image in the target image pair through the feature extraction model.

[0048] Specifically, after acquiring the target image pair to be identified, the face recognition device can input the target image pair into a feature extraction model to determine the target facial features of each target face image in the target image pair. The feature extraction model can be a pre-trained model used to extract facial features from face images.

[0049] Step S300: Determine the initial similarity of the target image pairs based on the target facial features of each target.

[0050] Specifically, after determining the target facial features of each target face image in the target image pair, the face recognition device can determine the initial similarity of the target image pair based on each target facial feature.

[0051] It is important to note that in the various embodiments of this invention, the similarity of image pairs can be used to characterize the similarity of facial features between two face images in an image pair. Furthermore, different comparison scenarios and the feature vector space distribution corresponding to different people each have their own particularities. For example, in some scenarios, the facial features of certain individuals may appear more compact in the vector space, resulting in a generally higher similarity score. Conversely, in some scenarios, the facial features of certain individuals may appear more loosely distributed in the vector space, resulting in a generally lower similarity score. Here, the initial similarity of the target image pair can be specifically understood as the similarity of the target image pair determined when ignoring the aforementioned particularities in the feature vector space distribution and only considering the general score calculation logic. It should be understood that if the face recognition device generates face recognition results solely based on the initial similarity, it is difficult to ensure the accuracy of the generated face recognition results. Therefore, in subsequent steps, the face recognition device can perform scene compensation on the initial similarity of the target image pair to obtain the target similarity, and then use the target similarity to generate the face recognition result.

[0052] Optionally, in this embodiment, the similarity of the image pair can be represented by the vector distance between the facial features of the face images in the image pair in the feature vector space. Specifically, in step S300, the face recognition device can calculate the Euclidean distance between the target facial features of the two target face images in the feature vector space, and determine the Euclidean distance as the initial similarity of the target image pair. It should be understood that the above-described method for determining the initial similarity of the target image pair is merely illustrative. In actual applications, based on the design and instructions of relevant personnel, the face recognition device may also use other methods to determine the initial similarity of the target image pair, and this application does not impose any limitations on this.

[0053] Step S400: Determine the scene compensation value for each target face image to match the current scene.

[0054] Specifically, after determining the initial similarity of the target image pairs, the face recognition device can determine the scene compensation value for matching each target face image with the current scene. This scene compensation value can be used to compensate for the initial similarity of the target image pairs. It is important to note that different comparison scenarios and different people have their own unique feature vector space distributions. Here, the scene compensation value can specifically be used to compensate for differences in the feature vector space distribution of the target face images within the current scene.

[0055] Optionally, in this embodiment, for any target face image, the face recognition device can determine the average similarity between the target face features of the target face image and each benchmark face feature in the current scene as the scene compensation value for matching the target face image with the current scene.

[0056] Figure 2 This is a flowchart illustrating a method for determining scene compensation values ​​according to an embodiment of the present invention. It should be understood that by executing... Figure 2 The scene compensation value determination method shown allows the face recognition device to determine the scene compensation value that matches each target face image with the current scene, thus achieving step S400 above. Figure 2 As shown, the method for determining the scene compensation value may specifically include the following steps:

[0057] Step S410: For each target face image, calculate the similarity between the target face features of the target face image and each benchmark face feature in the reference distribution database to determine the corresponding multiple benchmark similarities.

[0058] Specifically, for each target face image in the target image pair, the face recognition device can calculate the similarity between the target face features of the target face image and the baseline face features stored in the reference distribution database to determine multiple baseline similarities corresponding to the target image. The baseline face features can be face features extracted by the face recognition device from face images collected in the current scene. It should be noted that in this embodiment of the invention, the collection of each face image is carried out with the authorization of relevant personnel, and the collected face images will only be used for the face recognition scheme in this embodiment of the invention and will not be used for other purposes.

[0059] Step S420: For each target face image, determine the scene compensation value for matching the target face image with the current scene based on the corresponding multiple benchmark similarities.

[0060] Specifically, for each target face image in the target image pair, the face recognition device can determine the scene compensation value for matching the target face image with the current scene based on multiple benchmark similarities corresponding to the target face image.

[0061] Optionally, in step S420, as a method for determining the scene compensation value, the face recognition device can determine the average of multiple benchmark similarities corresponding to the target face image as the scene compensation value for matching the target face image with the current scene. It should be understood that the scene compensation value determination method given above is merely illustrative. In actual applications, based on the design and instructions of relevant personnel, the face recognition device may also use other methods to determine the scene compensation value for matching the target face image with the current scene, and this application does not impose any limitations on this.

[0062] Step S500: Determine the target similarity of the target image pair based on the initial similarity and each of the scene compensation values.

[0063] Specifically, after determining the initial similarity and the scene compensation value for matching each target face image with the current scene, the face recognition device can determine the target similarity of the target image pair based on the initial similarity and the scene compensation value.

[0064] Optionally, in step S500, as a method for determining target similarity, the face recognition device can determine the target similarity of the target image pair by continuously subtracting the scene compensation values ​​from the initial similarity. It should be understood that the target similarity determination methods given above are merely illustrative. In practical applications, based on the design and instructions of relevant personnel, the face recognition device may also use other methods to determine target similarity, and this application does not impose any limitations on this.

[0065] Step S600: Generate face recognition results based on the target similarity.

[0066] Specifically, after determining the target similarity of a pair of target images, the face recognition device can generate a face recognition result based on the target similarity. This face recognition result can be used to characterize whether different target face images originate from the same target object.

[0067] Therefore, by introducing an adaptive similarity post-processing step in the process of providing face recognition services, the embodiments of the present invention can improve the robustness and accuracy of the face recognition system in various complex scenarios (e.g., international scenarios, or scenarios where there is a large difference in the male-to-female ratio).

[0068] Optionally, in step S600, as a method for generating face recognition results, the face recognition device can compare the target similarity with a preset similarity threshold to generate a face recognition result. The preset similarity threshold can be pre-set by relevant personnel according to actual circumstances, and this application does not limit its specific value. Specifically, the face recognition device can generate face recognition results indicating that each target face image comes from a different target object when the target similarity is less than the preset similarity threshold, and generate face recognition results indicating that each target face image comes from the same target object when the target similarity is greater than or equal to the preset similarity threshold.

[0069] Optionally, in this embodiment, the feature extraction model used by the face recognition device can be trained by the face recognition device itself, or it can be trained by related devices and then deployed on the face recognition device. This application does not impose any restrictions on this.

[0070] Figure 3 This is a flowchart illustrating a feature extraction model training method according to an embodiment of the present invention. It should be understood that by executing... Figure 3 The feature extraction model training method shown can be used to train a feature extraction model using a face recognition device or related equipment. It should be noted that the following description will primarily focus on the example of the feature extraction model training method being performed by related equipment. Figure 3 As shown, the feature extraction model training method may specifically include the following steps:

[0071] Step S710: Determine multiple sample image groups.

[0072] Specifically, the relevant equipment can determine multiple sample image groups. These sample image groups can include hard-to-positive sample image pairs and hard-to-negative sample image pairs where the sample face images overlap. For example, hard-to-positive sample image pair A includes sample face images a and b, and hard-to-negative sample image pair E includes sample face images a and c. Therefore, hard-to-positive sample image pair A and hard-to-negative sample image pair E can be considered as a single sample image group. It is worth noting that, in this embodiment of the invention, a hard-to-positive sample image pair can include two sample face images from the same sample object but with low facial feature similarity. A hard-to-negative sample image pair can include two sample face images from different sample objects but with high facial feature similarity.

[0073] Figure 4 This is a flowchart illustrating a method for determining a group of sample images according to an embodiment of the present invention. It should be understood that by performing... Figure 4 The method for determining sample image groups shown allows the relevant equipment to determine multiple sample image groups that meet the requirements, thus achieving step S710 described above. Figure 4As shown, the method for determining the sample image group may specifically include the following steps:

[0074] Step S711: Obtain multiple positive sample image pairs.

[0075] Specifically, the relevant equipment can acquire multiple positive sample image pairs. A positive sample image pair can include two sample face images from the same sample object. For example, if sample face images a and b both come from the same sample object, then sample face images a and b can be considered as a single positive sample image pair.

[0076] Optionally, the sample face image in the positive sample image pair can be any type of face image, such as a selfie, an artistic photo, a photo of a personal ID document, or a scanned image, and this application does not impose any restrictions on this.

[0077] Optionally, in step S711, the sample objects corresponding to different positive sample image pairs can be completely different. Alternatively, the sample objects corresponding to some positive sample image pairs can also overlap, and this application does not impose any restrictions on this.

[0078] Optionally, in step S711, the relevant device can acquire multiple positive sample image pairs by receiving user input, or by acquiring multiple positive sample image pairs automatically, or by combining the above two methods. This application does not impose any restrictions on this.

[0079] Step S712: Determine the difficult positive sample image pairs among the plurality of positive sample image pairs.

[0080] Specifically, after acquiring multiple positive sample image pairs, the relevant device can identify the difficult positive sample image pairs among the multiple positive sample image pairs.

[0081] Figure 5 This is a flowchart of a method for determining difficult-to-correct sample image pairs according to an embodiment of the present invention. It should be understood that by performing... Figure 5 The method for determining difficult-to-correct sample image pairs shown allows the relevant equipment to identify difficult-to-correct sample image pairs from multiple positive sample image pairs, thus implementing step S712 above. For example... Figure 5 As shown, the method for determining difficult-to-correct sample image pairs may specifically include the following steps:

[0082] Step S7121: Determine the similarity of each pair of positive sample images.

[0083] Specifically, the relevant equipment can determine the similarity of each positive sample image pair.

[0084] Optionally, in step S7121, the similarity of each positive sample image pair can be determined by the relevant device using a basic face recognition model. The basic face recognition model can be a pre-trained model used to estimate the similarity of sample image pairs during the sample training phase. Further optionally, the basic face recognition model can be trained using related face image pairs that have undergone similarity annotation. Alternatively, to avoid additional annotation and model training work, the basic face recognition model can also be a feature extraction model to be iterated; this application does not impose any limitations on this. Alternatively, in some embodiments, the similarity of each positive sample image pair can also be obtained through manual annotation. Correspondingly, in this embodiment, the relevant device can determine the similarity of each positive sample image pair based on the annotation information provided by the annotator.

[0085] Step S7122: Determine the attribute information of each positive sample image pair.

[0086] Specifically, the relevant equipment can determine the attribute information of each positive sample image pair. This attribute information is used to characterize whether there are appearance variations between the sample face images in the positive sample image pair. It should be understood that appearance variations between sample face images can specifically refer to visual changes in the face images caused by personal factors of the subject and / or sampling environment factors.

[0087] Optionally, the appearance changes represented by the attribute information can be set and adjusted by relevant personnel according to actual needs, and this application does not impose any restrictions on this. Illustratively, as one setting method, the appearance changes represented by the attribute information may specifically include one or more of the following combinations: occlusion changes, makeup changes, beard changes, age changes, hairstyle and hair accessory changes, and lighting condition changes.

[0088] Optionally, in step S7122, the attribute information of each positive sample image pair can be determined by the relevant device using a basic face attribute model. The basic face attribute model can be a pre-trained model used to annotate the appearance variations between face images in a face image pair. Further optionally, the basic face attribute model can be trained using related face image pairs that have undergone appearance variation annotation; this application does not impose any limitations on this. Alternatively, in some embodiments, the attribute information of each positive sample image pair can also be obtained through manual annotation. Correspondingly, in this embodiment, the relevant device can determine the attribute information of each positive sample image pair based on the annotation information provided by the annotator.

[0089] Step S7123: Determine the difficult positive sample image pair from the plurality of positive sample image pairs based on the similarity of each positive sample image pair and the attribute information.

[0090] Specifically, after determining the similarity and attribute information of each positive sample image pair, the relevant device can determine the difficult positive sample image pair from multiple positive sample image pairs based on the similarity and attribute information of each positive sample image pair.

[0091] Optionally, in step S7123, as a method for determining difficult-to-correct sample image pairs, the relevant device can specifically determine a corresponding number of positive sample image pairs with the lowest similarity and whose attribute information indicates a preset appearance change as difficult-to-correct sample image pairs. It should be understood that the above-described method for determining difficult-to-correct sample image pairs is merely illustrative. In actual applications, based on the design and instructions of relevant personnel, face recognition devices may also use other methods to determine difficult-to-correct sample image pairs, and this application does not impose any limitations on this.

[0092] Step S713: Determine the difficult negative sample image pair based on the plurality of positive sample image pairs.

[0093] Specifically, after identifying the difficult positive sample image pairs, the relevant equipment can determine the difficult negative sample image pairs based on multiple positive sample image pairs.

[0094] Figure 6 This is a flowchart of a method for determining hard-to-bear sample image pairs according to an embodiment of the present invention. It should be understood that by performing... Figure 6 The method for determining difficult-to-negative sample image pairs shown allows the relevant equipment to determine difficult-to-negative sample image pairs based on multiple positive sample image pairs, thus implementing step S713 above. For example... Figure 6 As shown, the method for determining the hard-to-bear sample image pairs may specifically include the following steps:

[0095] Step S7131: Combine the positive sample image pairs to obtain multiple negative sample image pairs.

[0096] Specifically, the relevant device can combine positive sample image pairs to obtain multiple negative sample image pairs. These negative sample image pairs can include sample face images from different sample objects. For example, assuming positive sample image pair A includes sample face images a and b, and positive sample image pair B includes sample face images c and d, the relevant device can combine positive sample image pairs A and B to obtain negative sample image pair E including sample face images a and c, negative sample image pair F including sample face images a and d, negative sample image pair G including sample face images b and c, and negative sample image pair H including sample face images b and d.

[0097] Step S7132: Determine the similarity of each negative sample image pair.

[0098] Specifically, after obtaining multiple negative sample image pairs, the relevant device can determine the similarity of each negative sample image pair. It should be understood that in step S7132, the relevant device can determine the similarity of each negative sample image pair in the same way as determining the similarity of each positive sample image pair, and its specific implementation will not be elaborated further here.

[0099] Step S7133: Determine the difficult negative sample image pair from the plurality of negative sample image pairs according to the similarity of each negative sample image pair.

[0100] Specifically, after determining the similarity of each negative sample image pair, the relevant equipment can determine the difficult negative sample image pair from multiple negative sample image pairs based on the similarity of each negative sample image pair.

[0101] Optionally, in step S7133, as a method for determining difficult negative sample image pairs, the relevant device can specifically determine the corresponding number of negative sample image pairs with the highest similarity as difficult negative sample image pairs. It should be understood that the method for determining difficult negative sample image pairs given above is only for illustration. In actual applications, based on the design and instructions of relevant personnel, face recognition devices may also use other methods to determine difficult negative sample image pairs, and this application does not impose any restrictions on this.

[0102] Step S714: For any sample face image that has overlapping pairs of hard positive and hard negative sample images, determine the hard positive and hard negative sample image pairs as the sample image group.

[0103] Specifically, after identifying the difficult-to-positive sample image pairs and the difficult-to-negative sample images, the relevant equipment can determine the difficult-to-positive sample image pairs and the difficult-to-negative sample image pairs with overlapping sample face images as sample image groups, thereby obtaining multiple sample image groups. For example, assuming that the difficult-to-positive sample image pair A includes sample face images a and b, and the difficult-to-negative sample image pair E includes sample face images a and c, then the relevant equipment can determine the difficult-to-positive sample image pair A and the difficult-to-negative sample image pair E with overlapping sample face images as a sample image group.

[0104] Step S720: For each of the sample image groups, train an initial model based on the sample image group to obtain the feature extraction model.

[0105] Specifically, the model training process is essentially a continuous process of adjusting the parameters in the model using sample data. After determining multiple sets of sample images, the relevant equipment can use these sets as sample data to train an initial model based on each set, thereby obtaining a feature extraction model. It should be understood that the feature extraction model can be an initial model that meets the performance requirements after multiple rounds of training using multiple sets of sample images.

[0106] Optionally, in this embodiment, the initial model can be a neural network model or other types of models, and this application does not impose any restrictions on this.

[0107] Optionally, in step S720, for any set of sample images, the relevant devices can train an initial model based on the contrastive learning method according to the set of sample images.

[0108] Figure 7 This is a flowchart illustrating the training of an initial model based on a contrastive learning approach, as described in an embodiment of the present invention. It should be understood that by performing... Figure 7 The method steps shown allow the relevant equipment to train an initial model based on contrastive learning using any set of sample images. For example... Figure 7 As shown, the process of training the initial model based on contrastive learning may specifically include the following steps:

[0109] Step S721: Determine the similarity between the hard positive sample image pairs and the hard negative sample image pairs in the sample image group based on the initial model.

[0110] Specifically, for the current sample image set, the relevant equipment can determine the similarity between the hard positive sample image pairs and the hard negative sample image pairs in the sample image set based on the initial model.

[0111] Optionally, in step S721, when determining the similarity between the difficult-to-positive sample image pairs and the difficult-to-negative sample image pairs in the sample image group according to the initial model, the relevant device can first input the two sample face images in the difficult-to-positive sample image pair and the two sample face images in the difficult-to-negative sample image pair into the initial model respectively, so as to determine the sample face features of the two sample face images in the difficult-to-positive sample image pair and the sample face features of the two sample face images in the difficult-to-negative sample image pair through the initial model. Then, the similarity between the difficult-to-positive sample image pairs is determined according to the sample face features of the two sample face images in the difficult-to-positive sample image pair, and the similarity between the difficult-to-negative sample image pairs is determined according to the sample face features of the two sample face images in the difficult-to-negative sample image pair.

[0112] It should be understood that the way the relevant equipment determines the similarity of sample image pairs based on the sample facial features is the same as the way the face recognition equipment determines the initial similarity of target image pairs based on the target facial features, and will not be repeated here.

[0113] Step S722: Determine the contrast loss value based on the similarity between the hard positive sample image pairs and the hard negative sample image pairs in the sample image group.

[0114] Specifically, after determining the similarity between the hard positive and hard negative sample image pairs in the sample image set, the relevant device can determine the contrast loss value based on the similarity between the hard positive and hard negative sample image pairs in the sample image set. The contrast loss value can be used to characterize the quality assessment of the initial model output.

[0115] Optionally, in this embodiment, the comparison loss value L can be calculated and determined based on the following formula:

[0116] L = d(A, B) 2 +max(0,α-d(A,C) 2 )

[0117] Where d(A,B) can be the similarity of the hard-to-positive sample image pair (i.e., the vector distance between the sample face features of the sample face images in the feature vector space of the hard-to-positive sample image pair). d(A,C) can be the similarity of the hard-to-negative sample image pair (i.e., the vector distance between the sample face features of the sample face images in the feature vector space of the hard-to-negative sample image pair). α can be a marginal value, which is a predefined distance threshold used to control the minimum similarity of the hard-to-negative sample image pair (i.e., the minimum vector distance between the sample face features of the sample face images in the feature vector space of the hard-to-negative sample image pair).

[0118] It should be understood that by using a contrastive learning-based method to train the feature extraction model, this embodiment enables the feature extraction model to narrow the vector distance between facial features from the same object in the feature vector space, and widen the vector distance between facial features from different objects in the feature vector space when determining facial features. Therefore, this embodiment can improve the similarity of image pairs from the same object and reduce the similarity of image pairs from different objects when determining the similarity of image pairs based on the feature extraction model, thereby providing a foundation for the implementation of face recognition services.

[0119] Step S723: Adjust the model parameters of the initial model according to the contrast loss value.

[0120] Specifically, the contrastive loss value can be used to determine the direction and magnitude of model parameter adjustments. After determining the contrastive loss value, the relevant equipment can adjust the parameters of the initial model based on the contrastive loss value.

[0121] Therefore, by introducing a hybrid hard sample mining strategy and a model training method based on contrastive learning, this embodiment can achieve face recognition capability for complex target image pairs, thereby further improving the robustness and accuracy of the face recognition system in various complex scenarios.

[0122] It is important to note that current widely adopted face recognition solutions typically rely on classification loss to optimize and iterate the face feature extraction model. However, the classification layer used to calculate the classification loss consumes significant computational resources during model training, resulting in high iteration costs and a lack of rapid iteration capabilities for existing face recognition solutions. Compared to existing technologies, by removing redundant and cumbersome classification layers and employing a model training method based on contrastive learning to train the feature extraction model, this invention can reduce iteration costs and increase iteration speed, thereby achieving short, frequent, and fast recognition capability iterations.

[0123] Figure 8 This is a schematic diagram illustrating the training process of the feature extraction model according to an embodiment of the present invention. It should be understood that... Figure 8 Each square pattern in the diagram can be used to represent a sample face image. Two square patterns defined by a dashed frame can be used to represent a pair of sample images, and three square patterns defined by a dashed frame can be used to represent a group of sample images. Furthermore, multiple square patterns filled with the same pattern can be used to indicate that the sample face images represented by the multiple square patterns come from the same sample object.

[0124] like Figure 8 As shown, in order to train the feature extraction model, the relevant device can first acquire multiple positive sample image pairs 81. Each positive sample image pair can include two sample face images from the same sample object.

[0125] Furthermore, after acquiring multiple positive sample image pairs 81, on the one hand, the relevant device can perform hard positive sample mining on the multiple positive sample image pairs 81 to determine multiple hard positive sample image pairs 82. Here, a hard positive sample image pair may include two sample face images from the same sample object, but with low facial feature similarity. On the other hand, the relevant device can combine the multiple positive sample image pairs 81 to determine multiple negative sample image pairs 83, and perform hard negative sample mining on the multiple negative sample image pairs 83 to determine multiple hard negative sample image pairs 84. Here, a hard negative sample image pair may include two sample face images from different sample objects, but with high facial feature similarity.

[0126] Furthermore, after identifying multiple difficult positive sample image pairs 82 and multiple difficult negative sample image pairs 84, the relevant device can filter out difficult positive sample image pairs and negative sample image pairs that have overlapping sample face images from the multiple difficult positive sample image pairs 82 and multiple difficult negative sample image pairs 84 to identify multiple sample image groups 85.

[0127] Furthermore, after determining multiple sample image groups 85, the relevant equipment can train an initial model 86 based on contrastive learning for each sample image group to obtain a feature extraction model. It should be understood that the feature extraction model can be an initial model that meets the performance requirements after multiple rounds of training using multiple sample image groups.

[0128] It should be understood that by using a contrastive learning-based approach to train the feature extraction model, this embodiment enables the feature extraction model to narrow the vector distance between facial features in the feature vector space of facial images from the same object, and to widen the vector distance between facial features in the feature vector space of facial images from different objects, when determining facial features.

[0129] Figure 9 This is a schematic diagram illustrating the process of generating face recognition results according to an embodiment of the present invention. Figure 9 As shown, firstly, the face recognition device can acquire a target image pair 91 to be recognized. The target image pair 91 can include two target face images that need to be compared.

[0130] Furthermore, after acquiring the target image pair 91, the face recognition device can input the target image pair 91 into the feature extraction model 92 to determine the target face features 93 of each target face image in the target image pair 91 through the feature extraction model 92.

[0131] Furthermore, after determining the target facial features 93 of each target face image, the face recognition device can determine the initial similarity 95 of the target image pair 91 based on each target facial feature 93. Additionally, the face recognition device can also calculate the statistical mean of the similarity between the target facial features 93 of each target face image and each baseline facial feature stored in the reference distribution database 94 in the current scene, to determine the scene compensation value 96 for matching each target face image with the current scene. It should be understood that here, the baseline facial features specifically refer to the facial features extracted by the face recognition device from the face images collected in the current scene using the feature extraction model 92.

[0132] Furthermore, after determining the initial similarity 95 and the scene compensation value 96 for each target face image to match the current scene, the face recognition device can determine the target similarity 97 of the target image pair based on the initial similarity 95 and each scene compensation value 96.

[0133] Furthermore, after determining the target similarity 97, the face recognition device can generate a face recognition result 98 based on the target similarity 97. The face recognition result 98 can be used to characterize whether the target face images in the target image pair 91 come from the same target object.

[0134] In this embodiment of the invention, after acquiring the target image pair to be identified, the target image pair is input into a feature extraction model to determine the target facial features of each target face image in the target image pair. Based on these features, an initial similarity of the target image pair is determined, along with a scene compensation value for matching each target face image with the current scene. Then, based on the initial similarity and the scene compensation values, the target similarity of the target image pair is determined, and finally, a face recognition result is generated based on the target similarity. The scene compensation value is used to compensate for the initial similarity, and the face recognition result is used to characterize whether the target face images belong to the same target object. Therefore, by introducing an adaptive similarity post-processing step in the process of providing face recognition services, this embodiment of the invention can improve the robustness and accuracy of the face recognition system in various complex scenarios.

[0135] Figure 10 This is a schematic diagram of a face recognition device according to an embodiment of the present invention. Figure 10 As shown, the face recognition device of this embodiment includes a target image acquisition unit 101, a model calling unit 102, an initial similarity determination unit 103, a similarity compensation value determination unit 104, a target similarity determination unit 105, and a recognition result generation unit 106.

[0136] Specifically, the image acquisition unit 101 is used to acquire the target image pair to be identified;

[0137] The model calling unit 102 is used to input the target image pair into the feature extraction model, so as to determine the target face features of each target face image in the target image pair through the feature extraction model;

[0138] The initial similarity determination unit 103 is used to determine the initial similarity of the target image pairs based on the target facial features of each target;

[0139] The similarity compensation value determination unit 104 is used to determine the scene compensation value that matches each of the target face images with the current scene, and the scene compensation value is used to perform scene compensation on the initial similarity;

[0140] The target similarity determination unit 105 is used to determine the target similarity of the target image pair based on the initial similarity and each of the scene compensation values;

[0141] The recognition result generation unit 106 is used to generate a face recognition result based on the target similarity, and the face recognition result is used to characterize whether each of the target face images comes from the same target object.

[0142] In this embodiment of the invention, after acquiring the target image pair to be identified, the target image pair is input into a feature extraction model to determine the target facial features of each target face image in the target image pair. Based on these features, an initial similarity of the target image pair is determined, along with a scene compensation value for matching each target face image with the current scene. Then, based on the initial similarity and the scene compensation values, the target similarity of the target image pair is determined, and finally, a face recognition result is generated based on the target similarity. The scene compensation value is used to compensate for the initial similarity, and the face recognition result is used to characterize whether the target face images belong to the same target object. Therefore, by introducing an adaptive similarity post-processing step in the process of providing face recognition services, this embodiment of the invention can improve the robustness and accuracy of the face recognition system in various complex scenarios.

[0143] In this embodiment of the invention, after acquiring the target image pair to be identified, the target image pair is input into a feature extraction model to determine the target facial features of each target face image in the target image pair. Based on these features, an initial similarity of the target image pair is determined, along with a scene compensation value for matching each target face image with the current scene. Then, based on the initial similarity and the scene compensation values, the target similarity of the target image pair is determined, and finally, a face recognition result is generated based on the target similarity. The scene compensation value is used to compensate for the initial similarity, and the face recognition result is used to characterize whether the target face images belong to the same target object. Therefore, by introducing an adaptive similarity post-processing step in the process of providing face recognition services, this embodiment of the invention can improve the robustness and accuracy of the face recognition system in various complex scenarios.

[0144] Figure 11 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Figure 11 As shown, Figure 11The illustrated electronic device can specifically be the face recognition device described in the above embodiments, which includes a general computer hardware structure, including at least a processor 111 and a memory 112. The processor 111 and the memory 112 are connected via a bus 113. The memory 112 is adapted to store instructions or programs executable by the processor 111. The processor 111 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 111 executes the instructions stored in the memory 112 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 113 connects the above-mentioned components together, and also connects the above-mentioned components to the display controller 114, the display device, and the input / output (I / O) device 115. The input / output (I / O) device 115 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 115 is connected to the system via an input / output (I / O) controller 116.

[0145] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0147] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0148] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0149] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0150] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0151] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A face recognition method, characterized in that, The method includes: Obtain the target image pair to be identified; The target image pair is input into the feature extraction model to determine the target face features of each target face image in the target image pair through the feature extraction model; The initial similarity of the target image pairs is determined based on the target facial features of each target; Determine a scene compensation value for each target face image to match the current scene, and the scene compensation value is used to perform scene compensation on the initial similarity; The target similarity of the target image pair is determined based on the initial similarity and each of the scene compensation values; A face recognition result is generated based on the target similarity, and the face recognition result is used to characterize whether each of the target face images comes from the same target object.

2. The method according to claim 1, characterized in that, The feature extraction model is trained in the following manner: Multiple sample image groups are identified, wherein the sample image groups include hard positive sample image pairs and hard negative sample image pairs where sample face images overlap; For each of the sample image groups, an initial model is trained based on the sample image groups to obtain the feature extraction model.

3. The method according to claim 2, characterized in that, Determining multiple sample image groups includes: Acquire multiple positive sample image pairs, wherein the positive sample image pairs include sample face images from the same sample object; Among the plurality of positive sample image pairs, identify the difficult positive sample image pairs; Based on the plurality of positive sample image pairs, determine the hard negative sample image pairs; For any sample face image that has overlapping pairs of hard positive and hard negative sample images, the hard positive and hard negative sample image pairs are determined as the sample image group.

4. The method according to claim 3, characterized in that, The step of determining the difficult positive sample image pairs among the plurality of positive sample image pairs includes: Determine the similarity of each pair of positive sample images; Determine the attribute information of each positive sample image pair, wherein the attribute information is used to characterize whether there are appearance changes between the sample face images in the positive sample image pair; The difficult positive sample image pair is determined from the plurality of positive sample image pairs based on the similarity of each positive sample image pair and the attribute information.

5. The method according to claim 4, characterized in that, The appearance changes include one or more of the following combinations: changes in concealment, changes in makeup, changes in beard, changes in age, changes in hairstyle and hair accessories, and changes in lighting conditions.

6. The method according to claim 3, characterized in that, The step of determining the hard negative sample image pair based on the plurality of positive sample image pairs includes: The positive sample image pairs are combined to obtain multiple negative sample image pairs, which include sample face images from different sample objects; Determine the similarity of each of the negative sample image pairs; The difficult negative sample image pair is determined from the plurality of negative sample image pairs based on the similarity of each negative sample image pair.

7. The method according to claim 2, characterized in that, The step of training the initial model based on the sample image set includes: The similarity between the hard positive sample image pairs and the hard negative sample image pairs in the sample image group is determined based on the initial model. The contrast loss value is determined based on the similarity between the hard positive sample image pairs and the hard negative sample image pairs in the sample image group. Adjust the model parameters of the initial model based on the comparative loss value.

8. The method according to claim 7, characterized in that, The step of determining the similarity between hard positive sample image pairs and hard negative sample image pairs in the sample image group based on the initial model includes: The sample face features of the sample face images in the difficult-to-correct sample image pair are determined based on the initial model; The similarity of the difficult-to-correct sample image pair is determined based on the sample face features of the sample face images in the difficult-to-correct sample image pair; The sample face features of the sample face images in the hard-to-bear sample image pair are determined based on the initial model; The similarity of the difficult-to-bear sample image pair is determined based on the sample face features of the sample face images in the difficult-to-bear sample image pair.

9. The method according to claim 1, characterized in that, The determination of the scene compensation value for matching each target face image with the current scene includes: For each target face image, the similarity between the target face features of the target face image and each benchmark face feature in the reference distribution database is calculated to determine the corresponding multiple benchmark similarities, wherein the benchmark face features are face features extracted from face images collected in the current scene; For each target face image, a scene compensation value is determined based on multiple corresponding benchmark similarities to match the target face image with the current scene.

10. The method according to claim 1, characterized in that, The step of determining the scene compensation value for matching the target face image with the current scene based on multiple corresponding benchmark similarities includes: The average value of the corresponding multiple benchmark similarities is determined as the scene compensation value for matching the target face image with the current scene.

11. The method according to claim 1, characterized in that, The step of generating face recognition results based on the target similarity includes: The target similarity is compared with a preset similarity threshold to generate the face recognition result.

12. A face recognition device, characterized in that, The device includes: The image acquisition unit is used to acquire the target image pair to be identified. The model calling unit is used to input the target image pair into the feature extraction model, so as to determine the target face features of each target face image in the target image pair through the feature extraction model; An initial similarity determination unit is used to determine the initial similarity of the target image pairs based on the target facial features of each target; A similarity compensation value determination unit is used to determine a scene compensation value for each target face image and the current scene, wherein the scene compensation value is used to perform scene compensation on the initial similarity. A target similarity determination unit is used to determine the target similarity of the target image pair based on the initial similarity and each of the scene compensation values; The recognition result generation unit is used to generate a face recognition result based on the target similarity, wherein the face recognition result is used to characterize whether each of the target face images comes from the same target object.

13. A computer-readable storage medium storing computer program instructions thereon, characterized in that, The computer program instructions, when executed by a processor, implement the method as described in any one of claims 1-11.

14. An electronic device, characterized in that, The device includes: Memory is used to store one or more computer program instructions; A processor, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-11.

15. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-11.

Images