A face recognition method, system and device

Abstract

The embodiment of the application provides a face recognition method, system and device, relates to the technical field of image processing, and the method comprises the following steps: obtaining a to-be-recognized face image; determining an image region to which a gaze point of a target user belongs from the to-be-recognized face image as a to-be-recognized gaze region according to a preset region determination mode; performing feature extraction based on the to-be-recognized face image and the to-be-recognized gaze region to obtain a to-be-recognized face feature; and comparing the to-be-recognized face feature with a preset face feature to obtain a recognition result of the to-be-recognized face image. According to the embodiment of the application, the reliability of face recognition can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a face recognition method, system, and apparatus. Background Technology

[0002] With the rapid development of image processing technology, users' identities can be authenticated based on facial images in scenarios such as access control and building systems.

[0003] In related technologies, electronic devices can acquire a user's (or registered user's) facial image (or registered facial image) during registration. Subsequently, the matching degree between the authenticated user's facial image and the registered facial image can be calculated to authenticate the user's identity based on the matching degree.

[0004] It is evident that in related technologies, if a malicious attacker obtains a registered user's facial image through malicious means, they can perform identity authentication based on the obtained facial image, resulting in low reliability of facial recognition. Summary of the Invention

[0005] The purpose of this application is to provide a face recognition method, system, and apparatus to improve the reliability of face recognition. The specific technical solution is as follows:

[0006] According to a first aspect of the embodiments of this application, a face recognition method is provided, the method comprising:

[0007] Acquire the face image to be identified; wherein, when displaying the face image to be identified, the gaze point of the target user in the face image to be identified is located in the face region of the face image to be identified;

[0008] According to the preset region determination method, the image region to which the gaze point of the target user belongs is determined from the face image to be identified, and is used as the gaze region to be identified;

[0009] Feature extraction is performed based on the image of the face to be identified and the gaze region to be identified to obtain the features of the face to be identified;

[0010] The facial features to be identified are compared with preset facial features to obtain the recognition result of the facial image to be identified;

[0011] The preset facial features are obtained by extracting features from the registered user's registered facial image and the registered gaze region to which the registered user's gaze point belongs in the registered facial image; the registered gaze region is determined according to a preset region determination method; when the registered facial image is displayed to the registered user, the registered user's gaze point is located in the specified facial region in the registered facial image.

[0012] Optionally, determining the image region to which the target user's gaze point belongs from the face image to be identified, according to a preset region determination method, as the gaze region to be identified, includes:

[0013] From the face image to be identified, a specified image region centered on the target user's gaze point is determined as the gaze region to be identified.

[0014] or,

[0015] Based on image segmentation, the segmented image regions of the face image containing the gaze point of the target user are identified as the gaze regions to be identified.

[0016] Optionally, the step of extracting features based on the image of the face to be identified and the gaze region to be identified to obtain the features of the face to be identified includes:

[0017] Based on the gaze region to be identified, the weight of each pixel in the face image to be identified is determined.

[0018] Feature extraction is performed based on the determined weights and the image of the face to be identified to obtain the features of the face to be identified.

[0019] Optionally, determining the weight of each pixel in the face image to be identified based on the gaze region to be identified includes:

[0020] Obtain the heatmap corresponding to the gaze region to be identified; where each point in the heatmap corresponds one-to-one with each pixel in the face image to be identified; each point in the heatmap represents the probability that the corresponding pixel in the face image to be identified is the gaze point of the target user;

[0021] For each pixel in the face image to be identified, the probability of the corresponding point in the heatmap to be identified is used as the weight of that pixel.

[0022] The step of extracting features based on the determined weights and the image of the face to be identified to obtain the features of the face to be identified includes:

[0023] The face image and heatmap to be identified are input into a pre-trained convolutional neural network to obtain the face features to be identified.

[0024] The classification network model, to which the convolutional neural network belongs, is trained based on the first sample face image, the first sample heatmap corresponding to the first sample face image, and the first sample label.

[0025] Optionally, the training process of the convolutional neural network includes the following steps:

[0026] Obtain the first sample face image, the first sample heatmap, and the first sample label;

[0027] The first sample face image and the first sample heatmap are input into the convolutional neural network in the first classification network model of the initial structure to obtain the first sample face features;

[0028] The facial features of the first sample are input into the classification layer of the first classification network model with the initial structure to obtain the first predicted label;

[0029] Calculate the first loss value based on the first predicted label and the first sample label;

[0030] The model parameters of the first classification network model with the initial structure are adjusted based on the first loss value, and training continues until the first classification network model with the initial structure converges.

[0031] Optionally, determining the weight of each pixel in the face image to be identified based on the gaze region to be identified includes:

[0032] The face image to be recognized is divided into multiple sub-regions to be recognized.

[0033] For each sub-region to be identified, the ratio of the portion of the gaze region to be identified that belongs to the sub-region to be identified to the size of the sub-region to be identified is calculated, and used as the weight of each pixel in the sub-region to be identified.

[0034] The step of extracting features based on the determined weights and the image of the face to be identified to obtain the features of the face to be identified includes:

[0035] Each sub-region to be identified is concatenated with its corresponding weight, and the concatenation results are input into a pre-trained self-attention network to obtain the facial features to be identified.

[0036] Among them, the classification network model to which the self-attention network belongs is trained based on the second sample face image, the second sample gaze region corresponding to the second sample face image, and the second sample label.

[0037] Optionally, the training process of the self-attention network includes the following steps:

[0038] Obtain the second sample face image, the second sample gaze region, and the second sample label;

[0039] The second sample face image is divided into multiple sample sub-regions;

[0040] For each sample sub-region, the ratio of the portion of the second sample gaze region belonging to that sample sub-region to the size of that sample sub-region is calculated, and used as the weight of each pixel in that sample sub-region.

[0041] Each sample sub-region is concatenated with its corresponding weight, and the concatenation results are input into the self-attention network in the second classification network model of the initial structure to obtain the face features of the second sample.

[0042] The facial features of the second sample are input into the classification layer of the second classification network model with the initial structure to obtain the second predicted label;

[0043] The second loss value is calculated based on the second predicted label and the second sample label;

[0044] The model parameters of the second classification network model with the initial structure are adjusted based on the second loss value, and training continues until the second classification network model with the initial structure converges.

[0045] According to a second aspect of the embodiments of this application, a face recognition device is provided, the device comprising:

[0046] The first acquisition module is used to acquire the face image to be recognized; wherein, when displaying the face image to be recognized, the gaze point of the target user in the face image to be recognized is located in the face region of the face image to be recognized;

[0047] The first determining module is used to determine the image region to which the gaze point of the target user belongs from the face image to be identified according to a preset region determining method, and use it as the gaze region to be identified.

[0048] The feature extraction module is used to extract features based on the image of the face to be identified and the gaze region to be identified, so as to obtain the features of the face to be identified;

[0049] The comparison module is used to compare the facial features to be identified with preset facial features to obtain the recognition result of the facial image to be identified;

[0050] Optionally, the first determining module includes:

[0051] The first gaze region determination submodule is used to determine a specified image region centered on the gaze point of the target user from the face image to be identified, as the gaze region to be identified;

[0052] or,

[0053] The second gaze region determination submodule is used to determine the face segmentation image region containing the gaze point of the target user in the face image to be identified, as the gaze region to be identified.

[0054] Optionally, the feature extraction module includes:

[0055] The weight determination submodule is used to determine the weight of each pixel in the face image to be identified based on the gaze region to be identified.

[0056] The feature acquisition submodule is used to extract features based on the determined weights and the face image to be identified, thereby obtaining the face features to be identified.

[0057] Optionally, the weight determination submodule includes:

[0058] The heatmap acquisition unit is used to acquire the heatmap corresponding to the gaze region to be identified; wherein, each point in the heatmap corresponds one-to-one with each pixel in the face image to be identified; each point in the heatmap represents the probability that the pixel corresponding to that point in the face image to be identified is the gaze point of the target user;

[0059] The first weight acquisition unit is used to determine the probability of the corresponding point in the heat map of the face image to be identified for each pixel in the face image to be identified, and use that probability as the weight of the pixel.

[0060] The feature acquisition submodule is specifically used to input the face image to be identified and the heat map to be identified into a pre-trained convolutional neural network to obtain the face features to be identified.

[0061] The classification network model, to which the convolutional neural network belongs, is trained based on the first sample face image, the first sample heatmap corresponding to the first sample face image, and the first sample label.

[0062] Optionally, the device further includes:

[0063] The first training module is used to acquire the first sample face image, the first sample heatmap, and the first sample label;

[0064] The first sample face image and the first sample heatmap are input into the convolutional neural network in the first classification network model of the initial structure to obtain the first sample face features;

[0065] The facial features of the first sample are input into the classification layer of the first classification network model with the initial structure to obtain the first predicted label;

[0066] Calculate the first loss value based on the first predicted label and the first sample label;

[0067] The model parameters of the first classification network model with the initial structure are adjusted based on the first loss value, and training continues until the first classification network model with the initial structure converges.

[0068] Optionally, the weight determination submodule includes:

[0069] The image segmentation unit is used to divide the face image to be recognized into multiple sub-regions to be recognized;

[0070] The second weight acquisition unit is used to calculate, for each sub-region to be identified, the ratio of the portion of the gaze region to be identified belonging to the sub-region to be identified to the size of the sub-region to be identified, and use it as the weight of each pixel in the sub-region to be identified.

[0071] The feature acquisition submodule is specifically used to concatenate each sub-region to be identified with its corresponding weight, and input each concatenation result into a pre-trained self-attention network to obtain the facial features to be identified.

[0072] Among them, the classification network model to which the self-attention network belongs is trained based on the second sample face image, the second sample gaze region corresponding to the second sample face image, and the second sample label.

[0073] Optionally, the device further includes:

[0074] The second training module is used to obtain the second sample face image, the second sample gaze region, and the second sample label;

[0075] The second sample face image is divided into multiple sample sub-regions;

[0076] For each sample sub-region, the ratio of the portion of the second sample gaze region belonging to that sample sub-region to the size of that sample sub-region is calculated, and used as the weight of each pixel in that sample sub-region.

[0077] Each sample sub-region is concatenated with its corresponding weight, and the concatenation results are input into the self-attention network in the second classification network model of the initial structure to obtain the face features of the second sample.

[0078] The facial features of the second sample are input into the classification layer of the second classification network model with the initial structure to obtain the second predicted label;

[0079] The second loss value is calculated based on the second predicted label and the second sample label;

[0080] The model parameters of the second classification network model with the initial structure are adjusted based on the second loss value, and training continues until the second classification network model with the initial structure converges.

[0081] According to a third aspect of the embodiments of this application, a face recognition system is provided, the system including an image acquisition component and a processing component;

[0082] The image acquisition component is used to acquire facial images as the facial images to be identified;

[0083] The processing component is used to execute any of the face recognition methods described above.

[0084] Optionally, the system further includes a display component for displaying the face image to be identified, and for displaying a reminder message when the target user's gaze point is not located in the face region of the currently displayed face image to be identified, so as to prompt the target user to adjust their gaze point.

[0085] According to a fourth aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising:

[0086] The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus.

[0087] Memory, used to store computer programs;

[0088] When a processor executes a program stored in memory, it implements the method steps described in any one of the first aspects above.

[0089] According to a fifth aspect of the embodiments of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the method steps described in any one of the first aspects above.

[0090] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in any one of the first aspects above.

[0091] The technical solutions provided by the embodiments in this review may include the following beneficial effects:

[0092] This application provides a face recognition method that can acquire a face image to be recognized. When displaying the face image to be recognized, the gaze point of the target user in the face image is located in the face region of the face image. The image region to which the target user's gaze point belongs is determined from the face image according to a preset region determination method, and is used as the gaze region to be recognized. Feature extraction is performed based on the face image to be recognized and the gaze region to be recognized to obtain the face features to be recognized. The face features to be recognized are compared with preset face features to obtain the recognition result of the face image to be recognized.

[0093] Based on the above processing, facial features can be identified by combining the facial image and the user's gaze region within the image. Therefore, even if a malicious attacker obtains a registered user's facial image, the gaze region corresponding to the obtained image will differ from the gaze region corresponding to the user's facial image during registration. Consequently, the attacker will fail to pass identity authentication, thus improving the reliability of facial recognition.

[0094] In addition, for different facial recognition scenarios, the same user can gaze at different facial regions during registration. Consequently, even if a malicious attacker obtains the user's facial image during registration in one scenario, they will not be able to use that facial image to authenticate the user in another scenario, which can further improve the reliability of facial recognition.

[0095] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0096] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0097] Figure 1 A flowchart illustrating one embodiment of the face recognition method provided in this application;

[0098] Figure 2 A flowchart illustrating a second embodiment of the face recognition method provided in this application.

[0099] Figure 3 A flowchart illustrating a third embodiment of the face recognition method provided in this application.

[0100] Figure 4 A flowchart illustrating a fourth embodiment of the face recognition method provided in this application.

[0101] Figure 5 A flowchart illustrating a fifth embodiment of the face recognition method provided in this application.

[0102] Figure 6 A schematic diagram illustrating the principle of obtaining facial features of a person to be identified based on a convolutional neural network, as provided in this application embodiment;

[0103] Figure 7 A flowchart illustrating a training method for a convolutional neural network provided in this application embodiment;

[0104] Figure 8 A flowchart of a sixth embodiment of the face recognition method provided in this application;

[0105] Figure 9 A schematic diagram illustrating the principle of acquiring facial features to be identified based on a self-attention network, as provided in an embodiment of this application.

[0106] Figure 10A flowchart of a self-attention network training method provided in this application embodiment;

[0107] Figure 11 An example diagram of a face recognition process provided in an embodiment of this application;

[0108] Figure 12 A structural diagram of a face recognition device provided in an embodiment of this application;

[0109] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0110] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0111] To improve the reliability of face recognition, embodiments of this application provide a face recognition method, apparatus, electronic device, and storage medium, which are described in detail below:

[0112] This application provides a face recognition method that can be applied to electronic devices. For example, the electronic device can be a device used to authenticate users in an access control system. Accordingly, the electronic device can acquire the user's facial image to perform face recognition and determine whether to allow the user to pass based on the recognition result.

[0113] See Figure 1 , Figure 1 This is a flowchart illustrating a face recognition method provided in an embodiment of this application. Figure 1 As shown, it includes the following steps:

[0114] Step S101: Obtain the face image to be identified.

[0115] Specifically, when displaying the image of a face to be identified, the gaze point of the target user in the image of the face to be identified is located in the face region of the image of the face to be identified.

[0116] Step S102: According to the preset region determination method, determine the image region to which the target user's gaze point belongs from the face image to be identified, and use it as the gaze region to be identified.

[0117] Step S103: Based on the face image to be identified and the gaze region to be identified, feature extraction is performed to obtain the face features to be identified.

[0118] Step S104: Compare the facial features to be identified with preset facial features to obtain the recognition result of the facial image to be identified.

[0119] The preset facial features are obtained by extracting features from the registered user's registered facial image and the registered gaze region to which the registered user's gaze point belongs in the registered facial image; the registered gaze region is determined according to a preset region determination method; when the registered facial image is displayed to the registered user, the registered user's gaze point is located in the specified facial region in the registered facial image.

[0120] Based on the above processing, facial features can be identified by combining the facial image and the user's gaze region within the image. Therefore, even if a malicious attacker obtains a registered user's facial image, the gaze region corresponding to the obtained image will differ from the gaze region corresponding to the user's facial image during registration. Consequently, the attacker will fail to pass identity authentication, thus improving the reliability of facial recognition.

[0121] In addition, for different facial recognition scenarios, the same user can gaze at different facial regions during registration. Consequently, even if a malicious attacker obtains the user's facial image during registration in one scenario, they will not be able to use that facial image to authenticate the user in another scenario, which can further improve the reliability of facial recognition.

[0122] Regarding step S101, when displaying the face image to be recognized, the gaze point of the target user in the face image to be recognized is located in the face region of the displayed face image to be recognized.

[0123] In one implementation, an image acquisition device and an image display can be provided for acquiring images. Then, when authentication is required, the image acquisition device can acquire an image of the object to be authenticated and display the acquired image on the image display. The image acquisition device and image display can be integrated into an electronic device. Alternatively, the image acquisition device and image display can be other devices independent of the electronic device; in this case, the electronic device can communicate with the image acquisition device to obtain the acquired image.

[0124] For example, when the object of authentication is a user, such as when a user is facing the aforementioned image capture device, the image capture device can capture the user's facial image (i.e., the face image to be recognized), and then display the captured face image to the user through an image display. It can then be determined whether the user's gaze point is located within the facial region of the displayed face image to be recognized. In this case, the target user is the user currently requiring authentication.

[0125] For example, when the object to be authenticated is a non-living image (e.g., a facial image), such as when a user places a facial image to be identified in front of an image capture device, the image capture device can acquire an image corresponding to that facial image, and the acquired image is identical to the facial image to be identified. Then, the acquired facial image to be identified is displayed on an image display, and it can be determined whether the user's gaze point in the facial image to be identified is located within the facial region of the displayed facial image. In this case, the target user is the user in the facial image to be identified provided by that user.

[0126] The above scenario indicates that a malicious attacker obtains a registered user's facial image and attempts to authenticate it. Since the gaze region of the obtained facial image differs from the gaze region of the registered user's facial image during registration, the attacker fails to authenticate the user, thus improving the reliability of facial recognition.

[0127] In other words, if a registered user needs to pass identity verification, they can adjust their gaze point during the verification process so that it is located within the facial region of their face displayed on the image monitor, and that the facial region to which this gaze point belongs is consistent with the facial region to which the user's gaze point belonged during registration. For example, if a user's gaze point is located in the nose region during registration, then during facial recognition, if the user's gaze point is in the nose region, they can pass identity verification; if the user's gaze point is not located in the nose region during facial recognition, such as in the chin region or eye region, then they will fail identity verification.

[0128] In the above process, a preset gaze point determination algorithm can be used to determine whether the user's gaze point is located in the face region of the displayed face image to be recognized. The preset gaze point determination algorithm used in this embodiment can be based on a machine learning algorithm, or it can be implemented based on a deep learning algorithm.

[0129] In one implementation, the preset gaze point determination algorithm includes a gaze estimation algorithm, such as a two-dimensional gaze point estimation algorithm or a three-dimensional gaze point estimation algorithm. In another implementation, the preset gaze point determination algorithm includes a gaze point estimation algorithm; that is, the gaze of the target user in the image to be identified can be determined based on the gaze point estimation algorithm, and then the corresponding gaze point is calculated through spatial geometric transformation. The gaze point estimation algorithm can be a monocular / binocular gaze estimation algorithm, a gaze estimation algorithm based on semantic information, a full-face gaze estimation algorithm, a cross-frame gaze target estimation algorithm, etc.

[0130] In one implementation, if the user's gaze point is not located within the facial region of the currently displayed face image to be recognized, a reminder message can be displayed so that the user can adjust their gaze point after receiving the reminder message. Correspondingly, the image acquisition device can re-acquire a new face image corresponding to the user and display the new face image on an image display, and so on, until it is determined that the user's gaze point is located within the facial region of the currently displayed face image to be recognized.

[0131] For step S102, at least the following two methods can be used to obtain the gaze region to be identified:

[0132] Method 1: In one embodiment, see Figure 2 , Figure 2 This is a flowchart of a second embodiment of the face recognition method provided in this application. Figure 1 Based on this, step S102 above may include the following steps:

[0133] S1021: From the face image to be identified, determine a specified image region centered on the target user's gaze point as the gaze region to be identified.

[0134] The shape and size of the specified image area can be set according to requirements. For example, the specified image area can be a square, an ellipse, or a circle.

[0135] Method 2, in another embodiment, see [link to documentation]. Figure 3 , Figure 3 This is a flowchart of a third embodiment of the face recognition method provided in this application. Figure 1 Based on this, step S102 above may include the following steps:

[0136] S1022: Based on image segmentation, determine the face segmentation image region containing the gaze point of the target user in the face image to be identified, and use it as the gaze region to be identified.

[0137] In this embodiment, the electronic device can perform image segmentation on the face image to be recognized, and determine each segmented image region (i.e., face segmentation image region) in the face image to be recognized. For example, the determined face segmentation image regions can represent the image regions corresponding to each facial organ, such as the image region corresponding to the eyes, the image region corresponding to the nose, the image region corresponding to the mouth, the image region corresponding to the ears, and the image region corresponding to the forehead, etc.

[0138] After determining the segmented image regions of each face, the electronic device can determine the segmented image region containing the gaze point based on the image coordinates of the gaze point, which is then used as the gaze region to be identified.

[0139] Regarding step S103, the obtained face features to be identified are fused with the image features of the face to be identified and the image features of the gaze region to be identified. Accordingly, identification based on the face features to be identified can avoid the limitations of identification based solely on the face image to be identified and improve the reliability of identification.

[0140] In one implementation, features can be extracted from the face image to be identified and the gaze region to be identified separately to obtain their respective image features. Then, the obtained image features are fused to obtain the face features to be identified.

[0141] In one implementation, the face image to be identified and the gaze region to be identified can be fused, and then the fusion result can be used to extract features to obtain the face features to be identified.

[0142] The above processing can be based on machine learning or deep learning, but is not limited to these methods.

[0143] In one embodiment, see Figure 4 , Figure 4 This is a flowchart of a fourth embodiment of the face recognition method provided in this application. Figure 1 Based on this, S103 includes the following steps:

[0144] S1031: Determine the weight of each pixel in the face image to be identified based on the gaze region to be identified.

[0145] S1032: Based on the determined weights and the face image to be identified, feature extraction is performed to obtain the face features to be identified.

[0146] In this embodiment, the weight of a pixel in the image of the face to be identified reflects the correlation between the pixel and the gaze point of the target user. That is, the closer the pixel is to the gaze point of the target user, the stronger the correlation between the pixel and the gaze point of the target user, and the greater the weight of the pixel. Conversely, the farther the pixel is from the gaze point of the target user, the weaker the correlation between the pixel and the gaze point of the target user, and the smaller the weight of the pixel.

[0147] Based on the above processing, the facial features to be identified not only include the image features of the face image but also reflect the gaze point information of the target user. Therefore, recognition based on the extracted facial features can improve the reliability of the recognition process.

[0148] In one embodiment, see Figure 5 , Figure 5This is a flowchart of a fifth embodiment of the face recognition method provided in this application. Figure 4 Based on this, step S1031 above may include the following steps:

[0149] S10311: Obtain the heatmap corresponding to the gaze region to be identified.

[0150] In this process, each point in the heatmap corresponds one-to-one with each pixel in the face image to be identified; each point in the heatmap represents the probability that the corresponding pixel in the face image to be identified is the gaze point of the target user.

[0151] S10312: For each pixel in the face image to be identified, the probability of the corresponding point in the heat map to be identified is used as the weight of the pixel.

[0152] Accordingly, step S1032 above may include the following steps:

[0153] S10321: Input the face image to be identified and the heatmap to be identified into a pre-trained convolutional neural network to obtain the face features to be identified.

[0154] The classification network model, to which the convolutional neural network belongs, is trained based on the first sample face image, the first sample heatmap corresponding to the first sample face image, and the first sample label.

[0155] In this embodiment, after determining the gaze region to be identified, the electronic device can generate its corresponding heatmap (i.e., the heatmap to be identified). Each point in the heatmap to be identified corresponds one-to-one with a pixel in the face image to be identified. That is, the size of the heatmap to be identified is the same as that of the face image to be identified.

[0156] In one implementation, in the heatmap to be identified, the values ​​of all points in the region corresponding to the area to be identified (which can be called the target heatmap region) are greater than 0 and less than or equal to 1. Correspondingly, the values ​​of points in the heatmap to be identified other than the target heatmap region can be 0.

[0157] For example, the values ​​of each point in the target heatmap region follow a Gaussian distribution, and the value of the target point (i.e., the point corresponding to the target user's gaze point in the heatmap to be identified) is 1. In the target heatmap region, for any given point, the farther away the point is from the target point, the smaller the value of that point.

[0158] Each first sample face image can correspond to a first sample heatmap, which is determined based on the user's gaze point in the first sample face image.

[0159] If the user's identity is the same in two first sample face images, and the semantic information of the user's gaze point is the same, then the first sample labels corresponding to the two first sample face images are the same.

[0160] If the users in two first sample face images have different identities, but the semantic information of the users' gaze points is the same, then the first sample labels corresponding to the two first sample face images are different.

[0161] If the users in two first sample face images have different identities and different semantic information about their gaze points, then the first sample labels corresponding to the two first sample face images are different.

[0162] If two first sample face images contain the same user identity but different semantic information about the user's gaze points, then the first sample labels corresponding to the two first sample face images are different.

[0163] The semantic information of a gaze point represents the facial region of the gaze point in the image. For example, if two gaze points are both located in the region corresponding to the mouth, it means that the semantic information of the two gaze points is consistent; if one gaze point is located in the region corresponding to the mouth and the other gaze point is located in the region corresponding to the nose, it means that the semantic information of the two gaze points is inconsistent.

[0164] Based on this, features from the same user with similar semantic information about their gaze points can be brought closer together; features from different users, as well as features from the same user with distant semantic information about their gaze points, can be brought further apart. This means that facial features extracted based on the trained convolutional neural network can not only reflect the user's identity but also the semantic information of the user's gaze points. Consequently, recognition based on the extracted facial features can improve the reliability of the recognition process.

[0165] In one embodiment, see Figure 6 , Figure 6 This is a schematic diagram illustrating the principle of obtaining facial features of a person to be identified based on a convolutional neural network, as provided in an embodiment of this application.

[0166] Figure 6 In the image, the top left corner represents the face image to be identified, and the top right corner represents the heatmap corresponding to the gaze area to be identified. Figure 6 The user's gaze point is located in the region corresponding to the eyes. The elliptical portion in the upper right corner of the heatmap corresponds to the gaze region in the face image to be identified. Correspondingly, the face image and heatmap are input into a face feature extraction model (i.e., the convolutional neural network in this embodiment) to obtain the face features to be identified. Since these face features incorporate the weights of each pixel, they can be called face features with weighted face region information.

[0167] In one embodiment, see Figure 7 , Figure 7 A flowchart of a training method for a convolutional neural network provided in this application embodiment includes the following steps:

[0168] S701, acquire the first sample face image, the first sample heatmap, and the first label.

[0169] S702, input the first sample face image and the first sample heatmap into the convolutional neural network to obtain the first sample face features.

[0170] S703, input the facial features of the first sample into the first classification layer of the classification network model to obtain the first predicted label.

[0171] S704, Calculate the first loss value based on the first predicted label and the first sample label.

[0172] S705: Based on the first loss value, adjust the classification network model to which the convolutional neural network belongs, and continue training until the classification network model converges.

[0173] The first classification layer can be a fully connected layer.

[0174] In one implementation, the difference between the first predicted label and the first label can be calculated based on the cross-entropy loss function to obtain the first loss value, and the classification network model can be adjusted according to the gradient descent method based on the first loss value.

[0175] In one implementation, for every two first sample face images, the similarity between the first predicted labels corresponding to the two first sample face images (which can be called the first predicted similarity) can be calculated, and the similarity between the first sample labels corresponding to the two first sample face images (which can be called the first sample similarity) can be calculated. Then, based on the difference between the first predicted similarity and the first sample similarity, a first loss value can be calculated.

[0176] In one embodiment, see Figure 8 , Figure 8 This is a flowchart of a sixth embodiment of the face recognition method provided in this application. Figure 4 Based on this, S1031 includes the following steps:

[0177] S10313: Divide the face image to be recognized into multiple sub-regions to be recognized;

[0178] S10314: For each sub-region to be identified, calculate the ratio of the portion of the gaze region to be identified that belongs to the sub-region to be identified to the size of the sub-region to be identified, and use it as the weight of each pixel in the sub-region to be identified.

[0179] Accordingly, step S1032 above may include the following steps:

[0180] S10322: Concatenate each sub-region to be identified with its corresponding weight, and input the concatenation results into a pre-trained self-attention network to obtain the facial features to be identified.

[0181] Among them, the classification network model to which the self-attention network belongs is trained based on the second sample face image, the second sample gaze region corresponding to the second sample face image, and the second sample label.

[0182] In this embodiment of the application, the method for determining whether the second sample face image and the second sample gaze region are consistent is similar to the method for determining whether the first sample face image and the first sample gaze region are consistent, and can be referred to the relevant detailed description.

[0183] After determining the gaze region to be identified, the electronic device can divide the face image to be identified into multiple sub-regions (i.e., the sub-regions to be identified in this embodiment). For example, the electronic device can divide the face image to be identified into multiple sub-regions on an average basis. The number of sub-regions to be identified can be determined according to requirements; for example, the number can be determined based on the size of the gaze region to be identified; the size of the gaze region to be identified is negatively correlated with this number.

[0184] Then, for each sub-region to be identified, the portion of the gaze area to be identified that belongs to that sub-region is determined, and the ratio of the size of this portion to the size of the sub-region to be identified is calculated as the weight of each pixel in that sub-region. In other words, each pixel in the same sub-region to be identified has the same weight.

[0185] In this embodiment, the self-attention network can be a Transformer, such as the Vision Transformer.

[0186] See Figure 9 , Figure 9 This is a schematic diagram illustrating the principle of acquiring facial features of a person to be identified based on a self-attention network, as provided in an embodiment of this application.

[0187] Figure 9In the image on the left, the face image to be identified is represented by a grid. The grid in this face image represents the division of the face image, and each resulting image block represents a sub-region to be identified. The elliptical portions in the face image represent the gaze region to be identified. Based on the above division, the weights corresponding to pixels in each sub-region to be identified can be obtained. Then, each sub-region to be identified is concatenated with its corresponding weight, that is, a weighted encoding is performed on each sub-region to be identified and its corresponding weight. Figure 9 In the diagram, rectangles filled with diagonal lines represent sub-regions to be identified. The value preceding each rectangle with a diagonal line indicates the weight of the corresponding pixel within that sub-region.

[0188] Then, the stitched results can be input into a pre-trained self-attention network (including a Transformer Encoder and an MLP Head (Multilayer Perceptron Head)) and the output of the MLP Head can be obtained to get the facial features to be recognized. Since the facial features to be recognized combine the weights of each pixel, they can be called facial features with weighted information on the facial region.

[0189] Based on the above processing, the extracted facial features not only contain the image features of the face image to be identified, but also reflect the gaze point information of the target user. Therefore, recognition based on the extracted facial features can improve the reliability of the recognition process.

[0190] In one embodiment, see Figure 10 , Figure 10 A flowchart of a self-attention network training method provided in this application embodiment includes the following steps:

[0191] S1001, Obtain the second sample face image, the second sample gaze region, and the second sample label.

[0192] S1002, the second sample face image is divided into multiple sample sub-regions.

[0193] S1003, for each sample sub-region, calculate the ratio of the portion of the second sample gaze region belonging to that sample sub-region to the size of that sample sub-region, and use it as the weight of each pixel in that sample sub-region.

[0194] S1004, each sample sub-region is concatenated with its corresponding weight, and the concatenation results are input into the self-attention network in the second classification network model of the initial structure to obtain the second sample face features.

[0195] S1005, the facial features of the second sample are input into the classification layer of the second classification network model of the initial structure to obtain the second predicted label.

[0196] S1006, Calculate the second loss value based on the second predicted label and the second sample label.

[0197] S1007, Based on the second loss value, adjust the model parameters of the second classification network model with the initial structure, and continue training until the second classification network model with the initial structure converges.

[0198] The second classification layer can be a fully connected layer.

[0199] Each second sample face image can correspond to a second sample gaze region, which is determined based on the user's gaze point in the second sample face image.

[0200] If the users in two second sample face images have the same identity and the semantic information of the users' gaze points are the same, then the second sample labels corresponding to the two second sample face images are the same.

[0201] If the users in two second sample face images have different identities, but the semantic information of the users' gaze points is the same, then the second sample labels corresponding to the two second sample face images are different.

[0202] If the users in two second sample face images have different identities and different semantic information about their gaze points, then the second sample labels corresponding to the two second sample face images are different.

[0203] If two second sample face images contain the same user identity but different semantic information about the user's gaze points, then the second sample labels corresponding to the two second sample face images are different.

[0204] The method for training a self-attention network is similar to that for training a convolutional neural network, as described above; please refer to the detailed introduction for further information.

[0205] In this embodiment of the application, the method of segmenting the second sample face image is similar to the method of segmenting the face image to be identified described above, and can be referred to the relevant description.

[0206] In one implementation, the difference between the second predicted label and the second label can be calculated based on the cross-entropy loss function to obtain a second loss value, and the classification network model can be adjusted according to the gradient descent method based on the second loss value.

[0207] Regarding step S104, in one implementation, the similarity between the facial features to be identified and preset facial features can be calculated to obtain the corresponding identification result. If the similarity is not less than a preset threshold, it indicates that the identity authentication has been passed; if the similarity is less than the preset threshold, it indicates that the identity authentication has not been passed.

[0208] Furthermore, in different scenarios, subsequent processing can be performed based on the recognition results. For example, in an access control system, if identity authentication is successful, passage is permitted; otherwise, passage is denied. Registered users represent users with corresponding permissions; for example, in an access control system, registered users are users with access rights, such as company employees. The process of obtaining the registered user's face image is similar to the process of obtaining the face image to be recognized, and relevant descriptions can be found therein.

[0209] In one embodiment, see Figure 11 , Figure 11 This is an example diagram of a face recognition process provided in an embodiment of this application.

[0210] Figure 11 In this process, a user's facial image (i.e., the facial image to be identified in this embodiment) can be acquired and displayed on the screen. Then, based on gaze point estimation, it can be determined whether the user's eyes are focused on the face on the screen (i.e., the facial region in the displayed facial image) when the facial image is acquired. If not, the facial image is acquired again until the user's eyes are focused on the face on the screen when the acquired facial image is displayed.

[0211] Furthermore, the gaze region (i.e., the gaze region to be identified in this embodiment) can be obtained, and facial features can be extracted and compared in conjunction with the face image to be identified. That is, the similarity between the extracted facial features (i.e., the facial features to be identified in this embodiment) and the base database features (i.e., the preset facial features in this embodiment) is calculated, and it is determined whether the similarity exceeds a threshold. If the similarity exceeds the threshold, the corresponding operation is performed (e.g., allowing passage). If the similarity does not exceed the threshold, the process can be terminated.

[0212] Based on the same inventive concept, embodiments of this application also provide a face recognition system, which includes an image acquisition component and a processing component. The image acquisition component is used to acquire a face image as the face image to be recognized. The processing component is used to execute any of the face recognition methods described above.

[0213] The processing component may include a processor. The image acquisition component may include the image acquisition device described in the above embodiments. For example, the image acquisition device may be a camera, which can acquire a user's facial image as the face image to be identified.

[0214] In some embodiments, the system may further include a display component for displaying an image of a face to be identified, and for displaying a reminder message when the target user’s gaze point is not located in the face region of the currently displayed image of the face to be identified, so that the target user can adjust their gaze point after receiving the reminder message.

[0215] For example, the display component may include the image display described in the above embodiments. Accordingly, the image display may display an image of the face to be recognized, thereby determining whether the target user's gaze point is located within the face region of the currently displayed image of the face to be recognized. Furthermore, if the target user's gaze point is not located within the face region of the currently displayed image of the face to be recognized, a reminder message may be displayed on the image display. After viewing the reminder message, the user can adjust their gaze point so that the adjusted gaze point is located within the face region of the currently displayed image of the face to be recognized.

[0216] Based on the same inventive concept as the aforementioned face recognition method, this application also provides a face recognition device, see [link to relevant documentation]. Figure 12 , Figure 12 A structural diagram of a face recognition device provided in this application embodiment is shown. The device may include:

[0217] The first acquisition module 1201 is used to acquire a face image to be recognized; wherein, when displaying the face image to be recognized, the gaze point of the target user in the face image to be recognized is located in the face region of the face image to be recognized;

[0218] The first determining module 1202 is used to determine the image region to which the gaze point of the target user belongs from the face image to be identified according to a preset region determining method, and use it as the gaze region to be identified.

[0219] The feature extraction module 1203 is used to extract features based on the face image to be identified and the gaze region to be identified, so as to obtain the face features to be identified;

[0220] The comparison module 1204 is used to compare the facial features to be identified with preset facial features to obtain the recognition result of the facial image to be identified.

[0221] The preset facial features are obtained by extracting features from the registered user's registered facial image and the registered gaze region to which the registered user's gaze point belongs in the registered facial image; the registered gaze region is determined according to a preset region determination method; when the registered facial image is displayed to the registered user, the registered user's gaze point is located in the specified facial region in the registered facial image.

[0222] Optionally, the first determining module 1202 includes:

[0223] The first gaze region determination submodule is used to determine a specified image region centered on the gaze point of the target user from the face image to be identified, as the gaze region to be identified;

[0224] or,

[0225] The second gaze region determination submodule is used to determine the face segmentation image region containing the gaze point of the target user in the face image to be identified, as the gaze region to be identified.

[0226] Optionally, the feature extraction module 1203 includes:

[0227] The weight determination submodule is used to determine the weight of each pixel in the face image to be identified based on the gaze region to be identified.

[0228] The feature acquisition submodule is used to extract features based on the determined weights and the face image to be identified, thereby obtaining the face features to be identified.

[0229] Optional, the weight determination submodule includes:

[0230] The heatmap acquisition unit is used to acquire the heatmap corresponding to the gaze region to be identified; wherein, each point in the heatmap corresponds one-to-one with each pixel in the face image to be identified; each point in the heatmap represents the probability that the pixel corresponding to that point in the face image to be identified is the gaze point of the target user;

[0231] The first weight acquisition unit is used to determine the probability of the corresponding point in the heat map of the face image to be identified for each pixel in the face image to be identified, and use that probability as the weight of the pixel.

[0232] The feature acquisition submodule is specifically used to input the face image to be identified and the heatmap to be identified into a pre-trained convolutional neural network to obtain the face features to be identified;

[0233] The classification network model, to which the convolutional neural network belongs, is trained based on the first sample face image, the first sample heatmap corresponding to the first sample face image, and the first sample label.

[0234] Optionally, the device further includes:

[0235] The first training module is used to acquire the first sample face image, the first sample heatmap, and the first sample label;

[0236] The first sample face image and the first sample heatmap are input into the convolutional neural network in the first classification network model of the initial structure to obtain the first sample face features;

[0237] The facial features of the first sample are input into the classification layer of the first classification network model with the initial structure to obtain the first predicted label;

[0238] Calculate the first loss value based on the first predicted label and the first sample label;

[0239] The model parameters of the first classification network model with the initial structure are adjusted based on the first loss value, and training continues until the first classification network model with the initial structure converges.

[0240] Optional, the weight determination submodule includes:

[0241] The image segmentation unit is used to divide the face image to be recognized into multiple sub-regions to be recognized;

[0242] The second weight acquisition unit is used to calculate, for each sub-region to be identified, the ratio of the portion of the gaze region to be identified belonging to the sub-region to be identified to the size of the sub-region to be identified, and use it as the weight of each pixel in the sub-region to be identified.

[0243] The feature acquisition submodule is specifically used to concatenate each sub-region to be identified with its corresponding weight, and input each concatenation result into a pre-trained self-attention network to obtain the facial features to be identified.

[0244] Among them, the classification network model to which the self-attention network belongs is trained based on the second sample face image, the second sample gaze region corresponding to the second sample face image, and the second sample label.

[0245] Optionally, the device further includes:

[0246] The second training module is used to obtain the second sample face image, the second sample gaze region, and the second sample label;

[0247] The second sample face image is divided into multiple sample sub-regions;

[0248] For each sample sub-region, the ratio of the portion of the second sample gaze region belonging to that sample sub-region to the size of that sample sub-region is calculated, and used as the weight of each pixel in that sample sub-region.

[0249] Each sample sub-region is concatenated with its corresponding weight, and the concatenation results are input into the self-attention network in the second classification network model of the initial structure to obtain the face features of the second sample.

[0250] The facial features of the second sample are input into the classification layer of the second classification network model with the initial structure to obtain the second predicted label;

[0251] The second loss value is calculated based on the second predicted label and the second sample label;

[0252] The model parameters of the second classification network model with the initial structure are adjusted based on the second loss value, and training continues until the second classification network model with the initial structure converges.

[0253] This application also provides an electronic device, such as... Figure 13 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303, and a communication bus 1304. The processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0254] Memory 1303 is used to store computer programs;

[0255] When processor 1301 executes a program stored in memory 1303, it performs the following steps:

[0256] Acquire the image of the face to be identified;

[0257] According to the preset region determination method, the image region to which the gaze point of the target user belongs is determined from the face image to be identified, and is used as the gaze region to be identified;

[0258] Feature extraction is performed based on the image of the face to be identified and the gaze region to be identified to obtain the features of the face to be identified;

[0259] The facial features to be identified are compared with preset facial features to obtain the recognition result of the facial image to be identified;

[0260] The preset facial features are obtained by extracting features from the registered face image of the registered user and the registered gaze region to which the registered user's gaze point belongs in the registered face image; the registered gaze region is determined according to the preset region determination method; when the registered face image is displayed to the registered user, the registered user's gaze point is located in the designated face region in the registered face image.

[0261] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0262] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0263] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0264] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0265] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described face recognition methods.

[0266] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the face recognition methods described above.

[0267] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0268] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0269] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, systems, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0270] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A face recognition method, characterized in that, The method includes: Acquire a face image to be identified; wherein, when displaying the face image to be identified, the gaze point of the target user in the face image to be identified is located in the face region of the face image to be identified; According to a preset region determination method, the image region to which the gaze point of the target user belongs is determined from the face image to be identified, and is used as the gaze region to be identified; Feature extraction is performed based on the face image to be identified and the gaze region to be identified to obtain the face features to be identified; The facial features to be identified are compared with preset facial features to obtain the recognition result of the facial image to be identified; The preset facial features are obtained by extracting features from the registered face image of the registered user and the registered gaze region to which the registered user's gaze point belongs in the registered face image; the registered gaze region is determined according to the preset region determination method; when the registered face image is displayed to the registered user, the registered user's gaze point is located in the designated face region in the registered face image.

2. The method according to claim 1, characterized in that, The step of determining the image region to which the target user's gaze point belongs from the face image to be identified, according to a preset region determination method, as the gaze region to be identified, includes: From the face image to be identified, a designated image region centered on the gaze point of the target user is determined as the gaze region to be identified. or, Based on image segmentation, the facial segmentation image region containing the gaze point of the target user in the face image to be identified is determined as the gaze region to be identified.

3. The method according to claim 1, characterized in that, The step of extracting features based on the face image to be identified and the gaze region to be identified to obtain the face features to be identified includes: Based on the gaze region to be identified, the weight of each pixel in the face image to be identified is determined; Based on the determined weights and the face image to be identified, feature extraction is performed to obtain the face features to be identified.

4. The method according to claim 3, characterized in that, The step of determining the weight of each pixel in the face image to be identified based on the gaze region to be identified includes: Obtain the heatmap corresponding to the gaze region to be identified; wherein, each point in the heatmap corresponds one-to-one with each pixel in the face image to be identified; each point in the heatmap represents the probability that the pixel corresponding to that point in the face image to be identified is the gaze point of the target user; For each pixel in the face image to be identified, the probability of the corresponding point in the heat map to be identified is used as the weight of the pixel. The step of extracting features based on the determined weights and the face image to be identified, to obtain the face features to be identified, includes: The face image to be identified and the heatmap to be identified are input into a pre-trained convolutional neural network to obtain the face features to be identified. The classification network model to which the convolutional neural network belongs is trained based on the first sample face image, the first sample heatmap corresponding to the first sample face image, and the first sample label.

5. The method according to claim 4, characterized in that, The training process of the convolutional neural network includes the following steps: Obtain the first sample face image, the first sample heatmap, and the first sample label; The first sample face image and the first sample heatmap are input into the convolutional neural network in the first classification network model of the initial structure to obtain the face features of the first sample. The facial features of the first sample are input into the classification layer of the first classification network model of the initial structure to obtain the first predicted label; Calculate the first loss value based on the first predicted label and the first sample label; Based on the first loss value, the model parameters of the first classification network model with the initial structure are adjusted, and training continues until the first classification network model with the initial structure converges.

6. The method according to claim 3, characterized in that, The step of determining the weight of each pixel in the face image to be identified based on the gaze region to be identified includes: The face image to be identified is divided into multiple sub-regions to be identified; For each sub-region to be identified, the ratio of the portion of the gaze region to be identified belonging to that sub-region to the size of that sub-region is calculated, and this ratio is used as the weight of each pixel in that sub-region. The step of extracting features based on the determined weights and the face image to be identified, to obtain the face features to be identified, includes: Each sub-region to be identified is concatenated with its corresponding weight, and the concatenation results are input into a pre-trained self-attention network to obtain the facial features to be identified. The classification network model to which the self-attention network belongs is trained based on the second sample face image, the second sample gaze region corresponding to the second sample face image, and the second sample label.

7. The method according to claim 6, characterized in that, The training process of the self-attention network includes the following steps: Obtain the second sample face image, the second sample gaze region, and the second sample label; The second sample face image is divided into multiple sample sub-regions; For each sample sub-region, the ratio of the portion of the second sample gaze region belonging to that sample sub-region to the size of that sample sub-region is calculated, and used as the weight of each pixel in that sample sub-region. Each sample sub-region is concatenated with its corresponding weight, and the concatenation results are input into the self-attention network in the second classification network model of the initial structure to obtain the face features of the second sample. The facial features of the second sample are input into the classification layer of the second classification network model of the initial structure to obtain the second predicted label; Calculate the second loss value based on the second predicted label and the second sample label; Based on the second loss value, the model parameters of the second classification network model with the initial structure are adjusted, and training continues until the second classification network model with the initial structure converges.

8. A face recognition system, characterized in that, The system includes an image acquisition component and a processing component; The image acquisition component is used to acquire facial images as the facial images to be identified; The processing component is used to perform the steps of the method according to any one of claims 1-7.

9. The system according to claim 8, characterized in that, The system also includes a display component for displaying the face image to be identified, and for displaying a reminder message when the target user's gaze point is not located in the face region of the currently displayed face image to be identified, so that the target user can adjust their gaze point after receiving the reminder message.

10. A face recognition device, characterized in that, The device includes: The first acquisition module is used to acquire a face image to be identified; wherein, when the face image to be identified is displayed, the gaze point of the target user in the face image to be identified is located in the face region of the face image to be identified; The first determining module is used to determine the image region to which the gaze point of the target user belongs from the face image to be identified according to a preset region determining method, and use it as the gaze region to be identified. The feature extraction module is used to extract features based on the face image to be identified and the gaze region to be identified, so as to obtain the face features to be identified; The comparison module is used to compare the facial features to be identified with preset facial features to obtain the recognition result of the facial image to be identified.

Images