Face recognition method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a face recognition method and device, electronic equipment and computer readable storage medium, and relate to the fields of artificial intelligence, computer vision technology, intelligent transportation and cloud technology. The method comprises: determining a face feature vector of each sample face image and a class feature vector of each preset class according to a neural network model to be trained, and determining a recognition difficulty level of each sample face image; adjusting model parameters of the neural network model to be trained according to a real label category, the recognition difficulty level, the face feature vector and the class feature vector of each preset class of each sample face image until a training loss value of the neural network model meets a training end condition; and extracting a feature extraction layer in the trained neural network model as a face feature extraction model. According to the face feature extraction model obtained by the method, the face feature vector can be better extracted, so that the face recognition is better, and the practical requirement is met.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence, computers, computer vision technology, intelligent transportation and cloud technology. Specifically, this application relates to a face recognition method, device, electronic device, computer-readable storage medium and computer program product. Background Technology

[0002] With the comprehensive application of various technologies and the development of science and technology, facial recognition has become a popular field of computer research and technology. It belongs to biometric recognition technology and is a computer technology that identifies organisms based on their biological characteristics (generally referring to humans).

[0003] Improving the accuracy and stability of facial recognition systems is a goal pursued and explored by the industry. However, facial recognition models trained using related technologies cannot accurately reflect the true distribution of face categories. When performing facial recognition based on these models, the accuracy is still not ideal and needs further improvement. Summary of the Invention

[0004] This application provides a face recognition method, device, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the accuracy of face recognition.

[0005] According to one aspect of the embodiments of this application, a face recognition method is provided, the method comprising:

[0006] Obtain a sample image set that includes multiple sample face images, and the sample image set includes the true labeled category of each sample face image;

[0007] Multiple sample face images are input into the neural network model to be trained to obtain the face feature vector of each sample face image;

[0008] For each preset category, the category feature vector of each preset category is determined based on the facial feature vector of the sample face images belonging to that preset category in the sample image set;

[0009] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0010] Based on the real labeled categories, recognition difficulty levels, facial feature vectors, and category feature vectors of each preset category of the face images, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition. The feature extraction layer in the trained neural network model is then extracted as the face feature extraction model.

[0011] According to another aspect of the embodiments of this application, a face recognition method is provided, the method comprising:

[0012] Acquire the image of the face to be identified;

[0013] Based on the trained face feature extraction model, the face feature vector of the face image to be identified is obtained, wherein the trained face feature extraction model is trained according to the above method;

[0014] Based on the facial feature vector, the recognition result of the face image to be identified is determined.

[0015] Optionally, based on the facial feature vector, the recognition result of the face image to be recognized is determined, including:

[0016] Determine the first matching degree between the facial feature vector of the face image to be identified and at least one known facial feature vector stored in the face database, and determine the recognition result of the face image to be identified based on the first matching degree.

[0017] or,

[0018] A second matching degree is determined between the facial feature vector of the face image to be identified and the facial feature vector of the target object. Based on the second matching degree, it is determined whether the object to which the face image to be identified belongs is the target object.

[0019] According to another aspect of the embodiments of this application, a face recognition device is provided, the device including a sample image set acquisition module and a model training module.

[0020] The sample image set acquisition module is used to acquire a sample image set including multiple sample face images, wherein the sample image set includes the true labeled category of each sample face image;

[0021] The model training module is used to obtain the facial feature vector of each sample face image after inputting each sample face image in the sample image set into the neural network model to be trained.

[0022] For each preset category, the category feature vector of each preset category is determined based on the facial feature vector of the sample face images belonging to that preset category in the sample image set;

[0023] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0024] Based on the real labeled categories, recognition difficulty levels, facial feature vectors, and category feature vectors of each preset category of the face images, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition. The feature extraction layer in the trained neural network model is then extracted as the face feature extraction model.

[0025] Optionally, for each sample face image, when the model training module determines the recognition difficulty level of the sample face image based on the face feature vector of the sample face image and the category feature vector of at least one preset category, it is specifically used for:

[0026] Determine the first angular interval between the facial feature vector of the sample face image and the category feature vector of the first category, wherein the first category is any of the preset categories;

[0027] Increase the first angle interval according to the set angle interval;

[0028] Before the first angle interval adjustment, the first similarity between the feature vector of the sample face image and the category feature vector of the first category is determined, and the second similarity between the face feature vector of the sample face image and the category feature vector of the first category is determined after the first angle interval adjustment.

[0029] If the first similarity is less than or equal to the second similarity, the recognition difficulty level of the sample face image is determined to be Level 1; if the first similarity is greater than the second similarity, the recognition difficulty level of the sample face image is determined to be Level 2.

[0030] Optionally, the first similarity is the cosine value of the first angular interval, and the second similarity is the cosine value of the adjusted first angular interval.

[0031] Optionally, when the model training module adjusts the model parameters of the neural network model to be trained based on the true labeled categories, recognition difficulty levels, facial feature vectors, and category feature vectors of each preset category for each sample face image, it is specifically used for:

[0032] For each sample face image, the recognition difficulty coefficient of the sample face image is determined according to the recognition difficulty level of the sample face image;

[0033] The training loss value is determined based on the true labeled category, recognition difficulty coefficient, face feature vector, and category feature vector of each preset category of each sample face image.

[0034] If the training loss value does not meet the training termination condition, the model parameters of the neural network model to be trained are adjusted according to the training loss value.

[0035] Optionally, the recognition difficulty level of each sample face image is either level one or level two. When determining the recognition difficulty coefficient of a sample face image based on its recognition difficulty level, the model training module is specifically used for:

[0036] If the recognition difficulty level of the sample face image is level 1, then the recognition difficulty coefficient of the sample face image is set to a set value; if the recognition difficulty level of the sample face image is level 2, then the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category.

[0037] Among them, each non-target category is a category other than the true labeled category of the sample face image in each preset category.

[0038] Optionally, when determining the recognition difficulty coefficient of a sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module is specifically used for:

[0039] Based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, determine the recognition difficulty level of the sample face image relative to each non-target category;

[0040] The non-target category with the corresponding recognition difficulty level of level two is regarded as the second category;

[0041] The recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each second level.

[0042] Optionally, when determining the recognition difficulty level of a sample face image relative to each non-target category based on the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module is specifically used for:

[0043] For each non-target category, determine the second angular interval between the facial feature vector of the sample face image and the category feature vector of the non-target category;

[0044] Increase the second angle interval according to the set angle interval;

[0045] The third similarity between the feature vector of the sample face image and the center vector of the non-target category before the second angle interval adjustment, and the fourth similarity between the feature vector of the sample face image and the center vector of the non-target category after the second angle interval adjustment;

[0046] If the third similarity is greater than the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be level two.

[0047] Optionally, when determining the recognition difficulty coefficient of a sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module is specifically used for:

[0048] The recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vector of each non-target category.

[0049] The recognition difficulty coefficient is adjusted based on the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, wherein the recognition difficulty coefficient is not less than the set value.

[0050] Optionally, when the model training module adjusts the set value according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, it is specifically used for:

[0051] The sum of the recognition difficulty correction coefficient and the set value is determined as the recognition difficulty coefficient of the sample face image.

[0052] Optionally, the training loss value is a training loss value based on the decision boundary margin, which is determined in the following way:

[0053] The recognition difficulty level of each sample face image is used as the decision boundary margin corresponding to that sample face image;

[0054] Based on the true labeled categories of the sample face images, the decision boundary margin, the face feature vector, and the category feature vectors of each preset category, the training loss value based on the decision boundary margin is determined.

[0055] Optionally, the above neural network model includes a classification network layer, and the feature extraction layer and the classification network layer are cascaded in sequence. The model training module is also used for:

[0056] For each sample face image, based on the face feature vector of the sample face image, the predicted category of the sample face image in each preset category is determined through the classification network layer;

[0057] Based on the predicted category corresponding to each sample face image, a first sample set and a second sample set are determined. The first sample set includes sample face images in the sample image set whose true labeled category is the same as the predicted category, and the second sample set consists of sample face images in the sample image set other than the first sample set.

[0058] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category, including:

[0059] For each sample face image in the second sample set, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0060] According to another aspect of the embodiments of this application, a face recognition device is provided, which includes an image acquisition module, a face feature vector extraction module, and a face recognition module.

[0061] The image acquisition module is used to acquire images of the face to be identified.

[0062] The face feature vector extraction module is used to obtain the face feature vector of the face image to be identified based on the trained face feature extraction model. The trained face feature extraction model is trained according to the above method.

[0063] The face recognition module is used to determine the recognition result of the face image to be recognized based on the face feature vector.

[0064] Optionally, when determining the recognition result of the face image to be recognized based on the face feature vector, the face recognition module is specifically used for:

[0065] Determine the first matching degree between the facial feature vector of the face image to be identified and at least one known facial feature vector stored in the face database, and determine the recognition result of the face image to be identified based on the first matching degree.

[0066] or,

[0067] A second matching degree is determined between the facial feature vector of the face image to be identified and the facial feature vector of the target object. Based on the second matching degree, it is determined whether the object to which the face image to be identified belongs is the target object.

[0068] According to another aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0069] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the above-described method.

[0070] According to another aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0071] The beneficial effects of the technical solutions provided in this application are:

[0072] The present application provides a face recognition method, apparatus, electronic device, computer-readable storage medium, and computer program product. During training, for each preset category, the method determines the category feature vector of that preset category based on the face feature vectors of sample face images belonging to that preset category in the sample image set. For each sample face image, the recognition difficulty level of the sample face image is determined based on its facial feature vector and the category feature vector of at least one preset category. Then, based on the true labeled category, recognition difficulty level, facial feature vector, and category feature vector of each preset category for each sample face image, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition. The feature extraction layer in the trained neural network model is then extracted as the face feature extraction model. It is evident that during the training of the neural network model based on the sample image set, the face recognition difficulty of each sample face image and the category feature vector of each preset category can be combined to optimize and adjust the trained neural network model, improving the accuracy of the face feature extraction model in extracting facial feature vectors. This, in turn, improves the accuracy of face recognition when performing face recognition based on these facial feature vectors, better meeting practical needs. Attached Figure Description

[0073] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0074] Figure 1 A flowchart of the face recognition method provided in an embodiment of this application is shown.

[0075] Figure 2 A flowchart of the training operation in the face recognition method provided in the embodiments of this application is shown.

[0076] Figure 3 A schematic diagram of the neural network model during the training process in an embodiment of this application is shown.

[0077] Figure 4A schematic diagram of the face recognition method in an embodiment of this application is shown.

[0078] Figure 5a This diagram illustrates the face recognition model before optimization according to an embodiment of this application.

[0079] Figure 5b This diagram illustrates the optimized face recognition model according to an embodiment of the present application.

[0080] Figure 6 A flowchart of the face recognition method provided in an embodiment of this application is shown.

[0081] Figure 7 A flowchart illustrating the face recognition method provided in an embodiment of this application is shown.

[0082] Figure 8 A schematic diagram of the structure of a face recognition system provided in an embodiment of this application is shown.

[0083] Figure 9 A flowchart of the face recognition system in this application scenario is shown.

[0084] Figure 10 A schematic diagram of a face recognition device provided in an embodiment of this application is shown.

[0085] Figure 11 This diagram illustrates a face recognition system provided in an embodiment of this application.

[0086] Figure 12 A schematic diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0087] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0088] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “A,” or implementation as “A and B.”

[0089] Facial recognition specifically refers to computer technology that uses the analysis and comparison of facial visual features to identify individuals. Optionally, facial recognition can currently be broadly divided into four parts: face detection, face alignment, face verification, and face identification. Face detection involves detecting facial landmarks in an image, locating the face based on these landmarks, and circling the face. Face alignment, based on the detected faces, uses image transformation to align the eyes, nose, mouth, and facial contours to pre-defined, fixed positions. Face verification checks whether two detected faces belong to the same person. Face identification, given a detected face, determines who that face belongs to.

[0090] In related technologies, the number of samples in each category is typically counted, and different training parameters are set based on the number of samples in each category to train the face recognition model, thus solving the problem of imbalanced training samples. However, the accuracy of face recognition models obtained in this way needs to be improved.

[0091] The face recognition method provided in this application, by calling a neural network model, can determine the recognition difficulty level of each sample face image. Based on the real-world labeled category, recognition difficulty level, face feature vector, and category feature vectors of each preset category for each sample face image, the model parameters of the neural network model to be trained are optimized and adjusted. This improves the accuracy of the trained face feature extraction model when extracting face feature vectors, thereby enhancing the recognition accuracy when performing face recognition based on the extracted face feature vectors, better meeting practical needs. The face feature extraction model trained using the method provided in this application can extract face feature vectors that are as close as possible to the feature distribution of a face in the real world, achieving better recognition results and solving the technical problem of inaccurate face recognition due to imbalanced training samples, thus better meeting practical needs.

[0092] Optionally, the face recognition method provided in this application embodiment can be implemented based on Artificial Intelligence (AI) technology. AI is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain optimal results. With the research and progress of AI technology, it has been widely researched and applied in many fields. It is believed that with the development of technology, AI technology will be applied in more fields and play an increasingly important role.

[0093] The aforementioned facial recognition method can be based on a neural network model of Computer Vision (CV). Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, locating, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as image recognition and fingerprint recognition.

[0094] Optionally, the face recognition method provided in this application embodiment can be implemented based on cloud technology. For example, the data computation involved in training the neural network model can be carried out using cloud computing. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize data computation, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology applied based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. Cloud computing technology will become an important support. Cloud computing refers to the delivery and use model of IT infrastructure, which means obtaining the required resources through the network in an on-demand and easily scalable manner; in a broader sense, cloud computing refers to the delivery and use model of services, which means obtaining the required services through the network in an on-demand and easily scalable manner. Such services can be IT and software, Internet-related, or other services. With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the driving force of demands such as search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Unlike previous parallel and distributed computing, the emergence of cloud computing will drive a revolutionary change in the entire Internet model and enterprise management model from a conceptual perspective.

[0095] Optionally, the facial recognition method provided in this application embodiment can also be implemented in the field of intelligent transportation and applied to autonomous driving or traffic applications. For example, the facial recognition method can be used to determine the driver's facial information, and then further determine whether the vehicle being driven is a traffic violation vehicle based on the driver's facial information. Specifically, application scenarios in the field of intelligent transportation may include, but are not limited to, Intelligent Traffic Systems (ITS). Intelligent Traffic Systems, also known as Intelligent Transportation Systems, effectively integrate advanced technologies (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing, strengthening the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0096] Optionally, the face recognition method provided in this application embodiment can also be implemented based on blockchain technology. Specifically, the data used in this face recognition method, such as sample image sets and images of faces to be recognized, can be stored on the blockchain.

[0097] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0098] Figure 1 A flowchart of the face recognition method provided in an embodiment of this application is shown. Figure 2 A flowchart illustrating the training operation in the face recognition method provided in this application embodiment is shown. The executing entity of this face recognition method can be a face recognition device. Optionally, the face recognition device can be, but is not limited to, a terminal device or a server; optionally, the server can be a cloud server. The terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, or wearable device, etc. This face recognition method can also be implemented by a processor calling computer-readable instructions stored in memory.

[0099] Optionally, the aforementioned terminal devices include user terminals, which may include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, in-vehicle terminals, wearable electronic devices, AR / VR devices, etc.

[0100] like Figure 1 As shown, the face recognition method provided in this application embodiment may include steps S10 to S20.

[0101] Step S10: Obtain a sample image set including multiple sample face images. The sample image set includes the true annotation category of each sample face image, wherein the true annotation category is one of multiple preset categories.

[0102] This application does not restrict the method of obtaining sample face images in the sample image set. For example, the sample face images can be face images from the sample image set of existing face recognition models, images downloaded from the Internet, images captured by image acquisition devices, or images extracted from a video.

[0103] Optionally, multiple preset categories can be set based on the differences in faces. Face images within the same preset category (i.e., face images within a category) belong to the same person. Face images in different preset categories (i.e., face images between categories) do not belong to the same person. The true annotation category for each sample face image can be the actual category to which that sample face image belongs. For example, if the face in the sample face image belongs to user A, the true annotation category of the sample face image represents user A. This true annotation category can be represented in the form of a label, which can be numbers, letters, or a combination of numbers and letters, etc. This application does not limit the total number of true annotation categories for multiple sample face images. For example, if the true annotation categories for multiple sample face images include three categories, categories 0, 1, and 2 can be used to represent different people.

[0104] Step S20: Based on the sample image set, repeat the following training operation on the neural network model to be trained until the training loss value of the neural network model to be trained meets the training termination condition, and extract the feature extraction layer in the trained neural network model as the face feature extraction model.

[0105] The training operation in step S20 may include, for example: Figure 2 Steps S21 to S24 shown in the figure are as follows:

[0106] Step S21: Input the face images of each sample in the sample image set into the neural network model to be trained to obtain the face feature vector (i.e. the face representation features) of each sample face image.

[0107] Step S22: For each preset category, determine the category feature vector of each preset category based on the face feature vector of the sample face images belonging to the preset category in the sample image set.

[0108] Step S23: For each sample face image, determine the recognition difficulty level of the sample face image based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0109] Step S24: Adjust the model parameters of the neural network model to be trained based on the real labeled category, recognition difficulty level, face feature vector, and category feature vector of each preset category of the face image of each sample.

[0110] In this embodiment, for each preset category, the category feature vector can also be called the center vector of the category. The specific method for determining the category feature vector of the preset category based on the facial feature vectors of sample face images belonging to the preset category in the sample image set is not limited in this application. Optionally, the mean of the facial feature vectors of sample face images belonging to the preset category in the sample image set can be determined as the category feature vector of the preset category.

[0111] Specifically, for each sample face image, the greater the recognition difficulty of the sample face image, the more difficult it is to recognize, and the higher the probability of misclassification. Therefore, the larger the training loss value determined by the true labeled category, recognition difficulty level, face feature vector, and category feature vectors of each preset category for that sample face image, the greater the penalty imposed on the neural network model. Consequently, the greater the adjustment to the model parameters of the neural network model based on this training loss value.

[0112] Optionally, the training termination condition can be configured according to requirements, and may include, but is not limited to, loss function convergence, loss function value being less than a set value, or training iterations reaching a set number. The smaller this set value, the higher the accuracy of the trained face recognition model.

[0113] In practice, when faced with data exhibiting highly varied distributions, the different similarities between sample face images lead to varying recognition difficulties. That is, the distribution of sample face images within each category is extremely different; some people's faces are unlike any other and are easily distinguishable, while others' faces are very similar and difficult to differentiate. Facial features (i.e., the representational features of a face) are distributed differently in the feature space. This difference stems not only from the number of categories but also from the richness of a particular category and the difficulty of distinguishing that category. For sample face images of similar categories or with many poses within the same category, existing methods typically set the same boundary margin for all categories or adjust the boundary margin based on the number of categories. However, neither of these methods accurately reflects the true distribution of face categories, resulting in less than ideal face recognition results when face recognition models trained using these methods are applied.

[0114] This invention, based on the differences in facial feature distribution, designs a boundary margin and distribution range for each category according to the recognition difficulty of different sample facial images (different categories). Therefore, when the facial recognition model trained in this way performs facial recognition, the recognized facial features are as close as possible to the feature distribution of a face in the real world, thus achieving better recognition results. Based on this, according to the facial recognition method provided in the embodiments of this application, during the training process, for each preset category, the category feature vector of that preset category can be determined based on the facial feature vectors of sample facial images belonging to that preset category in the sample image set. For each sample face image, the recognition difficulty level of the sample face image is determined based on its facial feature vector and the category feature vector of at least one preset category. Then, based on the true labeled category, recognition difficulty level, facial feature vector, and category feature vector of each preset category for each sample face image, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition. The feature extraction layer in the trained neural network model is then extracted as the face feature extraction model. It is evident that in the process of training the neural network model based on the sample image set, the face recognition difficulty of each sample face image and the category feature vector of each preset category can be combined to optimize and adjust the neural network model to be trained, improving the recognition accuracy of the trained face recognition model and the recognition accuracy of the trained face feature extraction model, thus better meeting practical needs.

[0115] Optionally, the above neural network model includes a classification network layer, and the feature extraction layer and the classification network layer are cascaded in sequence. The above method further includes:

[0116] For each sample face image, based on the face feature vector of the sample face image, the predicted category of the sample face image in each preset category is determined through the classification network layer;

[0117] Based on the predicted category corresponding to each sample face image, a first sample set and a second sample set are determined. The first sample set includes sample face images in the sample image set whose true labeled category is the same as the predicted category, and the second sample set consists of sample face images in the sample image set other than the first sample set.

[0118] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category, including:

[0119] For each sample face image in the second sample set, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0120] The classification network layer is used to predict the category of a sample face image based on its facial feature vector. Specifically, for each sample face image, the output of the classification network layer can be the probability that the sample face image belongs to each of the multiple preset categories, including the true labeled category of the sample face image. The specific method for determining the predicted category of the sample face image using the classification network layer is not limited in this embodiment and can be configured according to actual application needs. For example, the preset category corresponding to the highest probability among the multiple probabilities can be used as the predicted category. Alternatively, to better improve the performance of the finally trained face feature extraction model, the preset category corresponding to the highest probability can only be used as the predicted category when the highest probability is greater than or equal to a set probability. If there is no highest probability that meets this condition, the predicted category can be determined to be neither of the preset categories, which can be understood as the predicted category not being the true labeled category of the sample face image.

[0121] The specific value of the above-mentioned probability can be configured according to actual needs. For example, it can be a large value in the range of [0,1], such as 1 or 0.8.

[0122] In this implementation, for each sample face image, after obtaining the face feature vector of the sample face image through the feature extraction layer in the neural network model, if the predicted category of the sample face image is determined to be the same as the true labeled category by the classification network layer in the neural network model, then it can be determined that the face feature vector of the sample face image extracted by the feature extraction layer in the neural network model can well represent the features of the face in the sample face image. Conversely, it can be determined that the face feature vector of the sample face image cannot be accurately extracted by the feature extraction layer in the neural network model, and further adjustments to the model parameters of the neural network model are needed to continue training the neural network model.

[0123] Based on the above, in order to save data and improve training efficiency, step S12 can be performed only on the sample face images in the second sample set (i.e., determining the recognition difficulty level and recognition difficulty of each sample face image, further calculating the training loss value of each sample face image, and further training the neural network model based on each training loss value). Of course, to improve the accuracy of the entire neural network model, step S12 can be performed simultaneously on both the first and second sample sets.

[0124] As an example, if the probability of the true labeled category in the prediction result of a sample face image is the above-mentioned set probability, such as 1 or 0.8, and the corresponding probability of other non-preset categories is 0 or 0.2, this is the ideal situation. It can be determined that when performing face recognition through this neural network model, the face recognition result is without deviation, and there is no need to add a penalty to the neural network model, i.e., calculate the training loss value.

[0125] If the probability of the true labeled category in the predicted result of a sample face image is not the aforementioned set probability, then it can be determined that the face recognition result is biased when using this neural network model, and a penalty needs to be added to the neural network model. Furthermore, the lower the probability of the true labeled category in the predicted result of the sample face image, the greater the bias in the predicted result obtained when using this neural network model for face recognition, the greater the penalty that needs to be added to the neural network model, and the larger the corresponding training loss value.

[0126] Taking a three-class classification problem (class 0, class 1, and class 2) as an example, for a sample face image whose true labeled class (i.e., target class) is class 0,

[0127] If a sample face image with a true labeled class 0 is predicted to have a probability of 0.6 for class 0, 0.3 for class 1, and 0.1 for class 2, the ideal prediction result is a probability of 1 for class 0. Therefore, this prediction result has a bias of 0.4. The above training method is needed to make the probability of class 0 in the prediction result of the neural network model approach 1.

[0128] By using a classification network layer to determine the predicted category of a sample face image among multiple preset categories, a second sample set is formed based on sample face images whose true labeled category differs from the predicted category. This second sample set then determines the face feature vector of each sample face image and the category feature vector of at least one preset category, thus determining the recognition difficulty level of each sample face image. Furthermore, the training loss value for each sample face image is calculated, and the neural network model is further trained based on these training loss values. This approach reduces the amount of data and improves the training efficiency of the entire neural network model.

[0129] Figure 3 A schematic diagram of the neural network model during the training process in an embodiment of this application is shown. Figure 3As shown, the neural network model can include a data layer (i.e., input layer), a feature network layer (i.e., feature extraction layer), and a classification layer (i.e., classification network layer) cascaded in sequence. Based on a large number of sample face images in the sample image set, the neural network model can be iteratively trained according to the above method, and the training loss value (i.e., the value of the model's loss function) corresponding to each training is calculated until the model's training loss value meets the training termination condition or the model's training times reach the set number of training times. The face feature extraction network obtained at this time can be used as the trained face feature extraction model.

[0130] The embodiments of this application do not limit the model structure of the aforementioned neural network model, and it can be constructed according to actual needs. Optionally, the neural network model can be a model based on convolutional neural networks, including but not limited to neural network models based on InceptionNet, ResNet, DenseNet, VggNet, SENet (Squeeze-and-Excitation Networks), etc.

[0131] In this implementation, the data layer is used to receive the input sample face image.

[0132] The feature network layer is used to extract the facial feature vector in each sample face image, so that for each preset category, the category feature vector of the preset category can be determined based on the facial feature vector of the sample face images belonging to the preset category in the sample image set; and for each sample face image, the recognition difficulty level of the sample face image can be determined based on the facial feature vector of the sample face image and the category feature vector of at least one preset category.

[0133] The classification layer is used to classify faces in the input image based on the facial feature vectors extracted by the feature network layer, determining the predicted category of the sample face image among multiple preset categories. The output of the classification layer can be the probability value of the sample face image corresponding to each preset category. The preset category corresponding to the highest probability value among the preset category probabilities in the output can be determined as the predicted category of the sample face image.

[0134] The loss function layer is used to determine the training loss value by considering the ground truth labeling category, recognition difficulty level, facial feature vector, and category feature vectors for each preset category in the face image samples. This training loss value is then sent to the feature network layer to determine whether the training termination condition is met. Specifically, if the training loss value meets the training termination condition, the feature network layer with that condition is used as the face feature extraction model; if the training loss value does not meet the training termination condition, the model parameters of the neural network model are adjusted until the training termination condition is satisfied.

[0135] This application does not limit the specific form of the loss function, and it can be selected according to actual needs. Optionally, the loss function characterizes the accuracy of recognition. For example, the loss function can include any of the following: softmax (normalization) loss function, cosface (cosine loss for deep face) loss function, arcface (arcsine loss for deep face) loss function, center loss, normalface loss function, triplet loss, AM-Softmax loss function, etc. This application does not limit this. For example, this application embodiment can use the AM-Softmax loss function.

[0136] Optionally, before extracting the facial feature vector of the sample face image based on the above feature network layer, it can be determined whether each sample face image contains a face. Figure 4 A schematic diagram of the face recognition method in an embodiment of this application is shown. Figure 4 As shown, after inputting each sample face image into the aforementioned neural network model, face detection can be performed on each sample face image (i.e., the input image). If it is determined that the sample face image contains a face, face features are aligned for the detectable faces to obtain aligned face images (i.e., face pictures). Then, features are extracted from the aligned face image through the aforementioned feature network layer to obtain the face feature vector in the sample face image, which is then used to further perform the aforementioned training operation based on the face feature vectors in each sample face image. If it is determined that a sample face image does not contain a face, it can be considered an invalid image, and the neural network model will no longer be trained based on that sample face image. Instead, other sample face images in the sample image set, excluding that sample face image, are input into the neural network model for training.

[0137] The size of the aligned face in the aligned face image can be determined according to the actual situation; for example, it can be... The facial feature information may include feature information of at least one dimension, and this application does not limit this; for example, the facial feature information may include feature information of 512 dimensions.

[0138] Optionally, for each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category, including:

[0139] Determine the first angular interval between the facial feature vector of the sample face image and the category feature vector of the first category, wherein the first category is any preset category;

[0140] Increase the first angle interval according to the set angle interval;

[0141] Before the first angle interval adjustment, the first similarity between the feature vector of the sample face image and the category feature vector of the first category is determined, and the second similarity between the face feature vector of the sample face image and the category feature vector of the first category is determined after the first angle interval adjustment.

[0142] If the first similarity is less than or equal to the second similarity, the recognition difficulty level of the sample face image is determined to be Level 1; if the first similarity is greater than the second similarity, the recognition difficulty level of the sample face image is determined to be Level 2.

[0143] Optionally, the first similarity is the cosine value of the first angular interval, and the second similarity is the cosine value of the adjusted first angular interval.

[0144] The angle interval can be a fixed value or selected according to actual needs; this application does not impose any restrictions on it. During model training, setting the angle interval can decrease the intra-class distance and increase the inter-class distance for each category. The smaller the intra-class distance and the larger the inter-class distance, the higher the accuracy of face recognition for each sample face image. Optionally, the angle interval can be set as shown in Formula 1 above. The specific value is 0.5.

[0145] If the first similarity is less than or equal to the second similarity, the sample face image is considered relatively easy to recognize, and its recognition difficulty level is classified as Level 1, i.e., a simple sample face image. If the first similarity is greater than the second similarity, the sample face image is considered relatively difficult to recognize, and its recognition difficulty level is classified as Level 2, i.e., a difficult sample face image. The recognition difficulty corresponding to the first level is less than the recognition difficulty corresponding to the second level.

[0146] By setting a predetermined interval and adjusting the first angle interval, the similarity between the facial feature vector of the sample face image and the category feature vector of the first category before and after adjusting the first angle interval can be used to determine the recognition difficulty level of the current sample face image. Then, based on the true labeled category, recognition difficulty level, facial feature vector, and category feature vector of each preset category of the sample face image, the training loss value can be determined, which can better train the above neural network model and obtain a more accurate face feature extraction model.

[0147] Optionally, based on the true labeled category, recognition difficulty level, facial feature vector, and category feature vector of each preset category of the sample face image, the model parameters of the neural network model to be trained are adjusted, including:

[0148] For each sample face image, the recognition difficulty coefficient of the sample face image is determined according to the recognition difficulty level of the sample face image;

[0149] The training loss value is determined based on the true labeled category, recognition difficulty coefficient, face feature vector, and category feature vector of each preset category of each sample face image.

[0150] If the training loss value does not meet the training termination condition, the model parameters of the neural network model to be trained are adjusted according to the training loss value.

[0151] Optionally, the recognition difficulty level of each sample face image is either Level 1 or Level 2. The recognition difficulty coefficient of the sample face image is determined based on its recognition difficulty level, including:

[0152] If the recognition difficulty level of the sample face image is level 1, then the recognition difficulty coefficient of the sample face image is set to a set value; if the recognition difficulty level of the sample face image is level 2, then the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category.

[0153] Among them, each non-target category is a category other than the true labeled category of the sample face image in each preset category.

[0154] Alternatively, the training loss value can be determined according to the following formula 1:

[0155] Formula 1

[0156] In the above formula, The training loss value is represented by N, the total number of face images in the sample image set is represented by s, and s is a fixed scale factor, which is a preset value. The specific value of s can be determined according to the actual situation (e.g., an empirical value). For example, s can be set to 60. n represents the total number of categories of multiple preset categories, i represents the i-th sample face image, y represents the true labeled category of the current sample face image, and j represents the current category of the current sample face image. This represents the similarity between the feature vector of a sample face image and the center vector of the true labeled category of that sample face image. This represents the similarity between the feature vector of a sample face image and the center vector of the non-target category of that sample face image. This indicates the difficulty of recognizing the sample face image. This represents the similarity between the category feature vector of the true labeled category of the adjusted sample face image and the face feature vector of the sample face image. This represents the similarity between the category feature vector of the non-target category of the adjusted sample face image and the face feature vector of the sample face image.

[0157] In practical applications, the set value can be a fixed value or selected according to actual needs. One option is to use... This indicates a set value. For example, you can set... The value is 0.5. Therefore, the corresponding recognition difficulty coefficient is, as shown in Formula 1 above. for: = =0.5. Based on this, it can be determined that the training loss value for sample face images with a recognition difficulty level of the first level is calculated using the following formula 2:

[0158] Formula 2

[0159] By determining different recognition difficulty coefficients based on the recognition difficulty level corresponding to each sample face image, and then determining the training loss value based on the true labeled category, recognition difficulty coefficient, face feature vector, and category feature vector of each preset category of the sample face image, the above neural network model can be trained better, resulting in a more accurate face recognition model.

[0160] Optionally, the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, including:

[0161] The recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vector of each non-target category.

[0162] The recognition difficulty coefficient is adjusted based on the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, wherein the recognition difficulty coefficient is not less than the set value.

[0163] It can be seen that the recognition difficulty coefficient of sample face images with a recognition difficulty level of the second level is greater than that of sample face images with a recognition difficulty level of the first level. That is, the greater the recognition difficulty of the sample face image, the greater the determined training loss value, and correspondingly, the greater the penalty obtained by the neural network model.

[0164] Optionally, the set value is adjusted according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, including:

[0165] The sum of the recognition difficulty correction coefficient and the set value is determined as the recognition difficulty coefficient of the sample face image.

[0166] In this implementation, the similarity between the facial feature vector of the sample face image and the category feature vectors of each non-target category can be the cosine of the angular interval between the facial feature vector of the sample face image and the category feature vectors of each non-target category.

[0167] The recognition difficulty correction coefficient for this sample face image can be the average similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category. The recognition difficulty correction coefficient can be... A= Where 0 < A < 1, For a known parameter, its specific value can be determined based on empirical values; for example, it can be 0.35. This represents the similarity between the facial feature vector of the sample face image and the category feature vector of the current non-target category. `m` represents the index of the non-target category. For multiple preset categories, the method described above is used to determine whether each preset category is a non-target category for the sample face image. When the current preset category is a non-target category for the sample face image, `m` is incremented by 1; this increment has no actual numerical meaning. Based on this, It is represented as the sum of the similarities between the facial feature vector of the sample face image and the category feature vectors of each non-target category. This indicates the number of non-target categories corresponding to the sample face image.

[0168] Assuming the set value is After determining the recognition difficulty correction coefficient, the sum of this set value and the recognition difficulty correction coefficient can be determined as the recognition difficulty coefficient. That is, for a sample face image with a recognition difficulty level of the second level, its recognition difficulty coefficient is... .

[0169] By using the above method, the recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category. Then, the set value is adjusted according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image. The recognition difficulty coefficient of the sample face image with the recognition difficulty level of the second level can be accurately determined.

[0170] Optionally, the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, including:

[0171] Based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, determine the recognition difficulty level of the sample face image relative to each non-target category;

[0172] The non-target category with the corresponding recognition difficulty level of level two is regarded as the second category;

[0173] The recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each second level.

[0174] By comparing the facial feature vector of a sample face image with a recognition difficulty level of level two with the category feature vectors of each non-target category, the recognition difficulty level of the sample face image with a recognition difficulty level of level two is determined relative to each non-target category. Furthermore, based on the similarity between the facial feature vector of the sample face image with a recognition difficulty level of level two and the category feature vectors of each level two category, the recognition difficulty coefficient of the sample face image with a recognition difficulty level of level two can be determined more accurately.

[0175] Optionally, based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, the recognition difficulty level of the sample face image relative to each non-target category is determined, including:

[0176] For each non-target category, determine the second angular interval between the facial feature vector of the sample face image and the category feature vector of the non-target category;

[0177] Increase the second angle interval according to the set angle interval;

[0178] The third similarity between the feature vector of the sample face image and the center vector of the non-target category before the second angle interval adjustment, and the fourth similarity between the feature vector of the sample face image and the center vector of the non-target category after the second angle interval adjustment;

[0179] If the third similarity is greater than the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be level two. Optionally, if the recognition difficulty level of the non-target category is level two, then the non-target category can be determined as the second category.

[0180] If the third similarity is less than or equal to the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be level one. Optionally, if the recognition difficulty level of the non-target category is level one, then the non-target category can be determined as the first category. The recognition difficulty coefficient of a non-target category as the first category is less than the recognition difficulty coefficient of a non-target category as the second category.

[0181] Similarly, the third similarity is the cosine value of the second angular interval, and the fourth similarity is the cosine value of the adjusted second angular interval.

[0182] Assuming the set value is After determining the recognition difficulty correction coefficient, the sum of this set value and the recognition difficulty correction coefficient can be determined as the recognition difficulty coefficient. That is, for a second-level sample face image, its recognition difficulty coefficient is... .in, The index represents the difficult non-target category. For multiple preset categories, it is sequentially determined whether the preset category is a difficult non-target category (i.e., the second category) of the sample face image according to the above method. When the current preset category is a difficult non-target category of the sample face image, ... Increasing the value by 1 has no actual quantitative meaning. Therefore, This is represented as the sum of the similarities between the facial feature vector of the sample face image and the category feature vectors of each non-target category (second category). This indicates the number of difficult non-target categories corresponding to the sample face image.

[0183] In this implementation, when the third similarity is greater than the fourth similarity, the non-target category is determined as the second category. This allows for the determination of the difficult non-target category corresponding to the sample face image. Based on the similarity between the face feature vector of the sample face image and the category feature vector of each second category, the recognition difficulty coefficient of the sample face image can be determined. This allows for a more accurate determination of the recognition difficulty coefficient of the second-level sample face images.

[0184] Based on the above, the recognition difficulty coefficient of the sample face images can be determined as follows:

[0185]

[0186] in, This is used to indicate that the recognition difficulty level of the sample face image is a simple sample image (i.e., level 1). This is used to indicate that the recognition difficulty level of the sample face image is a difficult sample image (i.e., level 2), and the corresponding non-target categories are all in the second category.

[0187] Optionally, the training loss value is a training loss value based on the decision boundary margin, which is determined in the following way:

[0188] The recognition difficulty level of each sample face image is used as the decision boundary margin corresponding to that sample face image;

[0189] Based on the true labeled categories of the sample face images, the decision boundary margin, the face feature vector, and the category feature vectors of each preset category, the training loss value based on the decision boundary margin is determined.

[0190] By using the recognition difficulty of each sample face image as the decision boundary margin for that sample face image, and determining the training loss value based on the ground truth labeling category, decision boundary margin, face feature vector, and category feature vectors of each preset category for each sample face image, the training loss value based on the decision boundary margin is determined. Whether to stop training the neural network model is determined based on whether the training loss value based on the decision boundary margin meets the training termination condition. This allows the recognition difficulty of each sample face image to participate in the optimization and adjustment of model parameters, enabling the model to improve its optimization efforts for difficult-to-recognize sample face images and enhance the model's recognition accuracy.

[0191] Figure 5a This diagram illustrates the face recognition model before optimization according to an embodiment of this application. Figure 5b This diagram illustrates an optimized face recognition model according to an embodiment of this application. Figure 5a and Figure 5bThe diagram illustrates a binary classification scenario (C1 and C2), where C1 represents the true labeled category region, and C2 represents the non-target category region. The region between C1 and C2 represents the inter-class region between the true labeled category and the non-target category. The angle between C1 (i.e., the intra-class angle between the true labeled categories of the sample face image) represents the angular interval between the face feature vector of the sample face image and the category feature vector of the true labeled category of the sample face image. The angle between C2 (i.e., the intra-class angle between the non-target categories of the sample face image) represents the angular interval between the face feature vector of the sample face image and the category feature vector of the non-target category of the sample face image. The angle between C1 and C2 (i.e., the inter-class angle between the true labeled category and the non-target category of the sample face image) represents the angular interval between the category feature vector of the true labeled category and the category feature vector of the non-target category of the sample face image.

[0192] like Figure 5a As shown, before optimizing the face recognition model according to the above method, although the intra-class angle of the true labeled category of the sample face image is adjustable ( Figure 5a (The black solid arrow in the middle) However, the inter-class angle between the true labeled category and the non-target category of the sample face image is not adjustable, making it impossible to further distinguish between intra-class and inter-class categories. That is, when performing face prediction on the face recognition model before optimization based on the above method, it is easy to misclassify the sample face image as a non-target category, resulting in errors.

[0193] like Figure 5b As shown, after optimizing the face recognition model according to the above method, the intra-class angle of the true labeled category of the sample face image can be adjusted. Figure 5b (The black solid arrow in the middle) The inter-class angle between the true labeled category and the non-target category of the sample face image can also be adjusted. Figure 5b (The black dashed arrow in the middle) and can reduce the intra-class angle and increase the inter-class angle, making the facial feature vector of the sample face image closer to the class feature vector of the true labeled class and farther away from the class feature vector of the non-target class. This further distinguishes between intra-class and inter-class features. That is, when the face recognition model optimized according to the above method is used to predict faces, it can avoid misclassifying the sample face image as a non-target class and improve the accuracy of the model for face recognition.

[0194] To facilitate understanding of the face recognition method provided in this application, the following description uses a first sample face image and a second sample face image from an acquired sample image set, with multiple preset categories including three categories: category 0, category 1, and category 2, as an example. The true labeled category of the first sample face image is category 0, and the true labeled category of the second sample face image is category 1. The face recognition method provided in this application is described with reference to Figure 5. Figure 6 A flowchart of the face recognition method provided in an embodiment of this application is shown, as follows: Figure 6 As shown, the method includes:

[0195] Obtain a sample image set, which includes a first sample face image and a second sample face image;

[0196] Each sample face image in the sample image set is input into the neural network model to be trained to obtain the face feature vector of the first sample face image, the face feature vector of the second sample face image, and the category feature vector of each preset category.

[0197] Obtain the predicted category of the first sample face image, wherein the predicted category is: the probability of the first sample face image corresponding to class 0 is 0.8, the probability of corresponding to class 1 is 0.1, and the probability of corresponding to class 2 is 0.1. Through the above method, it can be determined that the first sample face image is a simple sample face image (that is, the recognition difficulty level of the first sample face image is level 1).

[0198] Based on the above method, the recognition difficulty coefficient corresponding to the first sample face image is determined. The first training loss value is set to 0.5, thus determining the first training loss value corresponding to the first sample face image. ;

[0199] The neural network model is trained based on the first training loss value. When the second training loss value of the neural network model does not meet the training termination condition, the model parameters of the neural network model are adjusted until the training loss value of the neural network model meets the training termination condition. The neural network model that meets the training termination condition is taken as the first face recognition model.

[0200] Obtain the predicted category of the second sample face image, where the predicted category is: the probability of the second sample face image corresponding to class 0 is 0.4, the probability of corresponding to class 1 is 0.3, and the probability of corresponding to class 2 is 0.3. Through the above method, it can be determined that the second sample face image is a difficult sample face image (that is, the recognition difficulty level of the second sample face image is level 2).

[0201] Assumption The coefficient is 0.35. Based on the method described above, the recognition difficulty coefficient of the second sample face image is determined. for This allows us to determine the corresponding second training loss value for the second sample face image. ;

[0202] The first face recognition model is trained based on the second training loss value. When the second training loss value of the first face recognition model does not meet the training termination condition, the model parameters of the first face recognition model are adjusted until the second training loss value of the first face recognition model meets the training termination condition. The first face recognition model that meets the training termination condition is then taken as the trained face recognition model.

[0203] This application also provides a face recognition method. Figure 7 A flowchart illustrating the face recognition method provided in an embodiment of this application is shown. Figure 7 As shown, the method includes steps S31 to S33.

[0204] Step S31: Obtain the face image to be identified.

[0205] The face image to be identified can be an image acquired in real time by the image acquisition device of the terminal, or it can be any image; this application does not impose any restrictions on this.

[0206] Step S32: Based on the trained face feature extraction model, obtain the face feature vector of the face image to be identified, wherein the trained face recognition model is trained according to the above method.

[0207] Step S33: Determine the recognition result of the face image to be recognized based on the face feature vector.

[0208] Optionally, based on the facial feature vector, the recognition result of the face image to be recognized is determined, including:

[0209] Determine the first matching degree between the facial feature vector of the face image to be identified and at least one known facial feature vector stored in the face database, and determine the recognition result of the face image to be identified based on the first matching degree.

[0210] or,

[0211] A second matching degree is determined between the facial feature vector of the face image to be identified and the facial feature vector of the target object. Based on the second matching degree, it is determined whether the object to which the face image to be identified belongs is the target object.

[0212] Optionally, the object to which the face image to be identified belongs can be identified if the face feature vector of the face image to be identified matches at least one known face feature vector stored in the face database with a first matching degree greater than or equal to a first matching degree threshold. The first matching degree threshold can be determined based on actual conditions; for example, it can be determined based on empirical values, such as setting the first matching degree threshold to 0.8.

[0213] Optionally, after determining the first matching degree between the facial feature vector of the face image to be identified and at least one known facial feature vector stored in the face database, the object corresponding to the known facial feature vector with the highest first matching degree can be identified as the object to which the face image to be identified belongs.

[0214] Optionally, if the second matching degree is greater than or equal to the second matching degree threshold, the object to which the face image to be identified belongs can be determined as the target object. The second matching degree threshold can be determined based on actual circumstances; for example, it can be determined based on empirical values, such as setting the second matching degree threshold to 0.85.

[0215] To facilitate understanding of the application value of the face recognition method provided in this application embodiment, the following description is based on a specific application scenario.

[0216] Figure 8 This application provides a schematic diagram of the structure of a face recognition system according to an embodiment of the present application. Figure 8 As shown, the system includes a user terminal device 81, a network 82, an application server 83, and a model training server 84. The terminal device 81 communicates with the application server 83 via the network 82. The application server 83 and the model training server 84 can interact; for example, the application server 83 can receive a trained face recognition model sent by the model training server 84. The terminal device 81 may have an application for face recognition installed, or an application within the terminal device 81 may have a plugin for face recognition. By opening the application for face recognition or the application with the aforementioned plugin, the terminal can initiate face recognition. The terminal can be a mobile phone, iPad, computer, etc. For example, during payment or login / registration for a new application, face recognition can be completed through the application for face recognition, thereby completing a mobile payment transaction.

[0217] The model training server 84 can be used to train the neural network model to be trained based on a sample image set, resulting in a trained face feature extraction model. The trained face recognition model can be deployed in the application server 83, which can be used to execute the face recognition method provided in this embodiment. Based on the trained face feature extraction model, it extracts face feature vectors from the image to be recognized, thereby determining whether the face in the image is the target face. For example, if the terminal is a mobile phone and is in a pending payment state, a payment operation is performed when the face in the image to be recognized is determined to be the target face.

[0218] The following is combined with Figure 8The face recognition system shown illustrates the process of face recognition methods in this application scenario. Figure 9 A flowchart of a face recognition system for this application scenario is shown. Figure 9 As shown, the method may include the following steps S41 to S44.

[0219] Step S41: Obtain a sample image set (a large number of sample face images and the true labeled categories of each sample face image).

[0220] Step S42: Model training, which is to obtain a trained face recognition model based on the sample image set.

[0221] Step S43: Obtain the face image to be identified.

[0222] Step S44: Based on the trained face feature extraction model described above, obtain the face feature vector of the face image to be identified;

[0223] Determine the second matching degree between the facial feature vector of the face image to be identified and the facial feature vector of the target object, and determine whether the object to which the face image to be identified belongs is the target object based on the second matching degree;

[0224] If the second matching degree is greater than or equal to the second matching degree threshold, the object to which the face image to be identified belongs is determined to be the target object, and a payment operation is performed.

[0225] Steps S41 and S42 can be implemented by the model training server 84, step S43 can be implemented by the terminal device 81, and step S44 can be implemented by the application server 83.

[0226] Figure 10 A schematic diagram of a face recognition device provided in an embodiment of this application is shown. Figure 10 As shown, the face recognition device 500 includes a sample image set acquisition module 510 and a training module 520.

[0227] The sample image set acquisition module 510 is used to acquire a sample image set, which includes multiple sample face images with real annotation categories, and the real annotation category is one of multiple preset categories;

[0228] The model training module 520 is used to obtain the facial feature vector of each sample face image after inputting each sample face image from the sample image set into the neural network model to be trained.

[0229] For each preset category, the category feature vector of each preset category is determined based on the facial feature vector of the sample face images belonging to that preset category in the sample image set;

[0230] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0231] Based on the real labeled categories, recognition difficulty levels, facial feature vectors, and category feature vectors of each preset category of the face images, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition. The feature extraction layer in the trained neural network model is then extracted as the face feature extraction model.

[0232] Optionally, for each sample face image, when the model training module 520 determines the recognition difficulty level of the sample face image based on the face feature vector of the sample face image and the category feature vector of at least one preset category, it is specifically used for:

[0233] Determine the first angular interval between the facial feature vector of the sample face image and the category feature vector of the first category, wherein the first category is any preset category;

[0234] Increase the first angle interval according to the set angle interval;

[0235] Before the first angle interval adjustment, the first similarity between the feature vector of the sample face image and the category feature vector of the first category is determined, and the second similarity between the face feature vector of the sample face image and the category feature vector of the first category is determined after the first angle interval adjustment.

[0236] If the first similarity is less than or equal to the second similarity, the recognition difficulty level of the sample face image is determined to be Level 1; if the first similarity is greater than the second similarity, the recognition difficulty level of the sample face image is determined to be Level 2.

[0237] Optionally, the first similarity is the cosine value of the first angular interval, and the second similarity is the cosine value of the adjusted first angular interval.

[0238] Optionally, when adjusting the model parameters of the neural network model to be trained based on the true labeled categories, recognition difficulty levels, facial feature vectors, and category feature vectors of each preset category for each sample face image, the model training module 520 is specifically used for:

[0239] For each sample face image, the recognition difficulty coefficient of the sample face image is determined according to the recognition difficulty level of the sample face image;

[0240] The training loss value is determined based on the true labeled category, recognition difficulty coefficient, face feature vector, and category feature vector of each preset category of each sample face image.

[0241] If the training loss value does not meet the training termination condition, the model parameters of the neural network model to be trained are adjusted according to the training loss value.

[0242] Optionally, the recognition difficulty level of each sample face image is either level one or level two. When determining the recognition difficulty coefficient of the sample face image based on its recognition difficulty level, the model training module 520 is specifically used for:

[0243] If the recognition difficulty level of the sample face image is level 1, then the recognition difficulty coefficient of the sample face image is set to a set value; if the recognition difficulty level of the sample face image is level 2, then the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category.

[0244] Among them, each non-target category is a category other than the true labeled category of the sample face image from multiple preset categories.

[0245] Optionally, when determining the recognition difficulty coefficient of the sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module 520 is specifically used for:

[0246] Based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, determine the recognition difficulty level of the sample face image relative to each non-target category;

[0247] The non-target category with the corresponding recognition difficulty level of level two is regarded as the second category;

[0248] The recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each second level.

[0249] Optionally, when determining the recognition difficulty level of the sample face image relative to each non-target category based on the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module 520 is specifically used for:

[0250] For each non-target category, determine the second angular interval between the facial feature vector of the sample face image and the category feature vector of the non-target category;

[0251] Increase the second angle interval according to the set angle interval;

[0252] The third similarity between the feature vector of the sample face image and the center vector of the non-target category before the second angle interval adjustment, and the fourth similarity between the feature vector of the sample face image and the center vector of the non-target category after the second angle interval adjustment;

[0253] If the third similarity is greater than the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be level two.

[0254] Optionally, when determining the recognition difficulty coefficient of the sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module 520 is specifically used for:

[0255] The recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vector of each non-target category.

[0256] The recognition difficulty coefficient is adjusted based on the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, wherein the recognition difficulty coefficient is not less than the set value.

[0257] Optionally, when the model training module 520 adjusts the set value according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, it is specifically used for:

[0258] The sum of the recognition difficulty correction coefficient and the set value is determined as the recognition difficulty coefficient of the sample face image.

[0259] Optionally, the training loss value is a training loss value based on the decision boundary margin, which is determined in the following way:

[0260] The recognition difficulty level of each sample face image is used as the decision boundary margin corresponding to that sample face image;

[0261] Based on the true labeled categories of the sample face images, the decision boundary margin, the face feature vector, and the category feature vectors of each preset category, the training loss value based on the decision boundary margin is determined.

[0262] Optionally, the above neural network model further includes a classification network layer, wherein the feature extraction layer and the classification network layer are cascaded in sequence, and the model training module 520 is further used for:

[0263] For each sample face image, based on the face feature vector of the sample face image, the predicted category of the sample face image in each preset category is determined through the classification network layer;

[0264] Based on the predicted category corresponding to each sample face image, a first sample set and a second sample set are determined. The first sample set includes sample face images in the sample image set whose true labeled category is the same as the predicted category, and the second sample set consists of sample face images in the sample image set other than the first sample set.

[0265] For each sample face image, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category, including:

[0266] For each sample face image in the second sample set, the recognition difficulty level of the sample face image is determined based on the face feature vector of the sample face image and the category feature vector of at least one preset category.

[0267] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0268] This application also provides a face recognition system. Figure 11 A schematic diagram of a face recognition system provided in an embodiment of this application is shown. Figure 11 As shown, the face recognition system 600 includes an image acquisition module 610, a face feature vector extraction module 620, and a face recognition module 630.

[0269] Image acquisition module 610 is used to acquire the face image to be identified;

[0270] The face feature vector extraction module 620 is used to obtain the face feature vector of the face image to be identified based on the trained face feature extraction model. The trained face recognition model is trained according to the above method.

[0271] The face recognition module 630 determines the recognition result of the face image to be recognized based on the face feature vector.

[0272] Optionally, when determining the recognition result of the face image to be recognized based on the face feature vector, the face recognition module 630 is specifically used for:

[0273] Determine the first matching degree between the facial feature vector of the face image to be identified and at least one known facial feature vector stored in the face database, and determine the recognition result of the face image to be identified based on the first matching degree.

[0274] or,

[0275] A second matching degree is determined between the facial feature vector of the face image to be identified and the facial feature vector of the target object, and the object to which the face image to be identified belongs is determined based on the second matching degree.

[0276] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0277] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0278] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0279] Figure 12 This diagram illustrates an electronic device provided in an embodiment of this application. In one optional embodiment, an electronic device is provided, such as… Figure 12 As shown, Figure 12 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0280] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0281] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0282] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0283] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0284] Based on the same principles as the methods provided in the embodiments of this application, the embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in any of the optional embodiments of this application described above.

[0285] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0286] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0287] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0288] The above are merely optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A face recognition method, characterized in that, include: Obtain a sample image set including multiple sample face images, wherein the sample image set includes the true labeled category of each sample face image; Each of the sample face images in the sample image set is input into the neural network model to be trained to obtain the face feature vector of each sample face image; For each preset category, the category feature vector of each preset category is determined based on the face feature vector of the sample face images belonging to that preset category in the sample image set; For each of the sample face images, a first angular interval is determined between the face feature vector of the sample face image and the category feature vector of the first category, wherein the first category is any of the preset categories; According to the set angle interval, the first angle interval is increased; before the first angle interval is adjusted, the first similarity between the sample feature vector of the sample face image and the category feature vector of the first category is determined, and after the first angle interval is adjusted, the second similarity between the face feature vector of the sample face image and the category feature vector of the first category is determined. If the first similarity is less than or equal to the second similarity, the recognition difficulty level of the sample face image is determined to be the first level; if the first similarity is greater than the second similarity, the recognition difficulty level of the sample face image is determined to be the second level. Based on the true labeled category, recognition difficulty level, face feature vector, and category feature vector of each preset category of the sample face image, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition, and the feature extraction layer in the trained neural network model is extracted as the face feature extraction model.

2. The method according to claim 1, characterized in that, The first similarity is the cosine value of the first angular interval, and the second similarity is the cosine value of the adjusted first angular interval.

3. The method according to claim 1 or 2, characterized in that, The adjustment of the model parameters of the neural network model to be trained based on the true labeled category, recognition difficulty level, facial feature vector, and category feature vector of each of the sample face images includes: For each sample face image, the recognition difficulty coefficient of the sample face image is determined according to the recognition difficulty level of the sample face image; The training loss value is determined based on the true labeled category, recognition difficulty coefficient, face feature vector of each sample face image and the category feature vector of each preset category; If the training loss value does not meet the training termination condition, the model parameters of the neural network model to be trained are adjusted according to the training loss value.

4. The method according to claim 3, characterized in that, Each sample face image is assigned a recognition difficulty level of either Level 1 or Level 2. The process of determining the recognition difficulty coefficient of the sample face image based on its recognition difficulty level includes: If the recognition difficulty level of the sample face image is level 1, then the recognition difficulty coefficient of the sample face image is set to a set value; if the recognition difficulty level of the sample face image is level 2, then the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category. Wherein, each non-target category is a category other than the true labeled category of the sample face image in each preset category.

5. The method according to claim 4, characterized in that, The step of determining the recognition difficulty coefficient of the sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category includes: Based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, the recognition difficulty level of the sample face image relative to each of the non-target categories is determined; The non-target category with the corresponding recognition difficulty level of level two is regarded as the second category; The recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each second level.

6. The method according to claim 5, characterized in that, The step of determining the recognition difficulty level of the sample face image relative to each of the non-target categories based on the face feature vector of the sample face image and the category feature vectors of each non-target category includes: For each of the non-target categories, a second angular interval is determined between the facial feature vector of the sample face image and the category feature vector of the non-target category; Increase the second angle interval according to the set angle interval; The third similarity between the feature vector of the sample face image and the center vector of the non-target category before the second angle interval adjustment is determined, and the fourth similarity between the feature vector of the sample face image and the center vector of the non-target category after the second angle interval adjustment is determined. If the third similarity is greater than the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be the second level.

7. The method according to claim 4, characterized in that, The step of determining the recognition difficulty coefficient of the sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category includes: The recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vector of each non-target category. The set value is adjusted according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, wherein the recognition difficulty coefficient is not less than the set value.

8. The method according to claim 7, characterized in that, The step of adjusting the set value according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image includes: The sum of the recognition difficulty correction coefficient and the set value is determined as the recognition difficulty coefficient of the sample face image.

9. The method according to claim 1 or 2, characterized in that, The training loss value is a training loss value based on the decision boundary margin, and the training loss value based on the decision boundary margin is determined in the following way: The recognition difficulty level of each sample face image is used as the decision boundary margin corresponding to that sample face image; Based on the true labeled category, decision boundary margin, face feature vector, and category feature vector of each preset category of the sample face image, the training loss value based on the decision boundary margin is determined.

10. The method according to claim 1 or 2, wherein the neural network model includes a classification network layer, the feature extraction layer and the classification network layer are cascaded sequentially, and the method further includes: For each of the sample face images, based on the face feature vector of the sample face image, the predicted category of the sample face image in each of the preset categories is determined through the classification network layer; Based on the predicted category corresponding to each sample face image, a first sample set and a second sample set are determined. The first sample set includes sample face images in the sample image set whose true labeled category is the same as the predicted category. The second sample set consists of sample face images in the sample image set other than the first sample set. The recognition difficulty level of the sample face images includes: the recognition difficulty level of each sample face image in the second sample set.

11. A face recognition method, characterized in that, include: Acquire the image of the face to be identified; Based on the trained face feature extraction model, the face feature vector of the face image to be identified is obtained, wherein the trained face feature extraction model is trained by the method according to any one of claims 1 to 10; Based on the facial feature vector, the recognition result of the face image to be recognized is determined.

12. A face recognition device, characterized in that, include: The sample image set acquisition module is used to acquire a sample image set including multiple sample face images, wherein the sample image set includes the true labeled category of each sample face image; The model training module is used to obtain the facial feature vector of each of the multiple sample face images in the sample image set after inputting them into the neural network model to be trained. For each preset category, the category feature vector of each preset category is determined based on the face feature vector of the sample face images belonging to that preset category in the sample image set; For each of the sample face images, a first angular interval is determined between the face feature vector of the sample face image and the category feature vector of the first category, wherein the first category is any of the preset categories; According to the set angle interval, the first angle interval is increased; before the first angle interval is adjusted, the first similarity between the sample feature vector of the sample face image and the category feature vector of the first category is determined, and after the first angle interval is adjusted, the second similarity between the face feature vector of the sample face image and the category feature vector of the first category is determined. If the first similarity is less than or equal to the second similarity, the recognition difficulty level of the sample face image is determined to be the first level; if the first similarity is greater than the second similarity, the recognition difficulty level of the sample face image is determined to be the second level. Based on the true labeled category, recognition difficulty level, face feature vector, and category feature vector of each preset category of the sample face image, the model parameters of the neural network model to be trained are adjusted until the training loss value of the neural network model meets the training termination condition, and the feature extraction layer in the trained neural network model is extracted as the face feature extraction model.

13. The apparatus according to claim 12, characterized in that, The first similarity is the cosine value of the first angular interval, and the second similarity is the cosine value of the adjusted first angular interval.

14. The apparatus according to claim 12 or 13, characterized in that, When the model training module adjusts the model parameters of the neural network model to be trained based on the true labeled category, recognition difficulty level, face feature vector, and category feature vector of each preset category of the sample face image, it is specifically used for: For each sample face image, the recognition difficulty coefficient of the sample face image is determined according to the recognition difficulty level of the sample face image; The training loss value is determined based on the true labeled category, recognition difficulty coefficient, face feature vector of each sample face image and the category feature vector of each preset category; If the training loss value does not meet the training termination condition, the model parameters of the neural network model to be trained are adjusted according to the training loss value.

15. The apparatus according to claim 14, characterized in that, Each sample face image is assigned a recognition difficulty level of either Level 1 or Level 2. When determining the recognition difficulty coefficient of a sample face image based on its recognition difficulty level, the model training module is specifically used for: If the recognition difficulty level of the sample face image is level 1, then the recognition difficulty coefficient of the sample face image is set to a set value; if the recognition difficulty level of the sample face image is level 2, then the recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category. Wherein, each non-target category is a category other than the true labeled category of the sample face image in each preset category.

16. The apparatus according to claim 15, characterized in that, When determining the recognition difficulty coefficient of a sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module is specifically used for: Based on the facial feature vector of the sample face image and the category feature vectors of each non-target category, the recognition difficulty level of the sample face image relative to each of the non-target categories is determined; The non-target category with the corresponding recognition difficulty level of level two is regarded as the second category; The recognition difficulty coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vectors of each second level.

17. The apparatus according to claim 16, characterized in that, When the model training module determines the recognition difficulty level of the sample face image relative to each of the non-target categories based on the face feature vector of the sample face image and the category feature vectors of each non-target category, it is specifically used for: For each of the non-target categories, a second angular interval is determined between the facial feature vector of the sample face image and the category feature vector of the non-target category; Increase the second angle interval according to the set angle interval; The third similarity between the feature vector of the sample face image and the center vector of the non-target category before the second angle interval adjustment is determined, and the fourth similarity between the feature vector of the sample face image and the center vector of the non-target category after the second angle interval adjustment is determined. If the third similarity is greater than the fourth similarity, the recognition difficulty level of the sample face image relative to the non-target category is determined to be the second level.

18. The apparatus according to claim 15, characterized in that, When determining the recognition difficulty coefficient of a sample face image based on the similarity between the face feature vector of the sample face image and the category feature vectors of each non-target category, the model training module is specifically used for: The recognition difficulty correction coefficient of the sample face image is determined based on the similarity between the face feature vector of the sample face image and the category feature vector of each non-target category. The set value is adjusted according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, wherein the recognition difficulty coefficient is not less than the set value.

19. The apparatus according to claim 18, characterized in that, When the model training module adjusts the set value according to the recognition difficulty correction coefficient to obtain the recognition difficulty coefficient of the sample face image, it is specifically used for: The sum of the recognition difficulty correction coefficient and the set value is determined as the recognition difficulty coefficient of the sample face image.

20. The apparatus according to claim 12 or 13, characterized in that, The training loss value is a training loss value based on the decision boundary margin, and the training loss value based on the decision boundary margin is determined in the following way: The recognition difficulty level of each sample face image is used as the decision boundary margin corresponding to that sample face image; Based on the true labeled category, decision boundary margin, face feature vector, and category feature vector of each preset category of the sample face image, the training loss value based on the decision boundary margin is determined.

21. The apparatus according to claim 12 or 13, wherein the neural network model includes a classification network layer, the feature extraction layer and the classification network layer are cascaded in sequence, and the model training module is further configured to: For each of the sample face images, based on the face feature vector of the sample face image, the predicted category of the sample face image in each of the preset categories is determined through the classification network layer; Based on the predicted category corresponding to each sample face image, a first sample set and a second sample set are determined. The first sample set includes sample face images in the sample image set whose true labeled category is the same as the predicted category. The second sample set consists of sample face images in the sample image set other than the first sample set. in, The recognition difficulty level of the sample face images includes: the recognition difficulty level of each sample face image in the second sample set.

22. A face recognition device, characterized in that, include: The image acquisition module is used to acquire images of the face to be identified. A face feature vector extraction module is used to obtain the face feature vector of the face image to be identified based on a trained face feature extraction model; wherein the trained face feature extraction model is trained by the method according to any one of claims 1 to 10; The face recognition module determines the recognition result of the face image to be recognized based on the face feature vector.

23. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-10, or to implement the steps of the method according to claim 11.

24. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-10, or the steps of the method described in claim 11.

25. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1-10, or the steps of the method described in claim 11.

Images