Method, device, equipment and storage medium for face recognition and model training

By using feature matching calculation of sample images and base library images and twin network momentum update technology in the face recognition method, the accuracy problem of face model training under small sample data is solved, and efficient face recognition effect is achieved.

CN112507833BActive Publication Date: 2024-12-06BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202011379864.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-30
Publication Date
2024-12-06
Estimated Expiration
2040-11-30

AI Technical Summary

Technical Problem

Existing face recognition methods are difficult to achieve high-precision face model training without a large amount of face data, especially in small sample data, which is difficult to train efficient face recognition models.

Method used

By obtaining sample images and base library images containing different target objects from the training set, the matching calculations between sample image features and base library image features are performed, and the face recognition model is trained based on the matching calculation results, and the twin network and momentum are updated. Method to improve the diversity and accuracy of model training.

Benefits of technology

High-precision face model training is implemented under small sample face data, improving the diversity of face features and recognition accuracy, and ensuring that efficient face recognition models can be trained in the absence of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112507833B_ABST
    Figure CN112507833B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device, equipment and storage medium for face recognition and model training, and relates to computer vision, deep learning and artificial intelligence technologies in image processing. The specific implementation scheme is: by batch acquiring sample images and base library images containing different target objects from the training set during each training, and matching and calculating the sample image features of the sample images with the base library image features of the base library images, and training the first recognition model according to the matching calculation results, it is possible to couple the face features of the sample images and base library images containing different target objects in each training, and to improve the diversity of face features during the training process, so as to realize the training of high-precision face recognition models based on small sample face data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to computer vision, deep learning, and artificial intelligence technologies in image processing, and in particular to a method, apparatus, device, and storage medium for face recognition and model training. Background Art

[0002] Current face recognition methods require large amounts of face data to achieve higher face recognition accuracy. More face data provides more face diversity, and face model training can converge to a better result.

[0003] However, in reality, there is often a lack of large amounts of facial data for model training, and existing face recognition training methods find it difficult to achieve high-precision face model training in the absence of facial data. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for face recognition and model training.

[0005] According to one aspect of the present application, a method for training a face recognition model is provided, comprising:

[0006] Acquire a first number of sample images from a training set, and acquire the first number of base images, wherein the first number of base images and the first number of sample images contain target objects that are not completely identical;

[0007] The sample image features of the sample image are matched with the base image features of the base image, and the first recognition model is trained according to the matching calculation result.

[0008] According to another aspect of the present application, a method for face recognition is provided, comprising:

[0009] Get the face image to be recognized;

[0010] Extracting image features of the face image using a face recognition model, wherein the face recognition model is trained by the face recognition model training method as described above;

[0011] The image features of the face picture are matched with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

[0012] According to another aspect of the present application, a device for training a face recognition model is provided, comprising:

[0013] A training data management module, configured to obtain a first number of sample images from a training set, and obtain the first number of base images, wherein the first number of base images are not completely identical to the target objects contained in the first number of sample images;

[0014] The training module is used to perform matching calculation on the sample image features of the sample image and the base image features of the base image, and train the first recognition model according to the matching calculation result.

[0015] According to another aspect of the present application, a face recognition device is provided, comprising:

[0016] The module for obtaining the image to be identified is used to obtain the face image to be identified;

[0017] A feature extraction module, used to extract image features of the face image using a face recognition model, wherein the face recognition model is trained by the face recognition model training method as described above;

[0018] The recognition module is used to match the image features of the face picture with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

[0019] According to another aspect of the present application, there is provided an electronic device, including:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform any of the methods described above.

[0023] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any of the above methods.

[0024] According to another aspect of the present application, a computer program product is provided, which includes: a computer program, which is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in any one of the above aspects.

[0025] The technology of the present application realizes high-precision face model training with small sample face data.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0028] Figure 1 is a framework diagram of a face recognition system provided in an embodiment of the present application;

[0029] Figure 2 This is a flow chart of the method for training a face recognition model provided in the first embodiment of the present application;

[0030] Figure 3 This is a flow chart of a method for training a face recognition model provided in the second embodiment of the present application;

[0031] Figure 4 A framework diagram of the overall model training process provided in the second embodiment of the present application;

[0032] Figure 5 is a flow chart of a method for face recognition provided in the third embodiment of the present application;

[0033] Figure 6 Schematic diagram of a device for training a face recognition model provided in the fourth embodiment of the present application;

[0034] Figure 7 Schematic diagram of a device for training a face recognition model provided in the fifth embodiment of the present application;

[0035] Figure 8 is a schematic diagram of a face recognition device provided in the sixth embodiment of the present application;

[0036] Fig. 9 is a schematic diagram of another face recognition device provided in the sixth embodiment of the present application;

[0037] Fig.10 is a block diagram of an electronic device used to implement the method for training a face recognition model in an embodiment of the present application;

[0038] Fig.11 It is a block diagram of an electronic device used to implement the face recognition method of an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0040] Current face recognition methods require large amounts of face data to achieve high face recognition accuracy. More face data provides more face diversity, and face recognition model training can converge to a better result. In practical applications, there is often a lack of large amounts of face data for model training. Usually, database face images and captured face images are very easy to obtain. In the absence of face data, it is difficult for existing face recognition model training methods to achieve high-precision face recognition model training based on a small sample of one database face image for each person and one or a small number of captured face images (sample images).

[0041] The present application provides a method, apparatus, device and storage medium for face recognition and model training, which are applied to computer vision, deep learning and artificial intelligence in the field of image processing to train a high-precision face recognition model with small sample face data.

[0042] The method provided in this application can be specifically applied to Figure 1 The face recognition system shown in Figure 1 As shown, the face recognition system includes: a face recognition model training device and a face recognition device, wherein the face recognition model training device is used to train a high-precision face recognition model based on a small sample of training data, and the face recognition device is used to perform face recognition using a trained face recognition model. In specific applications, the face recognition model training device obtains a small sample of training data, and obtains a face recognition model based on the small sample of training data. The face recognition device can obtain a trained face recognition model when needed. When face recognition is required, the face recognition device can obtain an image to be detected, and use the trained face recognition model to perform face recognition on the image to be detected to obtain a recognition result, which can be output through an output device.

[0043] The face recognition device may obtain the image to be detected which may be an image to be detected input by a user, or may receive an image collected and sent in real time by a collection device, which is not specifically limited here.

[0044] In addition, the apparatus for training a face recognition model and the apparatus for face recognition may be located on the same electronic device, or the apparatus for training a face recognition model and the apparatus for face recognition may be located on two different electronic devices respectively.

[0045] Figure 2 1 is a flow chart of the method for training a face recognition model provided in the first embodiment of the present application. Figure 2 As shown, the method trains the first recognition model by looping through the following steps, and the trained first recognition model is used as the final face recognition model:

[0046] Step S101: obtain a first number of sample images from a training set, and obtain a first number of base library images, wherein the first number of base library images and target objects included in the first number of sample images are not completely the same.

[0047] In this embodiment, the training set is a set of training data consisting of a plurality of pre-acquired base image, sample images corresponding to the base image, and label information of the sample images. The label information of the sample image refers to the identification information of the base image corresponding to the pre-labeled sample image (e.g., the ID of the base image), which can represent the correspondence between the sample image and the base image. The base image corresponding to the sample image is consistent with the target object contained in the sample image, that is, the target object contained in the sample image and its corresponding base image is the same person.

[0048] The training set includes multiple base library images, each base library image corresponds to at least one sample image, and can be a small sample training set.

[0049] In one implementation, the method provided in this embodiment can implement high-precision face recognition model training based on a training set with only one base image and one sample image for each target object.

[0050] Base images refer to images that contain the face image of the target object and the identity information of the target object is known. Base images can usually be images such as ID photos that contain clear and complete face images.

[0051] The sample image corresponding to the base image may be a captured image of a face image of a target object contained in the base image.

[0052] When training a model based on a small sample training set, in order to improve the diversity of sample data, sample images and base library images are obtained in batches from the training set according to the size of the training batch value each time training is performed, and the target objects contained in the obtained sample images and base library images are not exactly the same. In this way, the diversity of training data can be increased by coupling the facial features of sample images and base library images containing different target objects during training.

[0053] The first quantity refers to the batch value set during each training, which can be set and adjusted according to the actual application scenario and experience value, and is not specifically limited in this embodiment.

[0054] Step S102: performing matching calculation on the sample image features of the sample image and the base image features of the base image, and training the first recognition model according to the matching calculation result.

[0055] In this embodiment, in order to distinguish the facial features extracted from the sample images and the base library images, the sample image features refer to the facial features extracted from the sample images, and the base library image features refer to the facial features extracted from the base library images.

[0056] During each training, based on the currently selected sample picture, the sample image features of the sample picture are extracted, based on the currently selected base picture containing different target objects from the sample picture, the base picture features of the base picture are extracted, and then the sample image features of the sample picture are matched with the base picture features of the base picture, and the first recognition model is trained according to the matching calculation results, that is, during the training process, the sample image features of the sample picture are coupled with the base picture features of the base picture, and the parameters of the first recognition model are updated according to the coupling results.

[0057] The first recognition model is trained by cyclically executing the above steps until the first recognition model converges, and the converged first recognition model is used as the final face recognition model.

[0058] The embodiment of the present application obtains sample images and base library images that contain target objects that are not completely the same from the training set in batches during each training, performs matching calculations on the sample image features of the sample images and the base library image features of the base library images, and trains the first recognition model based on the matching calculation results. This allows coupling of facial features of sample images and base library images that contain different target objects in each training, improves the diversity of facial features during the training process, and thereby enables high-precision training of facial recognition models based on small sample facial data.

[0059] Figure 3 It is a flow chart of the method for training a face recognition model provided by the second embodiment of the present application. On the basis of the above-mentioned first embodiment, in this embodiment, a first-in-first-out base feature library is used to store base image features of base pictures. In each training process, a second recognition model having a structure consistent with that of the first recognition model is used to extract base image features of base pictures, and the base image features in the base feature library are updated, and the model parameters of the second recognition model are updated according to the model parameters of the first recognition model updated after each training.

[0060] like Figure 3 As shown, the specific steps of this method are as follows:

[0061] Step S201: Obtain a training set, where the training set includes a plurality of base images and sample images of the target object contained in each base image.

[0062] Before training the first recognition model, first obtain the training set used for training. The training set is a collection of training data consisting of multiple pre-acquired base image pictures, sample images corresponding to the base image pictures, and label information of the sample images. Among them, the label information of the sample image refers to the identification information of the base image corresponding to the pre-labeled sample image (such as the ID of the base image), which can represent the correspondence between the sample image and the base image. The base image corresponding to the sample image is consistent with the target object contained in the sample image, that is, the target object contained in the sample image and its corresponding base image is the same person.

[0063] The training set includes multiple base library images, each base library image corresponds to at least one sample image, and can be a small sample training set.

[0064] In one implementation, the method provided in this embodiment can implement high-precision face recognition model training based on a training set with only one base image and one sample image for each target object.

[0065] Base images refer to images that contain the face of the target object and the identity information of the target object is known. Base images are usually ID photos and other images that contain clear and complete face images.

[0066] The sample image corresponding to the base image may be a captured image of a face image of a target object contained in the base image.

[0067] The process of training the first recognition model is to loop through subsequent steps S204-S210 until the first recognition model converges and then terminate the training, and the converged first recognition model is used as the final face recognition model.

[0068] Step S202: Obtain a preset number of base library images from the training set.

[0069] Among them, the preset number is greater than the batch value of obtaining training data during each training. The preset number can be set and adjusted according to the application scenario, and this embodiment does not make specific limitations here.

[0070] Exemplarily, the preset number of base library images can be obtained by randomly extracting from the training set, or can be selected in any other way, which has little impact on the technical effect achieved by this solution.

[0071] Step S203: extracting base image features of a preset number of base pictures using the second recognition model, and adding the base image features of the preset number of base pictures to a base feature library.

[0072] In order to ensure that there are enough base image features in the base feature library during each training, before training the first recognition model, the base image features of a preset number of base pictures are pre-stored in the base feature library through the above steps S202-S203 to ensure that there are enough base image features in the base feature library to provide base image features of base pictures of different target objects during each training.

[0073] In this embodiment, a second recognition model having the same structure as the first recognition model is used to specifically extract base image features of base pictures.

[0074] Among them, the structure of the second recognition model is consistent with that of the first recognition model. Before the first (round) of training, the model parameters of the second recognition model and the first recognition model are initialized respectively, and the initialized model parameter values ​​of the first recognition model and the second recognition model can be consistent or inconsistent. In each subsequent training process, after the model parameters of the first recognition model are updated, the model parameters of the updated first recognition model are used to momentum update the model parameters of the second recognition model, and thereafter the model parameters of the second recognition model and the first recognition model are different.

[0075] Step S204: Acquire a first number of sample images from the training set, and a base library image of the target object included in each sample image.

[0076] During each training, sample images and the base image corresponding to each sample image are obtained in batches from the training set. The base image corresponding to the sample image is the base image that is consistent with the target object contained in the sample image.

[0077] The first quantity refers to the batch value set during each training, which can be set and adjusted according to the actual application scenario and experience value, and is not specifically limited in this embodiment.

[0078] The sample images obtained in this step are used to extract sample image features and participate in the training process.

[0079] The base image corresponding to the sample image obtained in this step is not used for this training, but is used to extract base image features and add them to the base feature library to enrich the base image features in the base feature library.

[0080] Optionally, when obtaining sample images in batches from a training set, the sample images may be obtained in a certain order. Since the number of sample images in a small sample training set is small, it is necessary to cyclically obtain sample images from the training set, and each sample image may be used in multiple training processes.

[0081] In addition, when obtaining sample images in batches from the training set, they can also be randomly selected, or any method of batch obtaining training data in existing model training can be used to obtain them, which is not specifically limited in this embodiment.

[0082] Step S205: extract base image features of a first number of base images from the base feature library.

[0083] In this embodiment, the first number of base image features can be taken out from the base feature library according to the first-in-first-out principle. Since a certain number of base image features are pre-stored in the base feature library, each time training is performed, the base image features that are put in the earliest are extracted from the base feature library according to the first-in-first-out principle, and are used for coupling with the sample image features of the sample images obtained from the training set for model training. During this training, the base image features of the base images corresponding to the sample images obtained in the training set are newly added to the base feature library, and will not be used in this training. In this way, during each training, the base image features taken out from the base feature library are not the base image features of the base images corresponding to the currently obtained sample images, which can ensure that the sample image features used in this training are not completely the same as the target objects corresponding to the base image features, thereby increasing the diversity of facial features in training.

[0084] Optionally, the base database feature library is implemented using a queue data structure, or the base database feature library is implemented using a stack data structure, so that the base database feature library accesses data in a first-in-first-out principle. In addition, the base database feature library can also be implemented using any data structure that can access data in a first-in-first-out manner, which is not specifically limited in this embodiment.

[0085] Step S206: Update the base feature library using the base library images of the target object contained in each sample image.

[0086] In this embodiment, during each training, the base library feature library is updated by extracting the base library image features of the base library image of the target object contained in each sample image and adding the base library image features to the base library feature library. This ensures that there are always enough base library image features in the base library feature library, thereby ensuring the diversity of features in the base library feature library.

[0087] During each training, the earliest base image features are extracted from the base feature library according to the first-in-first-out principle, and are used for coupling with the sample image features of the sample images obtained from the training set for model training. During this training, the base image features of the base images corresponding to the sample images obtained in the training set are newly added to the base feature library, and will not be used in this training. In this way, during each training, the base image features taken out from the base feature library are not the base image features of the base images corresponding to the currently obtained sample images, which can ensure that the sample image features used in this training are not exactly the same as the target objects corresponding to the base image features, thereby increasing the diversity of facial features in training.

[0088] Through the above steps S204-S206, it is possible to obtain a first number of sample images from the training set and a first number of base images, wherein the first number of base images are not completely identical to the target objects contained in the first number of sample images. In this way, during each training, sample images and base images containing target objects that are not completely identical can be obtained in batches from the training set, so that the facial features of the sample images and base images containing different target objects in each training are coupled, which can improve the diversity of facial features during the training process, and realize the training of a high-precision face recognition model based on small sample face data.

[0089] After determining the image features of the sample images used in this training and the base image features of the base images, through steps S207-S208, the sample image features of the sample images can be matched with the base image features of the base images, and the first recognition model can be trained based on the matching calculation results.

[0090] Step S207: updating the weight of the loss function according to the first number of base library image features.

[0091] During the training process, the loss function used may be a classification loss function containing a "W*X" structure, where X represents the sample image features of the sample images used in this training, and W represents the weight of the loss function. W*X performs a mathematical operation on the sample image features and the weights to achieve coupling between the two. For example, the loss function may be softmax, etc.

[0092] In this embodiment, a matching calculation is performed on the sample image features of the sample image and the base image features of the base image, that is, a mathematical operation is performed on the weights in the loss function and the sample image features. The specific data operation depends on the selected loss function.

[0093] Exemplarily, in this step, the first number of base library image features may be used as weights of the loss function.

[0094] Optionally, in this step, the first number of base image features can be used as weights of the loss function after being subjected to a preset operation, which is not specifically limited here. The preset operation can be weighting, normalization, etc.

[0095] Step S208: Based on the updated loss function, the first recognition model is trained using the sample image features of the sample image, and the model parameters of the first recognition model are updated.

[0096] In the current model training process, the weight of the loss function is a parameter learned during the training process. In this embodiment, in order to achieve the coupling of the base image features of the base image and the sample image features of the sample image, the base image features obtained from the base feature library can be used as the weight of the loss function, so that in the process of calculating the loss using the loss function, the matching calculation of the base image features of the base image and the sample image features of the sample image is achieved, and the calculated loss value is used as the result of the matching calculation to supervise the first recognition model to update the model parameters.

[0097] In this embodiment, after obtaining the updated loss function, the process of performing model training and updating model parameters based on the updated loss function in this step is similar to the model training process in the prior art, and will not be repeated in this embodiment.

[0098] After each update of the parameters of the first recognition model, the test set is used to test whether the first recognition model has converged; if the first recognition model has converged, the training is completed and the first recognition model is used as the final face recognition model; if the first recognition model does not converge, the subsequent steps are continued and the first recognition model is trained again until the first recognition model converges.

[0099] Step S209: Update the model parameters of the second recognition model according to the updated model parameters of the first recognition model and the preset momentum value.

[0100] After the first recognition model is trained, the model parameters of the second recognition model are updated by momentum according to the updated model parameters of the first recognition model.

[0101] The first recognition model and the second recognition model have the same structure, and the types of model parameters are also completely the same. Each of the parameters is updated in the same manner through the following formula 1.

[0102] Exemplarily, the parameters of the second recognition model may be updated by momentum according to the following formula 1:

[0103] R2′=m*R2+(1-m)*R1 Formula 1

[0104] Among them, R2′ represents the parameters of the second recognition model after updating, R2 represents the parameters of the second recognition model before updating, R1 represents the parameters of the first recognition model after updating, and m represents the preset momentum value.

[0105] Exemplarily, the preset momentum value may be between 0.8 and 0.99. The preset momentum value may be set and adjusted according to actual application scenarios and empirical values, and is not specifically limited in this embodiment.

[0106] Step S210: extracting base image features of base images of the target object contained in each sample image through the second recognition model, and adding the base image features of the base image of the target object contained in each sample image to the base feature library.

[0107] In this embodiment, the second recognition model used to extract the base image features of the base pictures has the same structure as the first recognition model. The first recognition model and the second recognition model can be regarded as twin networks, that is, the twin network is used to extract facial features of sample pictures and base pictures, and the momentum update method is used to update the model parameters, which can achieve effective model training and improve the reliability and accuracy of the trained face recognition model.

[0108] In this embodiment, using a second recognition model with the same structure as the first recognition model but different model parameters to extract the base image features of the base picture is a possible implementation method. In another implementation of this embodiment, the first recognition model can also be directly used to extract the base image features of the base picture. This embodiment does not make any specific limitations on this.

[0109] Figure 4 A framework diagram of the overall process of model training is provided. Figure 4 The overall process of model training is described as follows: Figure 4 As shown, during the training process, for the currently acquired sample image, the first recognition model is input, and the sample image features of the sample image are extracted through the first recognition model; for the base library image corresponding to the currently acquired sample image, the second recognition model is input, and the base library image features of the base library image are extracted through the second recognition model, and the obtained base library image features are stored in the base library feature library. During training, the base library image features are obtained from the base library feature library according to the first-in-first-out principle, and the weights of the loss function are generated according to the base library image features obtained from the base library feature library. The base library image features obtained from the base library feature library and the sample image features of the sample image are matched and calculated through the loss function, and the parameters of the first recognition model are updated according to the matching calculation results. That is, the loss function is used to supervise the update of the parameters of the first recognition model, and then the parameters of the second recognition model are updated by momentum according to the updated parameters of the first recognition model.

[0110] In an optional embodiment, after training to obtain a converged first recognition model and using the first recognition model as the final face recognition model, the following steps S211-S212 can be used to use the face recognition model to establish a base image feature library for face recognition to provide base features for face recognition.

[0111] Step S211: extracting base image features of all base images through a first recognition model.

[0112] Step S212: Create a base image feature library according to base image features of all base pictures. The base image feature library includes base image features and their corresponding base picture information.

[0113] In this embodiment, the trained first recognition model is used as the final face recognition model. Before face recognition, the trained first recognition model is used to extract the base image features of all base pictures, and a base image feature library is created, which is used to match the face features of the input picture during face recognition, and determine the base pictures that match the input picture.

[0114] The embodiment of the present application stores a preset number of base image features in the base feature library in advance. During each training, the base image features that are put in the earliest are extracted from the base feature library according to the first-in-first-out principle, and are used for coupling with the sample image features of the sample images obtained from the training set for model training. During this training, the base image features of the base images corresponding to the sample images obtained in the training set are newly added to the base feature library, and will not be used in this training. In this way, during each training, the base image features taken out from the base feature library are not the base image features of the base images corresponding to the currently obtained sample images, which can ensure that the sample image features used in this training are not completely the same as the target objects corresponding to the base image features, thereby increasing the diversity of facial features in training. Furthermore, the second recognition model used to extract the base image features of the base pictures has the same structure as the first recognition model. The first recognition model and the second recognition model can be regarded as twin networks, that is, the twin network is used to extract facial features of sample pictures and base pictures, and the momentum update method is used to update the model parameters, which can achieve effective model training and improve the reliability and accuracy of the trained face recognition model.

[0115] Figure 5 : is a flow chart of the face recognition method provided by the third embodiment of the present application. Figure 5 As shown, this method uses a trained face recognition model to perform face recognition. The specific steps of this method are as follows:

[0116] Step S301: Obtain a face image to be recognized.

[0117] The face image to be identified may be a face image input by a user, or may be a face image collected by an image acquisition device, which is not specifically limited in this embodiment.

[0118] For example, the face image to be identified may be a face image of a target person captured by a monitoring system, or may be one or more frames of face images in a monitoring video shot by a monitoring system.

[0119] Step S302: extract image features of the face image using a face recognition model.

[0120] Among them, the face recognition model can be trained by the face recognition model training method provided by the first embodiment or the second embodiment above, and the specific training process will not be repeated here.

[0121] Step S303: Match the image features of the face picture with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

[0122] After using the face recognition model to extract the image features of the face picture, the image features of the face picture are matched with the base image features in the base image feature library to determine the base image features that have the highest matching degree with the image features of the face picture, and the base picture information (such as the base picture ID, etc.) corresponding to the matched base image features is determined to obtain the base picture information corresponding to the face picture.

[0123] After determining the base database picture information corresponding to the face picture, the base database picture corresponding to the face picture and the identity information of the target object contained therein may be further determined according to the base database picture information.

[0124] In another embodiment of the present application, before performing face recognition, a base image feature library can be created using a face recognition model.

[0125] Specifically, the base image features of all base pictures are extracted through the trained face recognition model. According to the base image features of all base pictures, a base image feature library is created, which includes base image features and their corresponding base picture information.

[0126] The trained face recognition model can be applied to at least the following face recognition scenarios:

[0127] A possible scenario is: a suspicious target is found in the surveillance video, but the identity of the target is unknown. The facial image of the suspicious target can be obtained from the surveillance video, and the facial image is used as the image to be identified. The facial features of the facial image are extracted using a face recognition model, and matched with the base image features in the base image feature library to determine the base image information corresponding to the facial image. In this way, the identity information of the suspicious target can be determined based on the identity information of the target object contained in the base image.

[0128] Another possible scenario is: according to user needs, it is necessary to determine whether a specified target appears in the surveillance video collected at a certain time period and location. The base library image of the specified target can be matched with each frame of the surveillance video containing a face image to determine whether there is at least one frame of the surveillance video containing the specified target.

[0129] Specifically, the face recognition model can be used to extract the facial features of the background image of the specified target, and the facial features of each frame of the surveillance video containing a facial image can be extracted. The facial features of the background image of the specified target are matched with the facial features of each frame of the surveillance video containing a facial image, and the images in the surveillance video whose matching degree meets the preset conditions are determined, and these images are output to facilitate further investigation by the user.

[0130] In the embodiments of the present application, the face recognition model trained by the methods provided in the first and second embodiments can be specifically applied to a variety of different application scenarios. The face recognition model is used to create a base library feature library. When performing face recognition, the image features of the face picture to be recognized by the face recognition model are used to match the image features of the face picture with the base library image features in the base library image feature library, and the base library image information corresponding to the face picture is determined to achieve accurate face recognition.

[0131] Figure 6 Schematic diagram of the apparatus for training a face recognition model provided in the fourth embodiment of the present application. The apparatus for training a face recognition model provided in the embodiment of the present application can execute the processing flow provided in the method embodiment of training a face recognition model. Figure 6 As shown, the face recognition model training device 40 includes: a training data management module 401 and a training module 402.

[0132] Specifically, the training data management module 401 is used to obtain a first number of sample images from the training set and obtain a first number of base images, wherein the first number of base images are not completely identical to the target objects included in the first number of sample images.

[0133] The training module 402 is used to perform matching calculations on the sample image features of the sample images and the base image features of the base images, and to train the first recognition model according to the matching calculation results.

[0134] The device provided in the embodiment of the present application can be specifically used to execute the method embodiment provided in the above-mentioned first embodiment, and the specific functions will not be repeated here.

[0135] The embodiment of the present application obtains sample images and base library images that contain target objects that are not completely the same from the training set in batches during each training, performs matching calculations on the sample image features of the sample images and the base library image features of the base library images, and trains the first recognition model based on the matching calculation results. This allows coupling of facial features of sample images and base library images that contain different target objects in each training, improves the diversity of facial features during the training process, and thereby enables high-precision training of facial recognition models based on small sample facial data.

[0136] Figure 7 Schematic diagram of a device for training a face recognition model provided in the fifth embodiment of the present application. Based on the fourth embodiment described above, in the embodiment of the present application, a first-in-first-out base feature library is used to store base image features of base pictures. In each training process, a second recognition model having a structure consistent with that of the first recognition model is used to extract base image features of base pictures, and the base image features in the base feature library are updated, and the model parameters of the second recognition model are updated according to the model parameters of the first recognition model updated after each training.

[0137] In an optional implementation, the training module is further used to:

[0138] The weight of the loss function is updated according to the first number of base library image features; based on the updated loss function, the first recognition model is trained using the sample image features of the sample images, and the model parameters of the first recognition model are updated.

[0139] In an optional implementation, the training data management module is further used to:

[0140] A first number of sample images and base image of the target object included in each sample image are obtained from a training set; base image features of the first number of base images are extracted from a base feature library; and the base feature library is updated using the base image of the target object included in each sample image.

[0141] In an optional implementation, the training data management module is further used to:

[0142] The base image features of the base pictures of the target object contained in each sample picture are extracted through the second recognition model, and the base image features of the base pictures of the target object contained in each sample picture are added to the base feature library; wherein the second recognition model has the same structure as the first recognition model.

[0143] In an optional implementation, the training data management module is further used to:

[0144] The model parameters of the second recognition model are updated according to the updated model parameters of the first recognition model and the preset momentum value.

[0145] In an optional implementation, the training data management module is further used to:

[0146] A preset number of base library pictures are obtained from the training set; base library image features of the preset number of base library pictures are extracted using the second recognition model; and base library image features of the preset number of base library pictures are added to the base library feature library.

[0147] In an optional implementation, the training data management module is further used to:

[0148] According to the first-in-first-out principle, a first number of base image features are taken out from the base feature library.

[0149] In an optional implementation, the base library feature library is implemented using a queue data structure, or the base library feature library is implemented using a stack data structure.

[0150] In an optional implementation, the training data management module is further used to:

[0151] A training set is obtained, where the training set includes multiple base images and sample images of the target object contained in each base image.

[0152] In an optional embodiment, if Figure 7 As shown, the face recognition model training device 40 also includes: a base image feature library creation module 403.

[0153] The base image feature library creation module 403 is used for:

[0154] The base image features of all base pictures are extracted through the first recognition model; based on the base image features of all base pictures, a base image feature library is created, the base image feature library includes base image features and their corresponding base picture information, and the base image feature library is used for face recognition.

[0155] The device provided in the embodiment of the present application can be specifically used to execute the method embodiment provided in the above-mentioned second embodiment, and the specific functions will not be repeated here.

[0156] The embodiment of the present application stores a preset number of base image features in the base feature library in advance. During each training, the base image features that are put in the earliest are extracted from the base feature library according to the first-in-first-out principle, and are used for coupling with the sample image features of the sample images obtained from the training set for model training. During this training, the base image features of the base images corresponding to the sample images obtained in the training set are newly added to the base feature library, and will not be used in this training. In this way, during each training, the base image features taken out from the base feature library are not the base image features of the base images corresponding to the currently obtained sample images, which can ensure that the sample image features used in this training are not completely the same as the target objects corresponding to the base image features, thereby increasing the diversity of facial features in training. Furthermore, the second recognition model used to extract the base image features of the base pictures has the same structure as the first recognition model. The first recognition model and the second recognition model can be regarded as twin networks, that is, the twin network is used to extract facial features of sample pictures and base pictures, and the momentum update method is used to update the model parameters, which can achieve effective model training and improve the reliability and accuracy of the trained face recognition model.

[0157] Figure 8 Schematic diagram of the face recognition device provided in the sixth embodiment of the present application. The face recognition device provided in the embodiment of the present application can execute the processing flow provided in the face recognition method embodiment. Figure 8 As shown, the face recognition device 50 includes: a to-be-recognized picture acquisition module 501, a feature extraction module 502 and a recognition module 503.

[0158] Specifically, the to-be-recognized picture acquisition module 501 is used to acquire a face picture to be recognized.

[0159] The feature extraction module 502 is used to extract image features of face images using a face recognition model.

[0160] The face recognition model is trained by the face recognition model training method provided in the first embodiment or the second embodiment.

[0161] The recognition module 503 is used to match the image features of the face picture with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

[0162] In an optional implementation, before performing face recognition, a base image feature library can be created using a face recognition model. Fig. 9As shown, the face recognition device 50 may also include a base image feature library creation module 504, which is used to: extract base image features of all base images through a face recognition model; create a base image feature library based on the base image features of all base images, and the base image feature library includes base image features and their corresponding base image information.

[0163] The device provided in the embodiment of the present application can be specifically used to execute the method embodiment provided in the third embodiment above, and the specific functions will not be repeated here.

[0164] In the embodiments of the present application, the face recognition model trained by the methods provided in the first and second embodiments can be specifically applied to a variety of different application scenarios. The face recognition model is used to create a base library feature library. When performing face recognition, the image features of the face picture to be recognized by the face recognition model are used to match the image features of the face picture with the base library image features in the base library image feature library, and the base library image information corresponding to the face picture is determined to achieve accurate face recognition.

[0165] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0166] According to an embodiment of the present application, the present application also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the method of face recognition model training in the above method embodiment.

[0167] like Fig.10 , is a block diagram of an electronic device according to a method for training a face recognition model according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0168] like Fig.10As shown, the electronic device includes: one or more processors Y01, a memory Y02, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor Y01 is taken as an example.

[0169] The memory Y02 is the non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the method for training a face recognition model provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method for training a face recognition model provided in the present application.

[0170] The memory Y02 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the method for training the face recognition model in the embodiment of the present application (for example, the attached Figure 6 The processor Y01 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory Y02, that is, the face recognition model training method in the above method embodiment is implemented.

[0171] The memory Y02 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of an electronic device trained according to a face recognition model, etc. In addition, the memory Y02 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory Y02 may optionally include a memory remotely disposed relative to the processor Y01, and these remote memories may be connected to the electronic device trained by the face recognition model via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0172] The electronic device of the face recognition model training method may also include: an input device Y03 and an output device Y04. The processor Y01, the memory Y02, the input device Y03 and the output device Y04 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0173] The input device Y03 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device trained with the face recognition model, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device Y04 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0174] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0175] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0176] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0177] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0178] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0179] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0180] According to an embodiment of the present application, the present application also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the face recognition method in the above method embodiment.

[0181] like Fig.11 As shown in FIG. 1 , it is a block diagram of an electronic device according to a method for face recognition according to an embodiment of the present application. Fig.11As shown, the electronic device includes: one or more processors X01, memory X02, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces, and may also include input devices X03 and output devices X04. The processor X01, memory X02, input device X03 and output device X04 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0182] The electronic device of the face recognition method in this embodiment and the processor X01, memory X02, input device X03 and output device X04 included therein are respectively Fig.10 The electronic device of the face recognition model training method shown and the processor Y01, memory Y02, input device Y03 and output device Y04 included therein are implemented in a similar manner and will not be described in detail in this embodiment.

[0183] The embodiment of the present application stores a preset number of base image features in the base feature library in advance. During each training, the base image features that are put in the earliest are extracted from the base feature library according to the first-in-first-out principle, and are used for coupling with the sample image features of the sample images obtained from the training set for model training. During this training, the base image features of the base images corresponding to the sample images obtained in the training set are newly added to the base feature library, and will not be used in this training. In this way, during each training, the base image features taken out from the base feature library are not the base image features of the base images corresponding to the currently obtained sample images, which can ensure that the sample image features used in this training are not completely the same as the target objects corresponding to the base image features, thereby increasing the diversity of facial features in training. Furthermore, the second recognition model used to extract the base image features of the base pictures has the same structure as the first recognition model. The first recognition model and the second recognition model can be regarded as twin networks, that is, the twin network is used to extract facial features of sample pictures and base pictures, and the momentum update method is used to update the model parameters, which can achieve effective model training and improve the reliability and accuracy of the trained face recognition model.

[0184] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0185] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A method for training a face recognition model, comprising: Acquire a first number of sample images from a training set, and acquire the first number of base images, wherein the first number of base images are not completely identical to target objects included in the first number of sample images, including acquiring a first number of sample images from a training set, and base images of the target objects included in each of the sample images; Retrieving the base image features of the first number of base pictures from the base feature library; Using the base image of the target object contained in each of the sample images, updating the base feature library, including extracting base image features of the base image of the target object contained in each of the sample images through a second recognition model, and adding the base image features of the base image of the target object contained in each of the sample images to the base feature library; Wherein, the structure of the second recognition model is consistent with that of the first recognition model; The sample image features of the sample image are matched with the base image features of the base image, and the first recognition model is trained according to the matching calculation result.

2. The method according to claim 1, wherein: Performing matching calculation on the sample image features of the sample image and the base image features of the base image, and training the first recognition model according to the matching calculation result, including: Updating the weight of the loss function according to the first number of base library image features; Based on the updated loss function, the first recognition model is trained using the sample image features of the sample image, and the model parameters of the first recognition model are updated.

3. The method according to claim 1, wherein: After performing matching calculation on the sample image features of the sample image and the base image features of the base image, and training the first recognition model according to the matching calculation results, the method further includes: The model parameters of the second recognition model are updated according to the updated model parameters of the first recognition model and the preset momentum value.

4. The method according to claim 1, before training the first recognition model, further comprises: Acquire a preset number of base library images from the training set; Extracting base image features of the preset number of base images using the second recognition model; The base image features of the preset number of base pictures are added to the base feature library.

5. The method according to claim 1, wherein: Extracting the base image features of the first number of base pictures from the base feature library includes: According to the first-in-first-out principle, the first number of base image features are taken out from the base feature library.

6. The method according to claim 1, wherein: The base database feature library is implemented by using a queue data structure, or the base database feature library is implemented by using a stack data structure.

7. The method according to claim 1 or 2, wherein: Before training the first recognition model, it also includes: The training set is obtained, where the training set includes a plurality of base images and sample images of the target object contained in each base image.

8. The method according to claim 1 or 2, further comprising: training the first recognition model until the first recognition model meets a convergence condition; Extracting base image features of all base images through the first recognition model; According to the base image features of all the base pictures, a base image feature library is created, wherein the base image feature library includes base image features and their corresponding base picture information, and the base image feature library is used for face recognition.

9. A method for face recognition, comprising: Get the face image to be recognized; Extracting image features of the face image using a face recognition model, wherein the face recognition model is trained by the method according to any one of claims 1 to 8; The image features of the face picture are matched with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

10. The method according to claim 9, further comprising: Extracting base image features of all base images through the face recognition model; According to the base image features of all the base pictures, the base image feature library is created, and the base image feature library includes the base image features and the corresponding base picture information.

11. A device for training a face recognition model, comprising: A training data management module, configured to obtain a first number of sample images from a training set, and obtain the first number of base images, wherein the first number of base images are not completely identical to the target objects contained in the first number of sample images; A training module, used for performing matching calculation on the sample image features of the sample image and the base image features of the base image, and training the first recognition model according to the matching calculation result; Wherein, the training data management module is also used for: Acquire a first number of sample images from a training set, and a base library image of a target object included in each of the sample images; Retrieving the base image features of the first number of base pictures from the base feature library; Using the base database image of the target object contained in each of the sample images, updating the base database feature library; Wherein, the training data management module is also used for: Extracting the base image features of the base picture of the target object contained in each of the sample pictures by the second recognition model, and adding the base image features of the base picture of the target object contained in each of the sample pictures to the base feature library; The structure of the second recognition model is consistent with that of the first recognition model.

12. The device according to claim 11, wherein: The training module is also used to: Updating the weight of the loss function according to the first number of base library image features; Based on the updated loss function, the first recognition model is trained using the sample image features of the sample image, and the model parameters of the first recognition model are updated.

13. The device according to claim 11, wherein: The training data management module is also used for: The model parameters of the second recognition model are updated according to the updated model parameters of the first recognition model and the preset momentum value.

14. The apparatus according to claim 11, wherein the training data management module is further configured to: Acquire a preset number of base library images from the training set; Extracting base image features of the preset number of base images using the second recognition model; The base image features of the preset number of base pictures are added to the base feature library.

15. The device according to claim 11, wherein The training data management module is also used for: According to the first-in-first-out principle, the first number of base image features are taken out from the base feature library.

16. The device according to claim 11, wherein The base database feature library is implemented by using a queue data structure, or the base database feature library is implemented by using a stack data structure.

17. The device according to claim 11 or 12, wherein: The training data management module is also used for: The training set is obtained, where the training set includes a plurality of base images and sample images of the target object contained in each base image.

18. The device according to claim 11 or 12, further comprising: The base image feature library creation module is used to: Extracting base image features of all base images through the first recognition model; According to the base image features of all the base pictures, a base image feature library is created, wherein the base image feature library includes base image features and their corresponding base picture information, and the base image feature library is used for face recognition.

19. A face recognition device, comprising: The module for obtaining the image to be identified is used to obtain the face image to be identified; A feature extraction module, used to extract image features of the face image using a face recognition model, wherein the face recognition model is trained by the method according to any one of claims 1 to 10; The recognition module is used to match the image features of the face picture with the base image features in the base image feature library to determine the base image information corresponding to the face picture.

20. The apparatus according to claim 19, further comprising: The base image feature library creation module is used to: Extracting base image features of all base images through the face recognition model; According to the base image features of all the base pictures, the base image feature library is created, and the base image feature library includes the base image features and the corresponding base picture information.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to cause the computer to execute the method of any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Facial image comparison method and apparatus, computer device, and storage medium

    WO2020015075A1

  • Facial recognition method and apparatus, terminal, and computer readable storage medium

    WO2020037937A1