Face recognition model training method and device, equipment and storage medium

Abstract

The invention provides a face recognition model training method and apparatus, a device and a storage medium. The method comprises the steps of obtaining seed face data; generating synthetic face data according to the seed face data; and training an initial face recognition model through the front character face data and the synthesized face data to obtain a target face recognition model. By applying the method, the synthetic face data can be generated by using less seed face data, the face privacy problem is well avoided, privacy data leakage is avoided, and the precision of the target face recognition model obtained by final training is high.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition, and in particular to a training method, device, equipment and storage medium for a face recognition model. Background Art

[0002] With the development of society, face recognition is being applied in more and more fields. For security reasons, higher face recognition accuracy is required. Therefore, in order to achieve higher recognition accuracy, a large number of face images are often required to train the face recognition algorithm. In addition, face data involves privacy issues, so it is difficult to obtain a large number of face images. And because faces in different regions are quite different, the face recognition model trained based on face images in one region will not work well when used in another region. Summary of the invention

[0003] The present disclosure provides a training method, device, equipment and storage medium for a face recognition model to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a method for training a face recognition model is provided, the method comprising: obtaining seed face data; generating synthetic face data based on the seed face data; training an initial face recognition model using the seed face data and the synthetic face data to obtain a target face recognition model.

[0005] In one possible implementation, after obtaining the seed face data, the method further includes: cleaning the seed face data; detecting the cleaned seed face data using a face detection algorithm; and correcting the cleaned seed face data using a face correction algorithm.

[0006] In one possible implementation, generating synthetic facial data based on the seed facial data includes: organizing the seed facial data to obtain multiple facial data sets; determining a corresponding first facial vector for each seed facial data; obtaining a second facial vector corresponding to the facial data set based on the first facial vector corresponding to the seed facial data contained in the facial data set; inputting the second facial vector into a comparative language-image pre-trained encoder for training to obtain a conditional vector; and generating an image based on the conditional vector to obtain synthetic facial data.

[0007] In one possible implementation, the initial face recognition model is trained using the seed face data and the synthetic face data to obtain a target face recognition model, including: adding the synthetic face data corresponding to the seed face data to the corresponding face data set to obtain a target face data set; preprocessing the face data contained in the target face data set to obtain target face data; and training the initial face recognition model using the target face data to obtain a target face recognition model.

[0008] In one possible implementation, the method further includes: obtaining a face data test set; and testing the target face recognition model using the face data test set to optimize parameters of the target face recognition model.

[0009] According to a second aspect of the present disclosure, a training device for a face recognition model is provided, the device comprising: an acquisition module for acquiring seed face data; a generation module for generating synthetic face data based on the seed face data; and a training module for training an initial face recognition model using the seed face data and the synthetic face data to obtain a target face recognition model.

[0010] In one possible implementation, the device also includes: a processing module, used to clean the seed face data after obtaining the seed face data; detect the cleaned seed face data through a face detection algorithm; and correct the cleaned seed face data through a face correction algorithm.

[0011] In one possible implementation, the generation module includes: an organization submodule, used to organize the seed face data to obtain multiple face data sets; a first determination submodule, used to determine the corresponding first face vector for each seed face data; the first determination submodule is also used to obtain the second face vector corresponding to the face data set based on the first face vector corresponding to the seed face data contained in the face data set; a first training submodule, used to input the second face vector into a comparative language-image pre-trained encoder for training to obtain a conditional vector; and a generation submodule, used to generate an image based on the conditional vector to obtain synthetic face data.

[0012] In one possible implementation, the training module includes: a second determination submodule, used to add the synthetic face data corresponding to the seed face data to the corresponding face data set to obtain a target face data set; a processing submodule, used to pre-process the face data contained in the target face data set to obtain the target face data; and a second training submodule, used to train the initial face recognition model using the target face data to obtain a target face recognition model.

[0013] In one possible implementation manner, the device further includes: a testing module, configured to obtain a face data test set; and train the target face recognition model using the face data test set to optimize parameters of the target face recognition model.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0018] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0019] The present invention discloses a training method, device, equipment and storage medium for a face recognition model, which first obtains seed face data, then generates synthetic face data based on the seed face data, and finally trains the initial face recognition model with the seed face data and the synthetic face data to obtain a target face recognition model. By using this method, synthetic face data can be generated using less seed face data, which better avoids face privacy issues and avoids the leakage of privacy data. The target face recognition model finally trained has a higher accuracy.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0022] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0023] Figure 1 The following is a schematic diagram showing the implementation process of a training method for a face recognition model according to an embodiment of the present disclosure. Figure 1 ;

[0024] Figure 2The following is a schematic diagram showing the implementation process of a training method for a face recognition model according to an embodiment of the present disclosure. Figure 2 ;

[0025] Figure 3 The following is a schematic diagram showing the implementation process of a training method for a face recognition model according to an embodiment of the present disclosure. Figure 3 ;

[0026] Figure 4 A module schematic diagram of a training device for a face recognition model according to an embodiment of the present disclosure is shown;

[0027] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0029] Figure 1 The following is a schematic diagram showing the implementation process of a training method for a face recognition model according to an embodiment of the present disclosure. Figure 1 .

[0030] See also Figure 1 According to the first aspect of the embodiments of the present disclosure, a method for training a face recognition model is provided, the method comprising: step 101, obtaining seed face data; step 102, generating synthetic face data based on the seed face data; step 103, training an initial face recognition model with the seed face data and the synthetic face data to obtain a target face recognition model.

[0031] The training method of the face recognition model of the present application first obtains seed face data, which is a real face image obtained in different scenes, usually obtained by collecting images of the face. For different application scenarios, different methods can be used to obtain seed face data, such as collecting images of the face through a camera. The seed face data includes the entire face of the person. When shooting, the face needs to be collected from the front, and a certain gap is left between the face image and the edge of the image to facilitate subsequent processing. For the same person, multiple images of different angles, different light, and different distances from the collection device can be collected as seed face data.

[0032] Then, synthetic face data is generated from the seed face data. Synthetic face data refers to new face image data generated by fusing different image feature information of the face using certain technical means. Finally, the seed face data and the synthetic face data are input into the initial face recognition model, and the initial face recognition model is trained to obtain the target face recognition model. In this step, the initial face recognition model can also be trained directly with the synthetic face data. In the training process, adding the seed face data can make the training of the initial face recognition model more effective, and the obtained target face recognition model has better recognition effect in the subsequent face recognition process.

[0033] By applying this method, it is possible to use less seed face data to generate synthetic face data, so that the face recognition model can be trained together with the seed face data and the synthetic face data. This can better avoid face privacy issues and prevent the leakage of privacy data. The target face recognition model finally trained has higher accuracy.

[0034] In one possible implementation, after obtaining the seed face data, the method further includes: cleaning the seed face data; detecting the cleaned seed face data using a face detection algorithm; and correcting the cleaned seed face data using a face correction algorithm.

[0035] In order to make the model training effect better, after obtaining the seed face data, the seed face data can be preprocessed first. First, the seed face data is cleaned and processed to delete the blurred and low-quality face data; then the seed face image is optimized, such as image graying, denoising, contrast enhancement, etc. After that, the preprocessed seed face image is detected by the face detection algorithm, the position of the face area in the seed face image is detected, and the face area is intercepted, and then the cleaned seed face data is corrected by the face correction algorithm. Specifically, the key feature points in the face, such as the facial features of a person, are further identified from the detected face area, which can provide local structural information of the face, so it can be used as a key feature point; then the face image is transformed according to the information of the key feature points to align it to the standard position. Since the insight-face software package comes with its own face detection and correction algorithm, this application can complete the detection and correction of seed face data through the insight-face software package.

[0036] In one possible implementation, Figure 2 As shown, generating synthetic face data based on seed face data includes:

[0037] Step 201, sorting the seed face data to obtain multiple face data sets;

[0038] Step 202, for each seed face data, determine the corresponding first face vector;

[0039] Step 203, obtaining a second face vector corresponding to the face data set according to the first face vector corresponding to the seed face data contained in the face data set;

[0040] Step 204, inputting the second face vector into the comparative language-image pre-trained encoder for training to obtain a conditional vector;

[0041] Step 205, generate an image according to the conditional vector to obtain synthetic face data.

[0042] The seed face data is sorted and divided into multiple face data sets according to the corresponding characters. The seed face data contained in each face data set is the face image of the same character, and the seed face data contained in different face data sets are the face images of different characters, and a corresponding unique identifier is assigned to each face data set. For each seed face data, it is input into the arc2face face recognition pre-training model, and the feature extraction of the seed face data is performed to obtain the corresponding first face vector. After each seed face data has obtained the corresponding first face vector, for each face data set, the first face vectors corresponding to all the seed face data contained in it are obtained, and after the multiple first face vectors of the same face data set are averaged, the second face vector corresponding to the face data set is obtained.

[0043] The second face vector is input into the Contrastive Language-Image Pre-Training (CLIP) encoder for training. The CLIP encoder is a neural network encoder trained on image and text pairs. Through contrastive learning, images and texts are embedded in the same vector space. In this space, related images and texts will be close to each other to form tight clusters. The CLIP encoder consists of two core components: an image encoder and a text encoder. The text editor part is used for conditional input of prompt words. In this application, the text encoder of the CLIP encoder uses a fixed prompt word, the picture encoder uses the second face vector, and the text encoder uses a fixed string "apicture of a[picture vector placeholder]people", where the picture vector placeholder corresponds to the second face vector. The above string is input into the CLIP editor as a whole for training to obtain a conditional vector, which combines the string information and the information of the second face vector. The conditional vector is input into the stable-diffusion model, encoded by UNet in the stable diffusion model, and then the image is generated based on stable diffusion to obtain the synthetic face data corresponding to the face image set. The synthetic face data is the approximate face images of different angles and expressions of the person corresponding to the face image set.

[0044] In one possible implementation, Figure 3 As shown, the initial face recognition model is trained by seed face data and synthetic face data to obtain a target face recognition model, including: adding the synthetic face data corresponding to the seed face data to the corresponding face data set to obtain the target face data set; preprocessing the face data contained in the target face data set to obtain the target face data; training the initial face recognition model by the target face data to obtain the target face recognition model.

[0045] Determine the person corresponding to the obtained synthetic face data, add the synthetic face data to the face data set corresponding to the person, and obtain the target face data set. The target face data set contains the seed face data and synthetic face data of the same person. Preprocess the seed face data and synthetic face data contained in the target face data set, which includes cleaning the face image and detecting and correcting the face image to obtain the target face data. Unify the target face data set into the form of image / identification, input the target face data set into the initial face recognition model, namely the arc2face model for training, and obtain the target face recognition model.

[0046] In one possible implementation, the method further includes: obtaining a face data test set; and testing a target face recognition model using the face data test set to optimize parameters of the target face recognition model.

[0047] After obtaining the target face recognition model, the target face recognition model can be tested by first obtaining a face data test set, inputting the face data test set into the target face recognition model, and obtaining a recognition result. The parameters in the target face recognition model can be optimized according to the recognition result, and the first loss between images in the same image set and the second loss between images in different image sets are calculated through the recognition result. The parameters of the target face recognition model are adjusted so that the first loss between images in the same image set is as small as possible, and the second loss between images in different image sets is as large as possible.

[0048] Figure 4 A module schematic diagram of a face recognition model training device according to an embodiment of the present disclosure is shown.

[0049] See also Figure 4 According to the second aspect of the present disclosure, a training device for a face recognition model is provided, the device comprising: an acquisition module 401, used to acquire seed face data; a generation module 402, used to generate synthetic face data based on the seed face data; a training module 403, used to train the initial face recognition model through the seed face data and the synthetic face data to obtain a target face recognition model.

[0050] In one possible implementation, the device also includes: a processing module 404, which is used to clean the seed face data after obtaining the seed face data; detect the cleaned seed face data through a face detection algorithm; and correct the cleaned seed face data through a face correction algorithm.

[0051] In one embodiment, the generation module 402 includes: an organization submodule 4021, which is used to organize the seed face data to obtain multiple face data sets; a first determination submodule 4022, which is used to determine the corresponding first face vector for each seed face data; the first determination submodule 4022 is also used to obtain the second face vector corresponding to the face data set based on the first face vector corresponding to the seed face data contained in the face data set; the first training submodule 4023 is used to input the second face vector into the comparison language-image pre-trained encoder for training to obtain a conditional vector; the generation submodule 4024 is used to generate an image according to the conditional vector to obtain synthetic face data.

[0052] In one embodiment, the training module 403 includes: a second determination submodule 4031, used to add the synthetic face data corresponding to the seed face data to the corresponding face data set to obtain a target face data set; a processing submodule 4032, used to pre-process the face data contained in the target face data set to obtain the target face data; a second training submodule 4033, used to train the initial face recognition model through the target face data to obtain the target face recognition model.

[0053] In one possible implementation, the device further includes: a testing module 405, configured to obtain a face data test set; and train a target face recognition model using the face data test set to optimize parameters of the target face recognition model.

[0054] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0055] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0056] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 800 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0057] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0058] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as a training method for a face recognition model. For example, in some embodiments, a training method for a face recognition model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the training method for a face recognition model described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute a training method for a face recognition model in any other appropriate manner (eg, by means of firmware).

[0059] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0060] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0061] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0062] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0063] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0064] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0065] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0066] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0067] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A training method for a face recognition model, characterized in that: The method comprises: Get seed face data; Generate synthetic face data according to the seed face data; The initial face recognition model is trained using the seed face data and the synthetic face data to obtain a target face recognition model.

2. The method according to claim 1, characterized in that After obtaining the seed face data, the method further includes: Cleaning the seed face data; Detect the cleaned seed face data using a face detection algorithm; The cleaned seed face data is corrected using a face correction algorithm.

3. The method according to claim 1, characterized in that The step of generating synthetic face data according to the seed face data comprises: Arrange the seed face data to obtain multiple face data sets; For each seed face data, determining a corresponding first face vector; Obtaining a second face vector corresponding to the face data set according to a first face vector corresponding to the seed face data contained in the face data set; Inputting the second face vector into a comparative language-image pre-trained encoder for training to obtain a conditional vector; The image is generated according to the conditional vector to obtain synthetic face data.

4. The method according to claim 3, characterized in that The step of training the initial face recognition model by using the seed face data and the synthetic face data to obtain a target face recognition model includes: Adding the synthetic face data corresponding to the seed face data to the corresponding face data set to obtain the target face data set; Preprocessing the face data contained in the target face data set to obtain target face data; The initial face recognition model is trained using the target face data to obtain a target face recognition model.

5. The method according to claim 1, characterized in that The method further comprises: Get the face data test set; The target face recognition model is tested using the face data test set to optimize the parameters of the target face recognition model.

6. A training device for a face recognition model, characterized in that: The device comprises: An acquisition module is used to obtain seed face data; A generation module, used to generate synthetic face data according to the seed face data; The training module is used to train the initial face recognition model using the seed face data and the synthetic face data to obtain a target face recognition model.

7. The device according to claim 6, characterized in that The device also includes: The processing module is used to clean the seed face data after obtaining it; detect the cleaned seed face data through a face detection algorithm; and correct the cleaned seed face data through a face correction algorithm.

8. The device according to claim 6, characterized in that The generating module comprises: A sorting submodule, used for sorting the seed face data to obtain multiple face data sets; A first determination submodule, used to determine a corresponding first face vector for each seed face data; The first determination submodule is further configured to obtain a second face vector corresponding to the face data set according to the first face vector corresponding to the seed face data contained in the face data set; A first training submodule is used to input the second face vector into a comparative language-image pre-trained encoder for training to obtain a conditional vector; The generation submodule is used to generate a picture according to the conditional vector to obtain synthetic face data.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

Images