Face generation method, face recognition method and device

Abstract

The present disclosure relates to a face generation method, a face recognition method and a device. The above-mentioned face generation method includes determining at least one reference face image; calculating the average pixel value of the corresponding pixel points in each of the above-mentioned reference face images to obtain a target face image; performing feature extraction on each of the above-mentioned reference face images to obtain feature information corresponding to the above-mentioned reference face images; calculating the average value of the feature information corresponding to each of the above-mentioned reference face images to obtain supervision information; performing the above-mentioned feature extraction on the above-mentioned target face image to obtain feature information of the target face; based on the difference between the feature information of the above-mentioned target face and the above-mentioned supervision information, correcting the above-mentioned target face image to obtain an average face image. The present disclosure can obtain a high-quality average face image, reduce the dependence on the number of reference face images, and even if a small number of reference face images are used, a high-quality average face image can still be generated.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a face generation method, a face recognition method and a device. Background Art

[0002] With the development of image processing technology, more and more applications based on image processing are being used. In many application scenarios, it is necessary to synthesize an average face image based on existing reference face images. However, the realism of the average face image synthesized currently is average, and there may be problems such as face deformation. In addition, the quality of the average face image is greatly affected by the number of reference face images. When the amount of reference face image data is small, the quality of the average face image decreases significantly. Summary of the invention

[0003] In order to solve at least one of the above-mentioned technical problems, the present disclosure proposes a face generation method, a face recognition method and a device.

[0004] According to one aspect of the present disclosure, a face generation method is provided, which includes: determining at least one reference face image; calculating the average pixel value of the corresponding pixel points in each of the reference face images to obtain a target face image; performing feature extraction on each of the reference face images to obtain feature information corresponding to the reference face image; calculating the average feature information corresponding to each of the reference face images to obtain supervisory information; performing the feature extraction on the target face image to obtain feature information of the target face; based on the difference between the feature information of the target face and the supervisory information, correcting the target face image to obtain an average face image. Based on the above configuration, in the process of generating the average face image, not only the pixel value information of the reference face image is utilized, but also the supervisory information determined based on the feature information of the reference face image is fully utilized. Under the action of the supervisory information, the quality of the average face image obtained is improved by correcting the target face image, and the dependence on the number of reference face images is reduced. Even if a small number of reference face images are used, a high-quality average face image can still be generated.

[0005] In some possible implementations, the feature extraction of each of the reference facial images to obtain the feature information corresponding to the reference facial image includes: inputting each of the reference facial images into a feature extraction network for feature extraction processing to obtain the feature information corresponding to the reference facial image; the feature extraction of the target facial image to obtain the feature information of the target face includes: inputting the target facial image into the feature extraction network for feature extraction processing to obtain the feature information of the target face; based on the difference between the feature information of the target face and the supervisory information, the target facial image is corrected to obtain the average facial image, including: determining the loss based on the difference between the feature information of the target face and the supervisory information; adjusting the parameters of the target facial image based on the loss feedback until the preset feedback adjustment stop condition is reached. Based on the above configuration, the target facial image can be continuously optimized by relying on the forward propagation and reverse feedback characteristics of the feature extraction network, thereby outputting a high-quality target facial image.

[0006] In some possible implementations, the adjustment of the parameters of the target facial image based on the loss feedback until a preset feedback adjustment stop condition is reached includes: performing reverse feedback on the loss along the feature extraction network to obtain the gradient of the target facial image; adjusting the parameters of the target facial image based on the gradient; if the feedback adjustment stop condition has not been reached, re-inputting the adjusted target facial image into the feature extraction network for feature extraction processing, and continuing to adjust the parameters of the target facial image based on the obtained feature information of the target face. Based on the above configuration, the target facial image can be continuously optimized by cyclically adjusting the target facial image to improve its ability to express an average face.

[0007] In some possible implementations, the method further includes: determining the weights corresponding to each of the reference facial images; the calculation of the average pixel values ​​of the corresponding pixels in each of the reference facial images to obtain the target facial image includes: calculating the weighted average pixel values ​​of the corresponding pixels in each of the reference facial images according to the weights corresponding to each of the reference facial images to obtain the target facial image; the calculation of the average feature information corresponding to each of the reference facial images to obtain the supervision information includes: calculating the weighted average feature information corresponding to each of the reference facial images according to the weights corresponding to each of the reference facial images to obtain the supervision information. Based on the above configuration, the purpose of adjusting the average facial image can be achieved by adjusting the contribution of each reference facial image in the process of generating the average facial image.

[0008] In some possible implementations, the method further includes: changing the weights corresponding to the reference facial images, and obtaining the average facial image based on the change results. Based on the above configuration, more average facial images can be obtained by changing the weights to meet the diverse needs of average facial images.

[0009] In some possible implementations, determining at least one reference face image includes: obtaining at least one face material image and a standard face image; performing face alignment processing on each face material image based on the standard face image to obtain a reference face image corresponding to each face material image. Based on the above configuration, the face material images can be aligned based on the standard face image, so that the face in the corresponding reference face image is corrected to a certain extent and is more standardized, thereby improving the quality of the average face image generated based on the reference face image.

[0010] In some possible implementations, the face alignment processing is performed on each of the face material images based on the standard face image to obtain a reference face image corresponding to each of the face material images, including: extracting key points from the standard face image to obtain first key point coordinates; performing face detection processing on each of the face material images to obtain a corresponding face detection frame; extracting key points from each of the face material images based on each of the face material images and its corresponding face detection frame to obtain corresponding second key point coordinates; aligning the face material image corresponding to the second key point coordinates to the standard face image based on the second key point coordinates and the first key point coordinates to obtain a corresponding reference face image. Based on the above configuration, the face material images can be aligned based on the key point extraction results of the standard face image and the face material image, so that the reference face image is more standardized, and the quality of the average face image is ultimately improved by improving the quality of the reference face image.

[0011] In some possible implementations, the acquisition of at least one face material image and a standard face image includes: determining a first object corresponding to the average face image; determining at least one second object having an association relationship with the first object, the association relationship including a social relationship or a kinship relationship; and determining the image corresponding to the face of the second object as the face material image. Based on the above configuration, the face material image can be determined based on the second object related to the first object. Since the second object has a social relationship or a kinship relationship with the first object, the face material image determined based on the second object can carry more information about the first object, so that the face in the final average face image is more similar to the real face of the first object, making the average face image more substitutable.

[0012] In some possible implementations, determining the weights corresponding to the reference facial images includes: determining a second object corresponding to each reference facial image; and determining the weights corresponding to each reference facial image based on the association between the second object corresponding to each reference facial image and the first object. Based on the above configuration, the weights corresponding to the reference facial images can be adjusted based on the closeness of blood relationship or social relationship, so that the face in the obtained average facial image is more similar to the real face of the first object.

[0013] According to the second aspect of the present disclosure, a face recognition method is provided, the method comprising: obtaining at least one average face image and an image to be recognized; determining the matching degree corresponding to each of the average face images, the matching degree representing the degree of similarity between the average face image and the image to be recognized; in response to the situation where the target matching degree is higher than a preset threshold, determining that the face included in the image to be recognized belongs to the object pointed to by the average face image corresponding to the target matching degree, the target matching degree being the maximum matching degree among the matching degrees corresponding to each of the average face images; wherein the average face image is generated based on the face generation method described in any one of the first aspects. Based on the above configuration, when the face data source is insufficient, the face data source can be supplemented by generating an average face image, so that face recognition can be performed on objects for which no real image exists, which expands the application space of face recognition and can also achieve a certain degree of face recognition accuracy.

[0014] According to a third aspect of the present disclosure, a face generation device is provided, the device comprising: a reference face image determination module, used to determine at least one reference face image; a target face image acquisition module, used to calculate the average pixel value of corresponding pixel points in each of the reference face images to obtain a target face image; a feature information acquisition module, used to extract features from each of the reference face images to obtain feature information corresponding to the reference face images; a supervisory information acquisition module, used to calculate the average feature information corresponding to each of the reference face images to obtain supervisory information; a target face feature acquisition module, used to extract features from the target face image to obtain feature information of the target face; and a correction module, used to correct the target face image based on the difference between the feature information of the target face and the supervisory information to obtain an average face image.

[0015] In some possible implementations, the feature information acquisition module is used to input each of the reference facial images into a feature extraction network for feature extraction processing to obtain the feature information; the target facial feature acquisition module is used to input the target facial image into the feature extraction network for feature extraction processing to obtain the feature information of the target face; the correction module is used to determine the loss based on the difference between the feature information of the target face and the supervision information; and adjust the parameters of the target facial image based on the loss feedback until a preset feedback adjustment stop condition is reached.

[0016] In some possible implementations, the correction module is used to perform reverse feedback on the loss along the feature extraction network to obtain the gradient of the target facial image; adjust the parameters of the target facial image based on the gradient; if the feedback adjustment stop condition has not been reached, re-input the adjusted target facial image into the feature extraction network for feature extraction processing, and continue to adjust the parameters of the target facial image based on the obtained feature information of the target face.

[0017] In some possible embodiments, the device also includes: a weight determination module, used to determine the weights corresponding to each of the reference facial images; the target facial image acquisition module is also used to calculate the weighted average of the pixel values ​​of the corresponding pixel points in each of the reference facial images according to the weights corresponding to each of the reference facial images, so as to obtain the target facial image; the supervision information acquisition module is also used to calculate the weighted average of the feature information corresponding to each of the reference facial images according to the weights corresponding to each of the reference facial images, so as to obtain the supervision information.

[0018] In some possible implementations, the device further includes: a weight changing module, configured to change the weights corresponding to the respective reference facial images, and obtain the average facial image based on the change result.

[0019] In some possible implementations, the reference facial image determination module includes: an image material determination unit, used to acquire at least one facial material image and a standard facial image; an alignment unit, used to perform facial alignment processing on each of the facial material images based on the standard facial image to obtain a reference facial image corresponding to each of the facial material images.

[0020] In some possible implementations, the alignment unit includes: a first key point extraction unit, used to extract key points from the standard face image to obtain first key point coordinates; a detection unit, used to perform face detection processing on each of the face material images to obtain a corresponding face detection frame; a second key point extraction unit, used to extract key points in each of the face material images based on each of the face material images and their corresponding face detection frames to obtain corresponding second key point coordinates; an alignment processing unit, used to align the face material image corresponding to the second key point coordinates to the standard face image based on the second key point coordinates and the first key point coordinates to obtain a corresponding reference face image.

[0021] In some possible implementations, the image material determination unit is used to determine a first object corresponding to the average facial image; determine at least one second object that has an association relationship with the first object, the association relationship including a social relationship or a kinship relationship; and determine an image corresponding to the face of the second object as the facial material image.

[0022] In some possible implementations, the weight determination module is used to determine the second object corresponding to each of the reference facial images; and determine the weight corresponding to each of the reference facial images based on the association relationship between the second object corresponding to each of the reference facial images and the first object.

[0023] According to a fourth aspect of the present disclosure, a face recognition device is provided, the device comprising: an image acquisition module, used to acquire at least one average face image and an image to be recognized; a matching degree determination module, used to determine the matching degree corresponding to each of the average face images, the matching degree representing the degree of similarity between the average face image and the image to be recognized; a recognition module, used to determine, in response to a target matching degree being higher than a preset threshold, whether a face included in the image to be recognized belongs to an object pointed to by the average face image corresponding to the target matching degree, the target matching degree being the maximum matching degree among the matching degrees corresponding to the average face images; wherein the average face image is generated based on the face generation method described in any one of the first aspects.

[0024] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements a face generation method as described in any one of the first aspect or a face recognition method as described in any one of the second aspect by executing the instructions stored in the memory.

[0025] According to the sixth aspect of the present disclosure, a computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the face generation method as described in any one of the first aspect or the face recognition method as described in any one of the second aspect.

[0026] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0027] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions and advantages in the embodiments of this specification or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram showing a flow chart of a method for generating a human face according to an embodiment of the present disclosure;

[0030] Figure 2 A schematic diagram showing a method flow for obtaining a reference face image according to an embodiment of the present disclosure is shown;

[0031] Figure 3 A schematic diagram showing a flow chart of a method for determining a face material image according to an embodiment of the present disclosure;

[0032] Figure 4 A schematic diagram showing a process of obtaining a rough image according to an embodiment of the present disclosure;

[0033] Figure 5 A schematic diagram showing a flow chart of a method for calculating supervisory information according to an embodiment of the present disclosure;

[0034] Figure 6 A schematic diagram of the target face image correction method according to an embodiment of the present disclosure is shown;

[0035] Figure 7 A schematic diagram showing a process of correcting a target face image based on a neural network according to an embodiment of the present disclosure;

[0036] Figure 8 A schematic diagram showing a flow chart of a face recognition method according to an embodiment of the present disclosure;

[0037] Fig. 9A block diagram of a face generation device according to an embodiment of the present disclosure is shown;

[0038] Fig.10 A block diagram of a face recognition device according to an embodiment of the present disclosure is shown;

[0039] Fig.11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0040] Fig.12 A block diagram of another electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0043] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0044] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0045] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0046] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0047] With the rapid development of image processing technology, image-based applications have penetrated all walks of life. Taking face recognition as an example, it can be widely used in various scenarios such as face filters, posture estimation, video surveillance, trajectory tracking, and identity recognition. Face recognition is based on the user's face image, but in actual use, it may be impossible to obtain the user's face image. In this case, face recognition is difficult to play a role. In view of this, the embodiment of the present disclosure provides a face generation method. When the user's face image cannot be obtained, a synthesized average face image can be used to replace the user's face image, so that related applications based on the user's face image as data can be used normally.

[0048] The disclosed embodiment provides a face generation method, which first obtains a rough target face image based on the pixel value of a reference face image, and then improves the quality of the target face image by continuously optimizing the target face. The average face image obtained according to the optimization result has high fidelity and good quality, and the generation process of the average face image has no obvious dependence on the number of reference face images. The technical solution provided by the disclosed embodiment can be applied to application scenarios such as face detection, face recognition, trajectory tracking, etc. of images or videos and their extensions, and the disclosed embodiment does not limit this. Exemplarily, the technical solution provided by the disclosed embodiment can be applied to a scenario of face recognition based on a face database. If there is no face image of user A in the face database, at least one average face image can be generated based on the face images of the relevant relatives of user A, and the average face image is stored as the face image of user A in the above face database, so that the face database can support face recognition for user A.

[0049] The face generation method and face recognition method provided by the embodiments of the present disclosure can be executed by a terminal device, a server or other types of electronic devices, wherein the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the face generation method and the face recognition method can be implemented by a processor calling a computer-readable instruction stored in a memory. The face generation method and the face recognition method of the embodiments of the present disclosure are described below by taking an electronic device as an example of an execution subject.

[0050] Figure 1 A schematic diagram of a process of generating a face according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the above method includes:

[0051] S101: Determine at least one reference face image.

[0052] In the disclosed embodiment, an existing face material image can be directly used as a reference face image, or face alignment processing can be performed on the face material image to obtain a corresponding reference face image. In the disclosed embodiment, the above-mentioned face material image can be directly obtained by an electronic device, or it can be selected from an existing face image database. Alternatively, the electronic device can obtain the above-mentioned face material image from other devices, for example, the electronic device can obtain the above-mentioned face material image from a camera device, a monitoring device, and other devices. In some implementations, the above-mentioned face material image can be an image frame in a video.

[0053] The disclosed embodiment can obtain an average face image through at least one reference face image, so that the face in the average face image can more accurately express the average face of the faces in each reference face image. The disclosed embodiment does not limit the identity information of the face in the reference face image. In some embodiments, the identity information of the face in the reference face image can be associated with the identity information of the face in the average face image. The association can be a social relationship or a kinship relationship.

[0054] For example, in order to obtain the face of user A, you can search for user B and user C who are related to user A. Based on the facial material image corresponding to the face of user B and the facial material image corresponding to the face of user C, you can obtain a reference facial image 1 containing the face of user B and a reference facial image 2 containing the face of user C. Based on reference facial image 1 and reference facial image 2, you can get an average facial image 3. The face in average facial image 3 can be considered to be a face that is highly similar to the face of user A.

[0055] The disclosed embodiments consider that common eating and living habits may also have an impact on appearance. In some scenarios, a reference face image 3 can be obtained based on a face material image corresponding to the face of a user with whom user A has a long-term social relationship. The reference face image 3 can also be used to obtain the above-mentioned average face.

[0056] In one embodiment, at least one facial material image and a standard facial image can be obtained; and, based on the standard facial image, facial alignment processing is performed on each of the facial material images to obtain a reference facial image corresponding to each of the facial material images. In the disclosed embodiment, the standard facial image can be considered as a facial image that meets preset requirements, and the preset requirements include not only facial posture requirements, but also facial content requirements. For example, the facial posture requirement can limit the face in the standard facial image to a face that is captured when the camera is facing the photographer, and the facial content requirement can limit the hair in the face in the standard facial image to not cover the ears, and the face should not wear glasses or masks. The disclosed embodiment does not limit the number of standard facial images. Of course, a single standard facial image can be used to align each facial material image.

[0057] The embodiments of the present disclosure do not limit the method for obtaining the standard facial image. In one embodiment, facial collection can be performed on multiple users based on facial posture requirements and facial content requirements to obtain multiple images, and any one of the multiple images can be used as the standard facial image, or the average facial image obtained based on the multiple images can be used as the standard facial image. In another embodiment, images in a standard image library in the related art can also be directly obtained as the standard facial image in the present disclosure. Since the facial material image may be affected by the shooting angle during shooting, the image may be deformed. Through the above configuration, the facial material images can be aligned based on the standard facial image, so that the face in the corresponding reference facial image is corrected to a certain extent and is more standardized, thereby improving the quality of the average facial image generated based on the reference facial image.

[0058] In one embodiment, please refer to Figure 2, showing a schematic flow chart of a method for obtaining a reference face image according to an embodiment of the present disclosure. The above-mentioned face alignment processing is performed on each of the above-mentioned face material images based on the above-mentioned standard face image to obtain a reference face image corresponding to each of the above-mentioned face material images, including:

[0059] S1: extract key points from the above standard face image to obtain the coordinates of the first key points.

[0060] In the embodiment of the present disclosure, key point extraction can be performed based on a key point extraction model, and the embodiment of the present disclosure does not limit the key point extraction model. Exemplarily, it can be obtained based on a cascaded pose regression model (Cascaded Pose Regression, CPR), a cascaded convolution-based face key point detection network (Deep Convolutional Network, DCNN), a multi-task cascaded convolutional neural network (Multi-task Cascaded Convolutional Networks, MTCNN) or a cascaded deep neural network (Deep Alignment Network, DAN).

[0061] The first key point coordinates corresponding to each object in the standard face image can be obtained by extracting key points. The above-mentioned object can be a part of the standard face image, such as face contour, eyebrows, eyes, nose, lips, etc. Correspondingly, the first key points obtained for these parts can include face key points, eyebrow key points, eye key points, nose key points, lip key points, etc. in the standard face image.

[0062] S2: Perform face detection processing on each of the above-mentioned face material images to obtain a corresponding face detection frame.

[0063] In the disclosed embodiment, face detection can be performed based on a face detection model, and the face detection result is expressed by the face detection frame, which represents the position of the face in the above-mentioned face material image. The disclosed embodiment does not limit the face detection model. Exemplarily, it can be obtained based on a sliding window technology (sliding window), a deformable component model (Deformable Part Model, DPM), a multi-task cascaded convolutional neural network (Multi-task Cascaded Convolutional Networks, MTCNN) or a cascaded deep neural network (Deep Alignment Network, DAN).

[0064] S3: Based on each of the above-mentioned face material images and the above-mentioned corresponding face detection frame, extract the key points in each of the above-mentioned face material images to obtain the corresponding second key point coordinates.

[0065] In the disclosed embodiment, the face detection frame can serve as position supervision information of the corresponding face material image, so as to facilitate the key point extraction model to extract key points from the image content in the face detection frame, thereby improving the efficiency of key point extraction and also improving the accuracy of key point extraction.

[0066] In the embodiment of the present disclosure, the first key point and the second key point can be obtained based on the same key point detection model. Corresponding to the relevant description of the first key point above, the second key point can also include the face key point, eyebrow key point, eye key point, nose key point, lip key point, etc. in the face material image, which will not be repeated here.

[0067] S4: Based on the second key point coordinates and the first key point coordinates, align the face material image corresponding to the second key point coordinates to the standard face image to obtain a corresponding reference face image.

[0068] In one embodiment, for each face material image, the corresponding second key point coordinates can be obtained. Based on the first key point coordinates and the second key point coordinates having a corresponding relationship, an affine transformation matrix is ​​obtained, and the face material image is transformed based on the affine transformation matrix to obtain a corresponding reference face image. If the first key point and the second key point correspond to the same object, the first key point coordinates of the first key point and the second key point coordinates of the second key point have the above-mentioned corresponding relationship.

[0069] In another embodiment, the first key point coordinates, the second key point coordinates and the face material image may be input into a preset face alignment model to obtain an aligned reference face image. The disclosed embodiment does not limit the face alignment model, and illustratively, it may be obtained based on a local binary feature algorithm (LBF), a convolutional neural network (CNN) or a 3D dense face alignment model (3DDFA).

[0070] Through the above configuration, the facial material images can be aligned based on the key point extraction results of the standard facial image and the facial material image, thereby reducing the deformation of the reference facial image and making the reference facial image more standardized. By improving the quality of the reference facial image, the quality of the average facial image is ultimately improved.

[0071] In some scenarios, it may be necessary to execute related applications based on the facial image of a first object, but the facial image of the first object does not exist in the facial database. In order to enable the related application to proceed normally, in the embodiment of the present invention, an average facial image can be generated based on a second object that has an association relationship with the first object, and the average facial image can be used as the facial image of the first object to support the normal operation of the related application. Please refer to Figure 3 , showing a flow chart of a method for determining a face material image according to an embodiment of the present disclosure, comprising:

[0072] S10. Determine the first object corresponding to the average face image.

[0073] The first object may be a user for whom there is no corresponding face image in the face database. For example, in a door access recognition scenario or a public security investigation scenario, there are users who have not entered their faces in time, and an average face may be generated for these users, and these users may be used as the first object of step S10.

[0074] S20. Determine at least one second object that has an association relationship with the first object, where the association relationship includes a social relationship or a kinship relationship.

[0075] The embodiments of the present disclosure do not limit the second object, which may be an object that has a social relationship or a kinship relationship with the first object. For example, the second object may have the following associations with the first object: the same race, the same nationality, the same place of origin, the same gender, similar age, or similar growth environment. In one embodiment, a user who has a blood relationship with the first object may be determined as the second object. The embodiments of the present disclosure believe that the facial similarity of objects with closer blood relationships is higher, so the second object can be determined based on the blood relationship. In other embodiments, a user who has a long-term close social relationship with the first object may also be determined as the second object. The embodiments of the present disclosure believe that the facial similarity of objects with long-term similarities or the same living habits may also be higher.

[0076] S30. Determine the image corresponding to the face of the second object as the face material image.

[0077] Through the above configuration, a facial material image can be determined based on a second object that has an association relationship with the first object, so that the face corresponding to the average facial image finally obtained based on the facial material image has a higher similarity with the real face of the first object, making the average facial image more substitutable.

[0078] S102. Calculate the average pixel value of the corresponding pixel points in each of the above reference face images to obtain a target face image.

[0079] The target face image in this step is obtained based on the average pixel value of the corresponding pixel points in each of the above-mentioned reference face images, and the feature information implicit in each of the reference face images has not been considered. Therefore, it is a relatively rough image. When calculating the average pixel value of the corresponding pixel points in each of the above-mentioned reference face images, the pixel points corresponding to the same object in each of the above-mentioned reference face images can be averaged, and the result obtained can be determined as the pixel value of the pixel points corresponding to the same object in the above-mentioned average face image. Exemplarily, the pixel values ​​of the pixel points representing the left face brow peak in each of the reference face images can be averaged to obtain the pixel value of the pixel point at the left face brow peak in the average face; the pixel values ​​of the pixel points representing the left face eye tail in each of the reference face images can be averaged to obtain the pixel value of the pixel point at the left face eye tail in the average face.

[0080] Please refer to Figure 4 , which shows a schematic diagram of the process of obtaining the above-mentioned rough image according to an embodiment of the present disclosure. In the embodiment of the present disclosure, N human face material images can be determined, where N is a positive integer greater than 1, and the multiple human face material images are aligned to obtain corresponding reference human face images, and the pixel points corresponding to each reference human face image are averaged to obtain the target human face image (rough image) in this step.

[0081] The embodiment of the present disclosure does not limit the weights in the averaging process. In one embodiment, the weights corresponding to each of the above-mentioned reference facial images can be determined. According to the weights corresponding to each of the above-mentioned reference facial images, the weighted average of the pixel values ​​of the corresponding pixels in each of the above-mentioned reference facial images is calculated to obtain the above-mentioned target facial image. Each reference facial image can have the same weight or different weights. The weight can be set as needed or automatically determined based on preset rules. More average facial images can also be obtained by changing the weights. The embodiment of the present disclosure does not limit this. Based on the above configuration, the contribution of each reference facial image in the process of generating the average facial image can be adjusted to achieve the purpose of adjusting the average facial image. More average facial images can also be obtained by changing the weights to meet the diverse needs of the average facial image. In some scenarios, it is necessary to use an average facial image instead of a real facial image for related applications. Multiple average facial images can be obtained by changing the weights so that the average facial image can fully play the role of substitution and ensure the normal operation of related applications.

[0082] In one embodiment, in order to obtain an average facial image that can replace the real face of the first object, the second object corresponding to each of the above reference facial images can be determined; according to the association relationship between the second object corresponding to each of the above reference facial images and the above first object, the weight corresponding to each of the above reference facial images can be determined. Exemplarily, the weight of the reference facial image corresponding to the second object of the direct blood relative is greater than the weight of the reference facial image corresponding to the second object of the collateral blood relative, and the weight of the reference facial image corresponding to the second object of the first direct blood relative is greater than the weight of the reference facial image corresponding to the second object of the second direct blood relative. The weight of the reference facial image corresponding to the second object with a kinship relationship is greater than the weight of the reference facial image corresponding to the second object with no kinship relationship but with a social relationship. Based on the above configuration, the weight of the reference facial image can be adjusted according to the association relationship, so that the face in the obtained average facial image is more similar to the real face of the first object.

[0083] S103. Perform feature extraction on each of the above reference facial images to obtain feature information corresponding to the above reference facial images.

[0084] The disclosed embodiment can use a feature extraction network to extract features from a reference face image. By inputting each of the above reference face images into the above feature extraction network, feature information corresponding to each reference face image can be obtained. The disclosed embodiment does not limit the feature extraction network. Exemplarily, the feature extraction network can be obtained based on a visual geometry group model (Visual Geometry Group, VGG), a feature pyramid network or a convolutional neural network (Convolutional Neural Networks, CNN).

[0085] S104. Calculate the average value of the feature information corresponding to each of the above reference face images to obtain supervision information.

[0086] Please refer to Figure 5 , which shows a flow chart of the method for calculating supervisory information according to an embodiment of the present disclosure. The supervisory information in this step is obtained by averaging the feature information corresponding to each of the above-mentioned reference facial images. The embodiment of the present disclosure does not limit the weights in the averaging process. As mentioned above, on the basis of determining the above-mentioned weights corresponding to each of the above-mentioned reference facial images, the weighted average of the feature information corresponding to each of the above-mentioned reference facial images can be calculated, and the weighted average can be used as the supervisory information.

[0087] S105. Perform the above feature extraction on the above target face image to obtain feature information of the target face.

[0088] In the embodiment of the present disclosure, step S105 and step S103 perform feature extraction based on the same method. The target face image in step S102 is a rough image. By performing the above feature extraction on the target face image, feature information of the target face can be obtained. Specifically, the target face image can be input into the feature extraction network used in step 103 to obtain feature information of the target face.

[0089] S106. Based on the difference between the feature information of the target face and the supervision information, the target face image is corrected to obtain an average face image.

[0090] In the disclosed embodiment, the process of correcting the target facial image based on the difference between the feature information and the supervisory information can be understood as a process of optimizing the rough image based on the difference, so that the feature information of the target face obtained based on the optimized target facial image gradually approaches the supervisory information, and accordingly, the target facial image becomes more and more expressive of the average face obtained based on the multiple reference facial images.

[0091] In one embodiment, step S106 can be implemented by looping, that is, adjusting the parameters of the target face image according to the difference between the extracted feature information of the target face and the above-mentioned supervision information, and continuing the above-mentioned feature extraction and parameter adjustment processing for the adjusted target face image until the loop stop condition is reached. The embodiments of the present disclosure do not limit the specific adjustment method and loop stop condition.

[0092] In one embodiment, please refer to Figure 6 , which shows a flow chart of a target face image correction method according to an embodiment of the present disclosure. The target face image is corrected based on the difference between the feature information of the target face and the supervisory information to obtain an average face image, including:

[0093] S1061. Determine the loss based on the difference between the feature information of the target face and the supervision information.

[0094] Please refer to Figure 7 , which shows a flow chart of correcting the above-mentioned target face image based on the feature extraction network according to an embodiment of the present disclosure. The feature extraction network in the embodiment of the present disclosure is a neural network, which includes at least one network layer and has forward propagation and reverse feedback characteristics. In the embodiment of the present disclosure, the last network layer determined according to the order of forward propagation of information in the feature extraction network is a network layer for outputting the feature information of the above-mentioned target face. Taking the feature extraction network including network layer 1, network layer 2 and network layer 3 as an example, after the target face image is input into network layer 1, the feature information of the target face can be output by network layer 3, and the loss generated by network layer 3 can be characterized by the difference between the feature information of the above-mentioned target face and the above-mentioned supervision information.

[0095] The embodiments of the present disclosure do not limit the specific method for determining the above loss based on the above difference. For example, the above loss can be equal to the above difference itself, or can be equal to the value of a preset loss function with the difference as a parameter. The embodiments of the present disclosure do not limit this.

[0096] S1062. Adjust the parameters of the target facial image based on the loss feedback until a preset feedback adjustment stop condition is reached.

[0097] The loss generated by the neural network can be fed back along the network layers in the neural network to adjust the parameters of the target facial image, and the adjusted target facial image continues to trigger the execution of steps S1061-S1062 as a new input of the feature extraction network until the above-mentioned reverse adjustment stop condition is reached. The disclosed embodiment does not limit the number of target facial images corrected based on the feature extraction network. A single target facial image or multiple target facial images can be corrected based on the feature extraction network, which will not be repeated here. Based on the above configuration, the target facial image can be continuously optimized by relying on the forward propagation and reverse feedback characteristics of the feature extraction network, thereby outputting a high-quality target facial image.

[0098] In one embodiment, adjusting the parameters of the target face image based on the loss feedback until a preset feedback adjustment stop condition is reached includes:

[0099] S10621. Perform reverse feedback on the above loss along the above feature extraction network to obtain the gradient of the above target face image.

[0100] In the embodiment of the present disclosure, the last network layer determined according to the order of reverse feedback of information in the feature extraction network is the network layer for receiving the above-mentioned target face image. Still taking the feature extraction network including network layer 1, network layer 2 and network layer 3 as an example, after the target face image is input into network layer 1, network layer 3 can output the feature information of the target face, and the gradient generated by network layer 1 can be considered to correspond to the gradient of the above-mentioned target face image.

[0101] S10622. Adjust the parameters of the target face image based on the gradient.

[0102] In the above-mentioned reverse feedback process, the parameters of the feature extraction network can be fixed, that is, the generated gradient is fed back to the target face image, triggering the parameter adjustment of the target face image. Exemplarily, the target face image can be expressed by multidimensional data, and the multidimensional data can be regarded as the parameters of the target face image. The purpose of adjusting the parameters of the target face image can be achieved by adjusting the multidimensional data. The embodiments of the present disclosure do not limit the specific method of parameter adjustment.

[0103] In a feasible embodiment, the parameters of the target face image can be adjusted based on the gradient descent method. The gradient descent method is a kind of iterative method, which can be used to solve the least squares problem (both linear and nonlinear). When solving unconstrained optimization problems, the gradient descent method is one of the most commonly used methods. The embodiments of the present disclosure do not limit the gradient descent method, and the gradient descent algorithm (GD), the stochastic gradient descent algorithm (SGD), etc. can be used.

[0104] S10623. If the feedback adjustment stop condition has not been reached, the adjusted target face image is re-input into the feature extraction network for feature extraction processing, and the parameters of the target face image are further adjusted based on the obtained feature information of the target face.

[0105] In the disclosed embodiment, the process of re-inputting the adjusted target facial image into the feature extraction network for feature extraction processing, and the process of continuing to adjust the parameters of the target facial image based on the obtained feature information of the target face can be referred to the previous text and will not be repeated here.

[0106] S10624. If the above feedback adjustment stopping condition is reached, the above target face image is determined as the above average face image.

[0107] Based on the above configuration, the target face image can be continuously optimized by cyclically adjusting the target face image, thereby improving its ability to express an average face obtained based on the reference face image, thereby improving the quality of the target face image.

[0108] The embodiments of the present disclosure do not limit the feedback adjustment stop conditions. In one embodiment, a loss threshold can be set. When the loss is less than the loss threshold, it can be indicated that the adjustment of the target facial image meets the requirements and can be applied. When the loss is greater than or equal to the loss threshold, the parameters of the target facial image can be feedback adjusted. The loss threshold can be a value set according to the requirements, such as 0.1, but it is not a specific limitation of the present disclosure. In another embodiment, a feedback adjustment number threshold can be set. When the number of adjustments is greater than the number threshold, it can be indicated that the adjustment of the target facial image meets the requirements and can be applied, otherwise the adjustment continues. The number threshold can be a value set according to the requirements and is not a specific limitation of the present disclosure.

[0109] The disclosed embodiment provides a face generation method, which can obtain a target face image based on the average pixel value of a reference face image, correct the target face image using the average feature of the reference face image as supervisory information, and obtain an average face image based on the correction result. In the process of generating the average face image, not only the pixel value information of the reference face image is used, but also the supervisory information determined based on the feature information of the reference face image is fully utilized, and the quality of the obtained average face image is improved by correcting the target face image under the effect of the supervisory information. By fully utilizing the information in the reference face image and fully utilizing the information for image correction, the dependence on the number of reference face images can be reduced, and even if a small number of reference face images are used, a high-quality average face image can still be generated.

[0110] The disclosed embodiments can be widely applied to various fields. For example, face recognition is an application with a high degree of implementation in the recognition and retrieval technology based on the visual features of images. It can be widely used in security, search and other scenarios. The disclosed embodiments further propose a face recognition method, such as Figure 8 As shown, it shows a flow chart of a face recognition method according to an embodiment of the present disclosure. The above method includes:

[0111] S201: Obtain at least one average face image and an image to be recognized.

[0112] In the embodiments of the present disclosure, each average facial image corresponds to an object, and the object corresponding to the average facial image is the owner of the face that the average facial image is intended to express. The average facial image can be generated based on the face generation method in the embodiments of the present disclosure. In some face recognition scenarios, it may be necessary to locate, check or search for an object of a known identity. If there is no real facial image of the object, one or more average facial images can be generated for it based on the above method provided in the embodiments of the present disclosure, and the average facial image can be used as a substitute for the real facial image to support face recognition of the object. Exemplarily, in a scenario where face recognition is performed based on a face retrieval and control system, the average facial image of the object can be input into the face retrieval and control system so that the face retrieval and control system can perform face recognition on the object.

[0113] For example, the face retrieval and control system needs to find user 1, but there is no face data of user 1. However, other users who are associated with user 1 can be found, and then an average face image corresponding to user 1 can be generated based on the faces of the other users. In some embodiments, at least one reference face image can be obtained based on the faces of the other users, and after determining the weights of each reference face image, an average face image corresponding to user 1 can be generated. Furthermore, the weights of each reference face image can be customized, and the weights can be changed to obtain multiple average face images corresponding to user 1, thereby increasing the probability that the face retrieval and control system recognizes user 1.

[0114] S202: Determine a matching degree corresponding to each of the average face images, where the matching degree represents a degree of similarity between the average face image and the image to be recognized.

[0115] The embodiments of the present disclosure do not limit the method for calculating the matching degree. For example, the matching degree can be calculated based on a face recognition model, or the first feature of the image to be recognized and the second feature of the average face image can be directly extracted, and the matching degree can be calculated based on the distance between the first feature and the second feature.

[0116] S203: In response to the target matching degree being higher than a preset threshold, determining that the face included in the image to be identified belongs to the object pointed to by the average face image corresponding to the target matching degree, wherein the target matching degree is the maximum matching degree among the matching degrees corresponding to the average face images.

[0117] The present disclosure embodiment does not limit the preset threshold value, and can be set according to actual conditions. If the target matching degree is higher than the preset threshold value, it is determined that the face included in the image to be identified is the face of the object (third object) pointed to by the average face image corresponding to the target matching degree.

[0118] In one embodiment, the average face image corresponding to the target matching degree can be further used as the face image corresponding to the third object and directly used in the next face recognition, without the need to use other average face images corresponding to the third object, thereby improving the face recognition speed.

[0119] In another embodiment, if multiple average face images are generated for the third object, each average face image can be weighted according to the matching degree corresponding to each average face image in previous face recognition processes, and the matching order of the average face images can be adjusted according to the weight.

[0120] For example, the average face image with better overall matching performance in all previous face recognition processes can be assigned a higher weight, and the average face image with poor overall performance can be assigned a lower weight. Images with higher weights are sorted first and used first to match the image to be recognized. If the matching degree is higher than the above-mentioned preset threshold, the matching result is directly output, thereby improving the face recognition speed.

[0121] Based on the above configuration, when the face data source is insufficient, the face data source can be supplemented by generating an average face image, so that face recognition can be performed on users for whom no real image exists, thereby expanding the application space of face recognition and achieving a certain degree of face recognition accuracy.

[0122] Those skilled in the art will appreciate that, in the above method of the specific implementation, the order in which the steps are written does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0123] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not go into details.

[0124] Fig. 9 FIG. 1 is a block diagram of a face generation device according to an embodiment of the present disclosure. Fig. 9 As shown, the above device comprises:

[0125] A reference face image determination module 101, configured to determine at least one reference face image;

[0126] The target face image acquisition module 102 is used to calculate the average pixel value of the corresponding pixel points in each of the above reference face images to obtain the target face image;

[0127] The feature information acquisition module 103 is used to extract features from each of the reference face images to obtain feature information corresponding to the reference face images;

[0128] A supervisory information acquisition module 104 is used to calculate the average value of the feature information corresponding to each of the above reference face images to obtain supervisory information;

[0129] The target face feature acquisition module 105 is used to perform the above feature extraction on the target face image to obtain feature information of the target face;

[0130] The correction module 106 is used to correct the target face image based on the difference between the feature information of the target face and the supervision information to obtain an average face image.

[0131] In some possible implementations, the feature information acquisition module is used to input each of the reference face images into the feature extraction network for feature extraction processing to obtain the feature information; the target face feature acquisition module is used to input the target face image into the feature extraction network for feature extraction processing to obtain the feature information of the target face; the correction module is used to determine the loss based on the difference between the feature information of the target face and the supervision information; and adjust the parameters of the target face image based on the loss feedback until a preset feedback adjustment stop condition is reached.

[0132] In some possible implementations, the correction module is used to perform reverse feedback on the loss along the feature extraction network to obtain the gradient of the target face image; adjust the parameters of the target face image based on the gradient; if the feedback adjustment stop condition has not been reached, re-input the adjusted target face image into the feature extraction network for feature extraction processing, and continue to adjust the parameters of the target face image based on the obtained feature information of the target face.

[0133] In some possible embodiments, the above-mentioned device also includes: a weight determination module, which is used to determine the weights corresponding to each of the above-mentioned reference facial images; the above-mentioned target facial image acquisition module is also used to calculate the weighted average of the pixel values ​​corresponding to the pixel points in the above-mentioned reference facial images according to the weights corresponding to each of the above-mentioned reference facial images, so as to obtain the above-mentioned target facial image; the above-mentioned supervision information acquisition module is also used to calculate the weighted average of the feature information corresponding to each of the above-mentioned reference facial images according to the weights corresponding to each of the above-mentioned reference facial images, so as to obtain the above-mentioned supervision information.

[0134] In some possible implementations, the apparatus further includes: a weight changing module, configured to change the weights corresponding to each of the reference facial images, and obtain the average facial image based on the change result.

[0135] In some possible implementations, the reference facial image determination module includes: an image material determination unit, used to acquire at least one facial material image and a standard facial image; an alignment unit, used to perform facial alignment processing on each of the facial material images based on the standard facial image to obtain a reference facial image corresponding to each of the facial material images.

[0136] In some possible implementations, the alignment unit includes: a first key point extraction unit, used to extract key points from the standard face image to obtain first key point coordinates; a detection unit, used to perform face detection processing on each of the face material images to obtain a corresponding face detection frame; a second key point extraction unit, used to extract key points in each of the face material images based on each of the face material images and its corresponding face detection frame to obtain corresponding second key point coordinates; an alignment processing unit, used to align the face material image corresponding to the second key point coordinates to the standard face image based on the second key point coordinates and the first key point coordinates to obtain a corresponding reference face image.

[0137] In some possible implementations, the image material determination unit is used to determine a first object corresponding to the average facial image; determine at least one second object having an association relationship with the first object, wherein the association relationship includes a social relationship or a kinship relationship; and determine an image corresponding to the face of the second object as the facial material image.

[0138] In some possible implementations, the weight determination module is used to determine the second object corresponding to each of the reference facial images; and determine the weight corresponding to each of the reference facial images according to the association relationship between the second object and the first object.

[0139] Fig.10 FIG. 1 is a block diagram of a face recognition device according to an embodiment of the present disclosure. Fig.10 As shown, the above device comprises:

[0140] An image acquisition module 201 is used to acquire at least one average face image and an image to be recognized;

[0141] A matching degree determination module 202 is used to determine a matching degree corresponding to each of the above average face images, wherein the above matching degree represents the degree of similarity between the above average face image and the above image to be recognized;

[0142] The recognition module 203 is used to determine, in response to a target matching degree being higher than a preset threshold, that the face included in the above-mentioned image to be recognized belongs to the object pointed to by the average face image corresponding to the above-mentioned target matching degree, and the above-mentioned target matching degree is the maximum matching degree among the matching degrees corresponding to the above-mentioned average face images; wherein the above-mentioned average face image is generated by the above-mentioned face generation method.

[0143] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0144] The present disclosure also provides a computer-readable storage medium, wherein at least one instruction or at least one program is stored in the computer-readable storage medium, and when the at least one instruction or at least one program is loaded and executed by a processor, the method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0145] The embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to perform the method.

[0146] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0147] Fig.11 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0148] Reference Fig.11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0149] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0150] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0151] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0152] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0153] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0154] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0155] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0156] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0157] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0158] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0159] Fig.12 1 is a block diagram of another electronic device according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Fig.12, the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0160] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0161] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0162] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0163] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0164] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0165] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C+, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0166] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0167] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0168] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0169] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the above-mentioned module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the standard function in the square box can also occur in a sequence different from the standard in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a special hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0170] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A face generation method, characterized in that: The method comprises: determining at least one reference face image; Calculate the average pixel value of the corresponding pixel points in each of the reference face images to obtain a target face image; Performing feature extraction on each of the reference facial images to obtain feature information corresponding to the reference facial image; Calculate the average value of the feature information corresponding to each of the reference face images to obtain supervision information; Performing the feature extraction on the target face image to obtain feature information of the target face; Based on the difference between the feature information of the target face and the supervision information, the target face image is corrected to obtain an average face image; the average face image corresponds to a first object, the reference face image corresponds to a second object, and the second object has an association relationship with the first object, and the association relationship includes a social relationship or a kinship relationship.

2. The method according to claim 1, characterized in that: The step of extracting features from each of the reference facial images to obtain feature information corresponding to the reference facial images includes: inputting each of the reference facial images into a feature extraction network to perform feature extraction processing to obtain feature information corresponding to the reference facial images; The step of performing the feature extraction on the target face image to obtain feature information of the target face includes: inputting the target face image into the feature extraction network to perform feature extraction processing to obtain feature information of the target face; The step of correcting the target face image based on the difference between the feature information of the target face and the supervisory information to obtain an average face image comprises: Determining a loss according to a difference between the feature information of the target face and the supervision information; The parameters of the target facial image are adjusted based on the loss feedback until a preset feedback adjustment stop condition is reached.

3. The method according to claim 2, characterized in that The adjusting the parameters of the target face image based on the loss feedback until a preset feedback adjustment stop condition is reached includes: Performing reverse feedback on the loss along the feature extraction network to obtain the gradient of the target face image; Adjusting parameters of the target face image based on the gradient; If the feedback adjustment stop condition has not been reached, the adjusted target face image is re-input into the feature extraction network for feature extraction processing, and the parameters of the target face image are further adjusted based on the obtained feature information of the target face.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Determining the weight corresponding to each of the reference face images; Calculating the average pixel value of the corresponding pixel points in each of the reference facial images to obtain the target facial image includes: calculating the weighted average pixel value of the corresponding pixel points in each of the reference facial images according to the weights corresponding to each of the reference facial images to obtain the target facial image; The step of calculating the average value of the feature information corresponding to each of the reference facial images to obtain the supervisory information includes: calculating the weighted average value of the feature information corresponding to each of the reference facial images according to the weights corresponding to each of the reference facial images to obtain the supervisory information.

5. The method according to claim 4, characterized in that The method further comprises: The weights corresponding to the reference facial images are changed, and the average facial image is obtained based on the change results.

6. The method according to claim 5, characterized in that The determining of at least one reference face image comprises: Acquire at least one face material image and a standard face image; Based on the standard face image, face alignment processing is performed on each of the face material images to obtain a reference face image corresponding to each of the face material images.

7. The method according to claim 6, characterized in that The performing face alignment processing on each of the face material images based on the standard face image to obtain a reference face image corresponding to each of the face material images includes: Extract key points from the standard face image to obtain first key point coordinates; Performing face detection processing on each of the face material images to obtain a corresponding face detection frame; Based on each of the face material images and the corresponding face detection frame, extract key points in each of the face material images to obtain corresponding second key point coordinates; Based on the second key point coordinates and the first key point coordinates, the face material image corresponding to the second key point coordinates is aligned to the standard face image to obtain a corresponding reference face image.

8. The method according to claim 6 or 7, characterized in that: The obtaining of at least one face material image and a standard face image comprises: Determining a first object corresponding to the average facial image; Determine at least one of the second objects having the association relationship with the first object; An image corresponding to the face of the second object is determined as the face material image.

9. The method according to claim 8, characterized in that The determining of the weights corresponding to the reference facial images comprises: Determine a second object corresponding to each of the reference facial images; The weight corresponding to each of the reference facial images is determined according to the association relationship between the second object corresponding to each of the reference facial images and the first object.

10. A face recognition method, characterized in that: The method comprises: Obtain at least one average face image and an image to be recognized; Determine a matching degree corresponding to each of the average face images, wherein the matching degree represents a degree of similarity between the average face image and the image to be recognized; In response to the target matching degree being higher than a preset threshold, determining that the face included in the image to be identified belongs to the object pointed to by the average face image corresponding to the target matching degree, the target matching degree being the maximum matching degree among the matching degrees corresponding to the average face images; The average face image is generated based on the face generation method described in any one of claims 1 to 9.

11. A face generation device, characterized in that: The device comprises: A reference face image determination module, used to determine at least one reference face image; A target face image acquisition module is used to calculate the average pixel value of the corresponding pixel points in each of the reference face images to obtain the target face image; A feature information acquisition module, used to extract features from each of the reference face images to obtain feature information corresponding to the reference face images; A supervisory information acquisition module, used to calculate the average value of the feature information corresponding to each of the reference face images to obtain supervisory information; A target face feature acquisition module, used to extract the features of the target face image to obtain feature information of the target face; A correction module is used to correct the target face image based on the difference between the feature information of the target face and the supervision information to obtain an average face image; the average face image corresponds to a first object, the reference face image corresponds to a second object, and the second object has an association relationship with the first object, and the association relationship includes a social relationship or a kinship relationship.

12. A face recognition device, characterized in that: The device comprises: An image acquisition module, used to acquire at least one average face image and an image to be recognized; A matching degree determination module, used to determine the matching degree corresponding to each of the average face images, wherein the matching degree represents the degree of similarity between the average face image and the image to be identified; a recognition module, configured to determine, in response to a target matching degree being higher than a preset threshold, that a face included in the image to be recognized belongs to an object pointed to by an average face image corresponding to the target matching degree, wherein the target matching degree is a maximum matching degree among the matching degrees corresponding to the average face images; The average face image is generated based on the face generation method described in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the face generation method as described in any one of claims 1 to 9, or the face recognition method as described in claim 10.

14. An electronic device, characterized in that: It includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the face generation method as described in any one of claims 1 to 9, or the face recognition method as described in claim 10 by executing the instructions stored in the memory.

Images