Model training method and device and face recognition method and device

Abstract

The invention discloses a model training method and device and a face recognition method and device. The method comprises the following steps: acquiring a plurality of face images of each person in a plurality of persons; and for one or more face images of each person, the face images are processed based on a plurality of image processing modes to obtain face images in a plurality of scenes, and different image processing modes correspond to different scenes. And training the face images of the plurality of persons in different scenes according to a preset algorithm to obtain a face recognition model.

Description

Model training method and device, face recognition method and device Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model training method and device, and a face recognition method and device. Background Art

[0002] In some scenarios, in order to identify a person, face recognition can be performed on a face image to obtain a facial feature code of the face.

[0003] When a person is in different environments, the clarity and brightness of the face image may vary. Therefore, the complexity of multiple scenes brings challenges to face recognition and increases the difficulty of face recognition.

[0004] Summary of the Invention

[0005] In one aspect, a model training method is provided, comprising: obtaining one or more facial images of each of a plurality of persons; processing each of the plurality of facial images of each person based on a plurality of image processing methods to obtain facial images in a plurality of scenarios, wherein different image processing methods correspond to different scenarios; and training the facial images of the plurality of persons in different scenarios according to a preset algorithm to obtain a facial recognition model.

[0006] In some embodiments, the method of "training facial images of multiple persons in different scenarios according to a preset algorithm to obtain a facial recognition model" may specifically include obtaining fused facial feature codes corresponding to facial images of each of the multiple persons in different scenarios. Training the fused facial feature codes of the multiple persons according to the preset algorithm to obtain the facial recognition model.

[0007] In some embodiments, the above-mentioned method of "obtaining the fused facial feature codes corresponding to the facial images of people in different scenes" can specifically include: for each of multiple people, inputting the facial images of the people in different scenes into a preset residual network model, and obtaining the fused facial feature codes corresponding to the facial images of the people in different scenes, so as to obtain the fused facial feature codes corresponding to the facial images of each of the multiple people in different scenes.

[0008] In some embodiments, the fused facial feature encoding has the same dimension as the facial feature encoding of the facial images in multiple scenes.

[0009] In some embodiments, the preset residual network model includes a first part, a second part, and a third part. The first part is used to extract common features from facial images of people in different scenarios. The second part is used to obtain facial feature codes for facial images of people in each scenario from the common features of facial images of people in different scenarios. The third part is used to fuse the facial feature codes of facial images of people in different scenarios to obtain a fused facial feature code.

[0010] In some embodiments, the second part includes a first branch and a second branch. The first branch may output facial feature codes for a face image of a person wearing a mask based on a coordinate attention mechanism. The second branch may output facial feature codes for a face image of a person not wearing a mask based on a channel attention mechanism.

[0011] In some embodiments, the method further includes: obtaining multiple target facial images with identity information. For each target facial image, the target facial image is processed based on multiple image processing methods to obtain facial images of the target facial image in different scenes. The facial images of the target facial image in different scenes are input into a face recognition model to obtain facial feature codes of the target facial image in different scenes, so as to obtain facial feature codes of each target facial image in different scenes in the multiple target facial images. Based on the facial feature codes of each facial image in different scenes in the multiple target facial images, multiple facial feature code libraries are constructed, and one facial feature code library includes facial feature codes of multiple facial images in the same scene and identity information corresponding to each facial feature code.

[0012] In some embodiments, facial images in multiple scenes include: facial images of people wearing masks in illuminated scenes, facial images of people not wearing masks in illuminated scenes, facial images of people wearing masks in infrared scenes, and facial images of people not wearing masks in infrared scenes.

[0013] In some embodiments, the multiple facial images of a person are multiple facial images with different facial postures and angles. The above-mentioned method of "obtaining multiple facial images of each of the multiple persons" may specifically include: obtaining multiple facial images to be cleaned of each person. For each person's multiple facial images to be cleaned, a second facial image is obtained based on the similarity between the reference facial image and the first facial image. The first facial image is the facial image other than the reference facial image in the multiple facial images to be cleaned; the second facial image is the facial image in the first facial image whose similarity with the reference facial image is greater than a first threshold. Based on the similarity between the second facial image and the third facial image, a fourth facial image is determined. The third facial image is the facial image other than the second facial image in the first facial image. The fourth facial image is the facial image in the third facial image whose similarity with the second facial image is greater than a second threshold. The multiple facial images of the person include the reference facial image, the second facial image, and the third facial image.

[0014] In some embodiments, the multiple image processing methods include at least one of a first processing method, a second processing method, and a third processing method. The first processing method is used to process the shape of the facial image and the face in the facial image; the second processing method is used to process the clarity of the facial image; and the third processing method is used to process the brightness of the facial image and process the facial image into a facial image of a person wearing a mask.

[0015] In some embodiments, the first processing method includes one or more of horizontal flipping, angular flipping, and radial transformation. The second processing method includes one or more of adding Gaussian noise to the facial image, image compression, and image resizing. The third processing method includes one or more of gamma change, grayscale conversion, and mask occlusion.

[0016] On the other hand, a face recognition method is provided, the method comprising: obtaining an image to be recognized. Performing face detection on the image to be recognized to obtain a face image of a person in the image to be recognized, wherein the quality score of the face image is greater than a preset value. Based on a face recognition model, obtaining a face feature code of the face image of the person; the face recognition model has the function of extracting face feature codes of face images in multiple scenes. Based on the face feature codes of the face image of the person, determining a first face feature code library from face feature code libraries corresponding to multiple scenes, and determining the identity information of the person based on the first face feature code library. The scene corresponding to the first face feature code library is the same as the scene corresponding to the face image, and the first face feature code library includes multiple face feature codes and identity information corresponding to each face feature code.

[0017] In some embodiments, the method of "performing face detection on an image to be identified to obtain a facial image of a person in the image to be identified" may specifically include: obtaining an image containing a face from the image to be identified based on a face detection frame in the image to be identified, performing facial key point detection on the image containing the face to obtain facial key points in the image containing the face. Based on the facial key points, performing face alignment on the image containing the face to obtain a facial image of the person in the image to be identified.

[0018] In some embodiments, the method further includes: resizing the image to be identified to an image of a preset size, performing face detection on the image of the preset size to obtain a first face detection frame, and mapping the first face detection frame to the image to be identified to obtain a second face detection frame of the image to be identified; wherein the position of the first face detection frame in the image of the preset size is the same as the position of the second face detection frame in the image to be identified.

[0019] In some embodiments, the method further includes: inputting a facial image in the image to be recognized into a facial quality assessment model to obtain a value for each of a plurality of facial feature codes in the facial image; and calculating a quality score for the facial image based on the values ​​corresponding to the plurality of facial feature codes and a weight for each facial feature code.

[0020] In some embodiments, the above-mentioned method of "determining a person's identity information based on a first facial feature code library" may specifically include: if there are one or more first facial feature codes in the first facial feature code library whose similarity with the facial feature code of the facial image is greater than or equal to a threshold, then the identity information of the facial feature code with the greatest similarity to the facial feature code of the facial image among the one or more first facial feature codes is used as the person's identity information. If the similarity between the facial feature codes in the first facial feature code library and the facial image is less than the threshold, then the person's identity information is determined to be the preset identity information.

[0021] In some embodiments, the above-mentioned "performing face alignment on an image including a face based on facial key points to obtain a facial image of the person" may specifically include: adjusting the position and size of the face detection frame based on the mapping relationship between the facial key points and the facial key points of the face template image, and obtaining the facial image of the person from the image to be identified based on the adjusted face detection frame.

[0022] In another aspect, a model training device is provided, which includes an acquisition unit, a processing unit, and a training unit.

[0023] The acquisition unit is configured to: acquire one or more facial images of each of the multiple persons.

[0024] The processing unit is configured to: process the multiple facial images of each person based on multiple image processing methods to obtain facial images in multiple scenarios, where different image processing methods correspond to different scenarios.

[0025] The training unit is configured to: train facial images of multiple people in different scenarios according to a preset algorithm to obtain a face recognition model.

[0026] In some embodiments, the training unit is specifically configured to obtain fused facial feature codes corresponding to facial images of each of the multiple persons in different scenarios, and train the fused facial feature codes of the multiple persons according to a preset algorithm to obtain a face recognition model.

[0027] In some embodiments, the acquisition unit is specifically configured to: for each of a plurality of persons, input the facial images of the person in different scenes into a preset residual network model, obtain the fused facial feature codes corresponding to the facial images of the person in different scenes, and obtain the fused facial feature codes corresponding to the facial images of the multiple persons in different scenes.

[0028] In some embodiments, the fused facial feature encoding has the same dimension as the facial feature encoding of the facial images in multiple scenes.

[0029] In some embodiments, the preset residual network model includes a first part, a second part, and a third part. The first part is used to extract common features from facial images of people in different scenarios. The second part is used to obtain facial feature codes for facial images of people in each scenario from the common features of facial images of people in different scenarios. The third part is used to fuse the facial feature codes of facial images of people in different scenarios to obtain a fused facial feature code.

[0030] In some embodiments, the second part includes a first branch and a second branch. The first branch may output facial feature codes for a face image of a person wearing a mask based on a coordinate attention mechanism. The second branch may output facial feature codes for a face image of a person not wearing a mask based on a channel attention mechanism.

[0031] In some embodiments, the acquisition unit is further configured to acquire multiple target facial images with identity information. The processing unit is further configured to process each target facial image based on multiple image processing methods to obtain facial images of the target facial image in different scenarios. The processing unit is further configured to input the facial images of the target facial image in different scenarios into a facial recognition model to obtain facial feature codes of the target facial image in different scenarios, thereby obtaining facial feature codes of each target facial image in different scenarios from the multiple target facial images.

[0032] The device also includes a construction unit, which is configured to: construct multiple facial feature coding libraries based on the facial feature coding of each facial image in different scenes in multiple target facial images, and a facial feature coding library includes facial feature coding of multiple facial images in the same scene and identity information corresponding to each facial feature coding.

[0033] In some embodiments, facial images in multiple scenes include: facial images of people wearing masks in illuminated scenes, facial images of people not wearing masks in illuminated scenes, facial images of people wearing masks in infrared scenes, and facial images of people not wearing masks in infrared scenes.

[0034] In some embodiments, the multiple facial images of a person are multiple facial images with different facial postures and angles, and the acquisition unit is specifically configured to: acquire multiple facial images to be cleaned for each person; obtain a second facial image based on a similarity between a reference facial image and the first facial image among the multiple facial images to be cleaned for each person, wherein the first facial image is a facial image other than the reference facial image among the multiple facial images to be cleaned; and the second facial image is a facial image among the first facial image whose similarity to the reference facial image is greater than a first threshold.

[0035] The processing unit is specifically configured to: determine a fourth facial image based on the similarity between the second facial image and the third facial image, where the third facial image is a facial image in the first facial image other than the second facial image, and the fourth facial image is a facial image in the third facial image whose similarity to the second facial image is greater than a second threshold; the multiple facial images of the person include a reference facial image, a second facial image, and a third facial image.

[0036] In some embodiments, the multiple image processing methods include one or more of a first processing method, a second processing method, and a third processing method. The first processing method is used to process the shape of the facial image and the face in the facial image; the second processing method is used to process the clarity of the facial image; and the third processing method is used to process the brightness of the facial image and process the facial image into a facial image of a face wearing a mask.

[0037] In some embodiments, the first processing method includes one or more of horizontal flipping, angular flipping, and radial transformation. The second processing method includes one or more of adding Gaussian noise to the facial image, image compression, and image resizing. The third processing method includes one or more of gamma change, grayscale conversion, and mask occlusion.

[0038] On the other hand, a face recognition device is provided, which includes: an acquisition unit, configured to: acquire an image to be recognized. A detection unit, configured to: perform face detection on the image to be recognized, and obtain a face image of the person in the image to be recognized, wherein the quality score of the face image is greater than a preset value. A processing unit, configured to: obtain a face feature code of the face image of the person based on a face recognition model; the face recognition model has the function of extracting face feature codes of face images in multiple scenes. A determination unit, configured to: determine a first face feature code library from face feature code libraries corresponding to multiple scenes based on the face feature codes of the face image of the person, and determine the identity information of the person based on the first face feature code library. The scene corresponding to the first face feature code library is the same as the scene corresponding to the face image, and the first face feature code library includes multiple face feature codes and identity information corresponding to each face feature code.

[0039] In some embodiments, the detection unit is configured to: obtain an image containing a face from the image to be identified based on a face detection frame in the image to be identified, and perform facial key point detection on the image containing the face to obtain facial key points in the image containing the face. The processing unit is configured to: perform face alignment on the image containing the face based on the facial key points to obtain a facial image of the person in the image to be identified.

[0040] In some embodiments, the apparatus further includes an adjustment unit configured to: adjust the image to be recognized to an image of a preset size, perform face detection on the image of the preset size to obtain a first face detection frame, and map the first face detection frame onto the image to be recognized to obtain a second face detection frame of the image to be recognized; wherein the position of the first face detection frame in the image of the preset size is the same as the position of the second face detection frame in the image to be recognized.

[0041] In some embodiments, the processing unit is further configured to input a facial image in the image to be recognized into a facial quality assessment model, obtain a value for each of multiple facial feature codes in the facial image, and calculate a quality score for the facial image based on the values ​​corresponding to the multiple facial feature codes and the weight of each facial feature code.

[0042] In some embodiments, the determining unit is configured to: if there are one or more first facial feature codes in the first facial feature code library whose similarity to the facial feature code of the facial image is greater than or equal to a threshold, use the identity information of the facial feature code with the greatest similarity to the facial feature code of the facial image among the one or more first facial feature codes as the identity information of the person. If the similarity between the facial feature codes in the first facial feature code library and the facial feature code of the facial image is less than the threshold, determine that the identity information of the person is the preset identity information.

[0043] In some embodiments, the adjustment unit is specifically configured to: adjust the position and size of the face detection frame according to the mapping relationship between the facial key points and the facial key points of the face template image, and obtain the facial image of the person from the image to be identified based on the adjusted face detection frame.

[0044] In another aspect, a model training device is provided, comprising a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute a computer program or instruction to implement the model training method of any of the above embodiments.

[0045] In another aspect, a face recognition device is provided, comprising a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute a computer program or instruction to implement the face recognition method described in any of the above embodiments.

[0046] In another aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores computer program instructions, which, when executed on a computer (e.g., a server), cause the computer to execute the model training method described in any of the above embodiments.

[0047] In another aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer program instructions, which, when executed on a computer (eg, a server), cause the computer to execute the face recognition method as described in any of the above embodiments.

[0048] In another aspect, a computer program product is provided, comprising computer program instructions, which, when executed on a computer (e.g., a server), cause the computer to perform the model training method described in any of the above embodiments.

[0049] In another aspect, a computer program product is provided, comprising computer program instructions, which, when executed on a computer (eg, a server), cause the computer to perform the face recognition method as described in any of the above embodiments.

[0050] In another aspect, a computer program is provided. When the computer program is executed on a computer (eg, a server), the computer program causes the computer to execute the model training method as described in any one of the above embodiments.

[0051] In another aspect, a computer program is provided. When the computer program is executed on a computer (eg, a server), the computer program enables the computer to execute the face recognition method as described in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] To more clearly illustrate the technical solutions of the present disclosure, the following briefly introduces the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure, and those skilled in the art can also derive other drawings based on these drawings. Furthermore, the drawings described below are schematic diagrams and are not intended to limit the actual dimensions of the products, actual processes of the methods, actual timing of signals, and the like involved in the embodiments of the present disclosure.

[0053] FIG1 is a structural diagram of a face recognition system according to some embodiments;

[0054] FIG2 is a schematic diagram of a model training method according to some embodiments;

[0055] FIG3 is a schematic diagram of a facial image cleaning method according to some embodiments;

[0056] FIG4 is a schematic diagram of gamma transformation of a facial image according to some embodiments;

[0057] FIG5 is a schematic diagram of a synthesized facial image according to some embodiments;

[0058] FIG6 is a structural diagram of a dual-headed residual attention network according to some embodiments;

[0059] FIG7 is a schematic diagram of branches of a dual-headed residual attention network according to some embodiments;

[0060] FIG8 is a schematic diagram of a method for constructing a facial feature encoding library according to some embodiments;

[0061] FIG9 is a schematic diagram of a face recognition method according to some embodiments;

[0062] FIG10 is a schematic diagram of a method for determining a quality score of a face according to some embodiments;

[0063] FIG11 is a schematic diagram of a method for correcting a facial image according to some embodiments;

[0064] FIG12 is a schematic diagram of a face recognition method according to some embodiments;

[0065] FIG13 is a schematic diagram of a model training apparatus according to some embodiments;

[0066] FIG14 is a schematic diagram of a face recognition device according to some embodiments;

[0067] FIG15 is a block diagram of a processing device according to some embodiments. DETAILED DESCRIPTION

[0068] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in some embodiments of the present disclosure. Obviously, the embodiments described are only some embodiments of the present disclosure, not all embodiments. Based on the embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present disclosure.

[0069] Unless the context requires otherwise, throughout the specification and claims, the term "comprise" and its other forms, such as the third person singular form "comprises" and the present participle form "comprising", are to be interpreted as open, inclusive, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" are intended to indicate that a particular feature or characteristic associated with the embodiment or example is included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features or characteristics may be included in any one or more embodiments or examples in any appropriate manner.

[0070] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "plurality" means two or more.

[0071] When describing some embodiments, the expressions "coupled" and "connected" and their derivatives may be used. For example, when describing some embodiments, the term "connected" may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term "coupled" may be used to indicate that two or more components are in direct physical or electrical contact. However, the term "coupled" or "communicatively coupled" may also refer to two or more components that are not in direct contact with each other, but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the contents of this document.

[0072] "At least one of A, B and C" has the same meaning as "at least one of A, B or C", both including the following combinations of A, B and C: A only, B only, C only, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B and C.

[0073] “A and / or B” includes the following three combinations: A only, B only, and a combination of A and B.

[0074] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined that" or "if [stated condition or event] is detected" are optionally interpreted to mean "upon determining" or "in response to determining" or "upon detecting [stated condition or event]" or "in response to detecting [stated condition or event]," depending on the context.

[0075] The use of "adapted to" or "configured to" herein is intended to be open and inclusive language that does not exclude devices adapted or configured to perform additional tasks or steps.

[0076] Additionally, the use of “based on” is meant to be open and inclusive, as a process, step, calculation, or other action “based on” one or more stated conditions or values ​​may, in practice, be based on additional conditions or values ​​beyond those stated.

[0077] With the continuous development of deep learning technology, facial recognition technology has greatly improved. In some scenarios, people wear masks. Wearing masks poses new challenges to the robustness of facial recognition systems. For example, factors such as the complexity of multiple scenarios, the accuracy of facial recognition models (also called facial feature extraction models, and henceforth collectively referred to as facial recognition models), the noise in the training data for facial recognition models, and the deployment of facial models all increase the difficulty of facial recognition. These factors are explained below.

[0078] 1. The complexity of multiple scenarios.

[0079] Typically, facial features in facial databases / facial feature libraries used by facial recognition systems are extracted from ID photos. However, in real-world scenarios, facial images captured by facial recognition systems may include: images of faces wearing masks, images of faces without masks, images of faces in daytime scenes (e.g., high light intensity), and images of faces in nighttime scenes (e.g., low light intensity). Facial images in these scenarios can introduce instability to the robustness of facial recognition systems due to variations in light intensity and facial integrity.

[0080] 2. Accuracy of face recognition model.

[0081] Generally, a face recognition system can recognize a face image using a face recognition model. However, the incompleteness of a face image of a person wearing a mask reduces the accuracy of the face recognition model.

[0082] In one possible implementation, to identify both masked and unmasked facial images, two models can be trained, one for each. These models can include a masked model for masked facial recognition and a normal model for unmasked facial recognition.

[0083] When using the two models above to recognize a face image, we first check whether the face image is wearing a mask. If so, the face image is fed into the masked model to extract the facial feature code. If not, the face image is fed into the normal model to extract the facial feature code.

[0084] Typically, when using the two models above for face recognition, the first step is to determine whether the face in the face image is wearing a mask. If the face is wearing a mask, the face image is fed into the mask model to obtain the facial feature code. If the face is not wearing a mask, the face image is fed into the normal model to obtain the facial feature code.

[0085] However, this implementation requires extracting and training two models, which is quite demanding. Furthermore, it requires designing models with different structures for different scenarios. Training a high-performance model consumes considerable computing resources and time.

[0086] In one example, if computing resources (such as a graphics processing unit (GPU)) are sufficient, it takes about 5 days to train a high-performance face recognition model. If the training data exceeds 20 million, training a model can even take more than 10 days. In this way, if two models are to be trained, the time will double. Therefore, it is impossible to quickly train a face recognition model. Secondly, if you want to deploy the face recognition system on a terminal or a device with weak computing power, using two models increases the complexity of the system and may even cause deployment problems.

[0087] 3. Noise data of training data for face recognition model.

[0088] The types of training data are numerous and contain a lot of noise. Training a high-performance model requires not only a good deep network structure, but also the amount of training data and as little noise as possible.

[0089] Generally speaking, the training data used is basically crawled from the Internet and then collected using data cleaning methods. However, such data still contains a lot of noise data, which has a negative impact on the training of face recognition models.

[0090] 4. Deployment of face recognition model.

[0091] Among them, facial recognition algorithms are complex to deploy and use in multiple scenarios. After training a model, a good deployment is crucial to maximize its performance.

[0092] In view of this, an embodiment of the present application provides a face recognition method, which can accurately perform face recognition in different scenarios, such as wearing a mask scenario, night infrared scene, daytime scene, etc., thereby improving the robustness of face recognition.

[0093] In an embodiment of the present application, after acquiring facial images corresponding to multiple identity document (ID) numbers, the facial images can be enhanced using multiple data enhancement methods to obtain facial images in multiple scenarios. In this way, the facial images in multiple scenarios can be used as training data for training, resulting in a facial recognition model capable of recognizing facial images in multiple scenarios. For more details, please refer to the embodiment shown in FIG2 below.

[0094] In one example, facial images in multiple scenes may include facial images in illuminated scenes (such as daytime) and facial images in infrared scenes (such as nighttime), wherein the facial images in each scene include facial images of people wearing masks and facial images of people not wearing masks.

[0095] To avoid the influence of noise in the training data, the multiple facial images corresponding to each ID can be cleaned based on the facial feature encoding of the facial images to delete facial images that do not belong to the ID. For details, see the embodiment shown in Figure 3 below.

[0096] Furthermore, after obtaining a face recognition model capable of recognizing face images in multiple scenarios, face recognition can be performed on the image to be recognized based on the face recognition model to obtain the identity information of the person in the image to be recognized.

[0097] The following will describe in detail the implementation of the embodiment of the present application in conjunction with the accompanying drawings.

[0098] As shown in Figure 1, Figure 1 is a face recognition system provided by an embodiment of the present application. The face recognition system may include an image acquisition device and a server. The image acquisition device is communicatively connected to the server. For example, the connection may be made by wired or wireless means. Among them, the wireless means may include Bluetooth, wireless fidelity (WIFI), a communication network (such as the fifth generation (5G), etc. The wired means may include a network cable, a system bus, etc.

[0099] The image acquisition device may be used to acquire image data including people. For example, the image acquisition device may include a video camera, a still camera, etc.

[0100] In some scenarios, the image acquisition device may be set up in an area where people pass by or are present. For example, the image acquisition device may capture video or image frames of the area in real time or periodically and transmit the captured image data to a server.

[0101] In some other scenarios, the image acquisition device may also be provided with an infrared detector. The infrared detector can be used to detect whether there are people in the area. If the infrared detector detects the presence of a person in the area, the infrared detector can trigger the image acquisition device to capture image data in the area.

[0102] The server may be configured to identify the identity information of a person in a video or image frame based on the video or image frame captured by the image acquisition device. For example, the server may perform facial recognition on the image frame captured by the camera to determine the person's information. The person's information may include identity information (such as one or more of ID, name, work number, employee, or visitor, etc.).

[0103] In some scenarios, the server can be a physical server or a cloud server. The server can be a standalone server or a cluster server, without restriction.

[0104] In some embodiments, the face recognition system provided by the embodiments of the present application may further include one or more display devices. The one or more display devices are communicatively connected to the server.

[0105] The display device may also be referred to as a display or a screen. The display device may be used to display information about people in an area.

[0106] In some examples, if the server can identify the identity information of the person in the image data (such as the person's name, ID, etc.), the server can display the facial image and identity information of the person in the image data through a display device.

[0107] In some other embodiments, if the server cannot recognize the identity information of the person in the image data, the server can display the facial image of the person through a display device and display the identity information of the person as preset identity information (such as visitor, stranger, etc.).

[0108] In some examples, the display device can also continuously update the information of people in the area. For example, when the people in the area change, the displayed information of the people can be updated.

[0109] Based on the above-mentioned face recognition system, the identity information of people in the area is recognized.

[0110] It should be noted that the execution entity of the method provided in the embodiment of the present application may be the server in Figure 1, or a device in the server, such as a chip or a system on chip.

[0111] The face recognition method provided in the embodiments of the present application may include: cleaning the face images corresponding to multiple persons, and training based on the cleaned face images to obtain a face recognition model process (referred to as the "face recognition model training process") and a process for determining the identity information of the person in the image to be identified based on the face recognition model (referred to as the "face image recognition process").

[0112] The following first describes the "training process of the face recognition model".

[0113] As shown in FIG. 2 , the “face recognition model training process” may include: S201 to S203 .

[0114] S201. Obtain one or more facial images of multiple persons.

[0115] The facial image of a person may include multiple facial images of the person at different facial postures and angles. One person corresponds to one ID. In other words, one ID corresponds to multiple facial images.

[0116] In one scenario, multiple facial images corresponding to a single ID may contain noisy data. For example, facial images of the same ID may be interspersed with facial images of other IDs. Another example is facial images of different IDs mixed with facial images corresponding to the same ID. Consequently, the performance of a facial recognition model trained on these multiple facial images may be relatively low.

[0117] In this scenario, to reduce noise in facial images of people, after acquiring multiple facial images to be cleaned, the server can clean each of the multiple facial images to be cleaned, thereby obtaining a cleaned facial image. Specifically, the cleaning method can be shown in FIG3 .

[0118] As shown in a of FIG3 , for any of the multiple persons, the server may select a facial image from the multiple facial images of the person as a reference facial image. For the facial images other than the reference person image (for ease of description, referred to as first facial images) in the multiple facial images of the person, the server may calculate the similarity between each first facial image and the reference facial image, and determine one or more second facial images based on the similarity between the first facial image and the reference facial image.

[0119] The second facial image is a facial image among the multiple first facial images of the person whose similarity with the reference facial image is greater than a first threshold (for example, in b of Figure 3, the second facial image is a facial image that has a connection relationship with the reference facial image). The reference facial image can be an avatar selected by the server from the facial images of the person in response to a selection operation. The selection operation can be an operation in which a staff member selects a reference facial image. The reference facial image can be a real facial image of the person.

[0120] In one example, the similarity between the first facial image and the reference facial image may include cosine distance and / or Euclidean distance. For example, the server may input each facial image of the person into an initial facial feature extraction model to obtain a feature vector for each facial image of the person, and determine one or more second facial images based on the similarity between the feature vector of the first facial image and the feature vector of the reference facial image.

[0121] For example, the cosine distance may be calculated as shown in the following formula 1. The Euclidean distance may be calculated as shown in the following formula 2.

[0122] Wherein, in Formula 1, A can represent the feature vector of the first face image, and B can represent the feature vector of the reference face image. i and n are positive integers, and n represents the dimension of the feature vector. 1i It can represent the parameters in the feature vector of the reference face image, x 2i Parameters in the feature vector of the first facial image may be identified.

[0123] After obtaining one or more second facial images, for the other facial images in the multiple first facial images except the second facial image (for the sake of convenience, referred to as third facial images), the server can calculate the similarity between the third facial image and each second facial image, and determine one or more fourth facial images (such as the facial images connected to the second facial image in c of Figure 3) based on the similarity between the third facial image and each second facial image. In c of Figure 3, the facial image connected to the second facial image is the facial image in the third facial image whose similarity with the second facial image is greater than the second threshold. The second threshold can be set as needed, and the second threshold can be the same as or different from the first threshold, without limitation.

[0124] After the server establishes the schematic diagram as shown in c of Figure 3, the server can delete the facial images of the person that have no association with the reference facial image (that is, the facial images in c of Figure 3 that have no connection with either the reference facial image or the second facial image) to obtain facial images of the person with less noise data.

[0125] S202: Based on multiple image processing methods, data enhancement is performed on multiple facial images of each person to obtain facial images of the person in multiple scenes.

[0126] Among them, an image processing method can process a face image into a face image in a scene. Different processing methods correspond to different scenes.

[0127] In one example, the multiple image processing methods may include one or more of a first processing method (also called a general data enhancement method), a second processing method (also called a robust face recognition enhancement method), and a third processing method (also called a special scene face recognition enhancement method).

[0128] The first processing method, the second processing method, and the third processing method are introduced below respectively.

[0129] 1. The first approach.

[0130] The first processing method can process the shape of the facial image and the face within the facial image to transform the facial image into facial images with different postures and angles. For example, the first processing method can include one or more of random horizontal flipping, random angular flipping, and random radial transformation. For more specific methods, please refer to the descriptions of 1-1 to 1-3 below.

[0131] 1-1. Random horizontal flipping may refer to mirroring the faces of a portion of facial images randomly selected from multiple facial images of a person so that the left and right sides of the face are swapped.

[0132] 1-2. Random angle flipping may refer to rotating a portion of a face image randomly selected from multiple face images of a person at a preset angle.

[0133] The preset angle can be set as needed. For example, the preset angle can be ±5 degrees. +5 degrees means rotating the face image 5 degrees clockwise, and -5 degrees means rotating the face image 5 degrees counterclockwise.

[0134] 1-3. Random radial transformation may refer to transforming the shape of a portion of a face image randomly selected from a plurality of face images of a person.

[0135] 2. The second approach.

[0136] The second processing method can process the definition of the facial image to produce facial images of different definitions. For example, the second processing method can include one or more of randomly adding Gaussian noise, randomly compressing the image, and randomly resizing the image. For more specific methods, please refer to the descriptions of 2-1 to 2-3 below.

[0137] 2-1. Randomly adding Gaussian noise refers to adding Gaussian noise to some face images randomly selected from multiple face images of a person.

[0138] Gaussian noise can be used to add multiple pixels to a facial image to simulate facial images of varying resolutions. For example, it can be used to simulate facial images in rainy, snowy, or foggy scenes.

[0139] 2-2. Random image compression may refer to changing the encoding format of a portion of a randomly selected facial image from multiple facial images of a person. The encoding format may be set as needed, for example, including Joint Photographic Experts Group (JPEG), bitmap (BMP), and tagged image file format (TIFF).

[0140] 2-3. Random image resizing may refer to resizing a portion of a face image randomly selected from a plurality of face images of a person.

[0141] 3. The third approach.

[0142] The third processing method can process the brightness of the facial image and transform the facial image into an image of a face wearing a mask. For example, the third processing method may include one or more of random gamma transformation, random grayscale conversion, and random mask occlusion. For more specific methods, please refer to the descriptions of 3-1 to 3-3 below.

[0143] 3-1. Random gamma change refers to changing the gamma value of a portion of randomly selected facial images from multiple facial images of a person.

[0144] Gamma variations can simulate facial images under varying lighting conditions. For example, as shown in Figure 4, when the gamma value of a facial image is greater than 1, the grayscale of the brighter areas is stretched, while the grayscale of the darker areas is compressed to a darker level, resulting in a darker overall facial image. When the gamma value is less than 1, the grayscale of the brighter areas is compressed, while the grayscale of the darker areas is stretched to a brighter level, resulting in a darker overall facial image.

[0145] 3-2. Random grayscaling refers to grayscaling a randomly selected portion of a person's facial images from multiple facial images. Grayscaling an image can also involve adjusting the number of channels in the image. For example, the number of channels in a facial image can be reduced from three to one.

[0146] 3-3. Random mask masking can refer to adding a mask effect to a portion of a randomly selected facial image from multiple facial images of a person. For example, mask masking methods may include pixel grayscale technology and synthesis technology.

[0147] The pixel grayscale technology can convert the pixels below the nose tip pad in the facial image to grayscale. For example, if the facial image is a 112*112 image, the server can convert the pixels at positions 56 to 112 of the facial image to grayscale.

[0148] The synthesis technology may refer to adding a mask image to the face of a face image through 3D technology. For example, the effect of the face image synthesis technology may be shown in FIG5 .

[0149] S203: training facial images of people in multiple scenarios according to a preset algorithm to obtain a facial recognition model.

[0150] The preset algorithm can be set as needed. For example, the preset algorithm can be a neural network algorithm, which is not limited. The face recognition model can have the function of recognizing a face image. For example, the face recognition model can extract the facial feature code of the input face image.

[0151] In one possible implementation, for each person's facial images in multiple scenarios, the server can input a preset residual attention network model to obtain the fused facial feature encoding corresponding to the person's facial images in different scenarios.

[0152] The preset residual attention network model can be used to determine facial feature codes for facial images of a person in multiple scenarios and fuse the facial feature codes of the facial images of the person in multiple scenarios to obtain a fused facial feature code. Furthermore, the server can train the fused facial feature code according to a preset algorithm to obtain a facial recognition model.

[0153] The fusion of facial feature codes of facial images in multiple scenes may refer to fusion of the feature codes of facial images in multiple scenes by weighted averaging to obtain a fused feature code, that is, compatible with feature codes in multiple scenes.

[0154] It should be noted that the dimensions of the facial feature encodings for multiple scenarios are the same as the dimensions of the fused facial feature encodings. For example, they are both 512-dimensional vectors or arrays. In this way, the server can be trained based on the fused facial feature encodings to obtain a facial recognition model capable of recognizing facial images in multiple scenarios.

[0155] In one example, as shown in Figure 6, a structural diagram of a dual-headed residual attention network provided in an embodiment of the present application is provided.

[0156] As shown in Figure 6(a), the dual-headed residual attention network can be composed of three parts: the first part is mainly used to extract common features of facial images of people in different scenes; the second part is mainly used to extract the feature codes of faces wearing masks and those not wearing masks. The third part is mainly used to fuse the facial feature codes.

[0157] The first part may include multiple blocks, for example, block 1, block 2, and block 3. Each block may include multiple modules. For example, as shown in FIG6 a, block 1 may include 4 operation modules, block 2 may include 14 operation modules, and block 3 may include 40 operation modules.

[0158] Among them, the operation module is defined as a BCBPCB module, B represents batch normalization operation (batch normalization), C represents convolution operation (convolution), and P represents activation operation (parametric rectified linear unit). A BCBPCB module may include multiple layers. For example, as shown in FIG6 b, the multiple layers may include multiple batch normalization (bN) layers, multiple convolution (conv) layers, and parameter correction linear unit (PRELU) layers. The connection relationship between the multiple layers can be shown in FIG6 b, which is not repeated here. Each layer is described below.

[0159] 6-1. BN layer.

[0160] The BN layer can be used to normalize batch data. This ensures that the input sample data of each BCBPCB module has the same distribution. The input sample data can also be divided into small batches for stochastic gradient descent. Moreover, when each batch of sample data is passed to the next layer, the input data of each layer is normalized, thereby accelerating model convergence and improving the speed of model training.

[0161] 6-2. CONV layer.

[0162] Among them, the CONV layer can also be called the feature extraction layer. The CONV layer can be used for feature extraction.

[0163] 6-3. PRELU layer.

[0164] Among them, the PRELU layer can also be called the activation function layer, which can be used to provide an activation function. The PRELU layer can increase the nonlinear relationship of the network and avoid zero gradient.

[0165] Furthermore, the output of the first part includes the coordinate attention (CA) mechanism and the channel attention mechanism (such as squeeze-and-excitation networks (SE)). Among them, CA encodes channel relationships and long-term dependencies through precise position information, focusing on the directional information in the common features. The directional information can refer to the deflection angle of the face image. SE mainly focuses on the channel (color) information in the common features. For example, the number of channels of the face image, the numerical value of the color of the face image, etc.

[0166] In the dual-headed residual attention network, the second part includes two branches. The first branch can output facial feature codes for images of faces wearing masks based on CA. The specific processing of the first branch is shown in Figure 7(a). The second branch can output facial feature codes for images of faces not wearing masks based on SE. The specific processing of the second branch is shown in Figure 7(b).

[0167] As shown in Figure 7(a), the first branch (residual) input is the common features of the facial image extracted by the public network, with dimensions C×H×W, where C is the output channel of the common features, and H and W are the height and width of the common features, respectively. C, H, and W are positive integers. The first branch focuses on learning the feature encoding of faces wearing masks. It uses multiple (e.g., four) BCBPCB modules, each of which is followed by a CA attention module. The CA attention module encodes channel relationships and long-term dependencies using precise positional information. Specifically, the CA attention module encodes each channel along the horizontal and vertical coordinates, that is, it aggregates features along two spatial directions. The second branch focuses on learning the features of normal facial images. It also uses four BCBPCB modules, each of which is followed by an SE attention module. The SE attention module explicitly models the interdependencies between channels and adaptively recalibrates the channel feature responses, thereby enhancing the representational power of the feature encoding of normal faces.

[0168] It should be noted that since the channels of the two branches of the dual-headed residual attention network are half the number of feature extraction channels of the common network, that is, C / 2, and can be calculated in parallel during inference, the network's computational load can be reduced. Compared with other networks with the same number of layers, the parameters in the dual-headed residual attention network are reduced by half. Because the two branches can focus on different facial information, the accuracy of face recognition can be improved.

[0169] Based on the technical solution of Figure 2, in an embodiment of the present application, a face recognition model capable of recognizing faces in multiple scenes can be obtained by processing face images into images in multiple scenes and training based on face images in multiple scenes.

[0170] In some embodiments, in order to accurately identify the identity information of a person subsequently, as shown in FIG8 , the method provided in the embodiment of the present application may further include S801 to S803 .

[0171] S801: Acquire multiple target face images with different identity information.

[0172] The target facial image refers to a facial image containing identity information. The multiple target facial images may refer to target facial images of multiple persons. For example, the target facial images may be ID photos of the persons.

[0173] In one possible implementation, the server may obtain target facial images of multiple persons in response to input from a tester. For example, the multiple persons may include staff members in a certain enterprise or factory.

[0174] S802: Based on multiple image processing methods, each target face image is processed into face images under multiple scenes, and the face images under multiple scenes are input into a face recognition model to obtain face feature codes corresponding to the respective scenes.

[0175] The multiple image processing methods may refer to the description in the above embodiments and will not be described in detail.

[0176] S803: Construct multiple facial feature coding libraries based on facial feature codes corresponding to multiple scenes of each target facial image in the multiple target facial images.

[0177] A facial feature coding library may include facial feature codes of multiple facial images in the same scene and the identity information corresponding to each facial feature code. For example, the multiple facial feature coding libraries may include facial feature coding libraries for facial images of people wearing masks in illuminated scenes, facial feature coding libraries for facial images without masks in illuminated scenes, facial feature coding libraries for facial images of people wearing masks in infrared scenes, and facial feature coding libraries for facial images without masks in infrared scenes.

[0178] Based on the embodiment of FIG8 , the server can identify facial images with identity information using a facial recognition model to obtain facial feature codes for multiple individuals. Based on the identity information and facial feature codes for the multiple individuals, the server can construct a facial feature code library for multiple scenarios. In this way, the server can accurately determine the identity information of the person to be identified based on this facial feature code library for multiple scenarios.

[0179] Next, the "face image recognition process" is described.

[0180] After constructing multiple facial feature encoding libraries using the above method, facial recognition can be performed on the image to be identified using the facial recognition model to obtain facial feature encodings for the face image in the image to be identified. Based on these multiple facial feature encoding libraries, the identity information of the person in the image to be identified can be determined. As shown in Figure 9, the "Facial Image Recognition Process" may include S901 to S904.

[0181] S901: Obtain an image to be recognized.

[0182] The image to be identified may include images of one or more persons.

[0183] In one example, the server may obtain the image to be recognized based on the above-mentioned image acquisition device. The image acquisition device may be the image acquisition device shown in FIG1 .

[0184] S902: Perform face detection on the image to be identified to obtain a face image of the person in the image to be identified.

[0185] The quality score of the facial image is greater than a preset value. The preset value can be set as needed, for example, 0.35, 0.4, etc., without limitation. The quality score of the facial image can determine whether the facial image includes a face.

[0186] In one possible implementation, the server may obtain an image containing a face from the image to be identified based on a face detection frame in the image to be identified, and perform facial key point detection on the image containing the face to obtain facial key points of the image containing the face. The server then performs face alignment on the image containing the face based on the facial key points to obtain a facial image of the person in the image to be identified.

[0187] In one example, the server can resize the image to be identified to a preset size and perform face detection on the resized image to obtain one or more first face detection frames in the resized image. The server can then map the one or more first face detection frames to the image to be identified, obtaining a second face detection frame corresponding to each first face detection frame. The position of the first face detection frame in the image to be identified is the same as the position of the corresponding second face detection frame in the image of the preset size. In this way, the server can obtain a facial image from the image to be identified based on the second face detection frame.

[0188] Furthermore, the server may also perform face alignment on the acquired face images to obtain aligned face images, and determine a quality score of each aligned face image.

[0189] In one example, as shown in FIG10 , the server can input the aligned facial image into the facial quality assessment model to obtain the value of each facial feature among multiple facial features of the facial image, and calculate the quality score of the facial image based on the value corresponding to each facial feature and the corresponding weight.

[0190] Among them, the face quality assessment model can adopt the method of multi-face information fusion to determine the multi-dimensional facial features of the face image quality. The multi-dimensional facial features may include: non-face, face, blur, occlusion, wearing sunglasses, large angle, wearing a mask, expression (such as smile), etc. The face quality assessment model can determine the probability value of each facial feature in the multi-dimensional facial features of the face image. The probability value can be between 0 and 1. The weight of each facial feature can be set as needed. For example, the weight of each facial feature can be the same, different, or partially the same, without limitation. The sum of the weights of the multiple facial features is 1.

[0191] Furthermore, if the quality score of the face image in the image to be recognized is less than a preset value, the server may delete the face image.

[0192] S903: Obtain facial feature coding of the facial image of the person based on the facial recognition model.

[0193] The face recognition model may be the model obtained through training in the above-mentioned “face recognition model training process”.

[0194] In one example, the server may input a facial image detected in an image to be recognized into a facial recognition model to obtain a facial feature encoding of the facial image.

[0195] S904: Determine the scene corresponding to the facial image based on the facial feature coding of the facial image of the person, and determine the identity information of the person based on the facial feature coding library corresponding to the scene.

[0196] Among them, the facial feature coding library corresponding to the scene can refer to the description in the above embodiment and will not be repeated here.

[0197] In one example, if there are one or more facial feature codes in the facial feature code library corresponding to the scene whose similarity with the facial feature code of the facial image of the person is greater than or equal to a third threshold, the server can use the identity information corresponding to the facial feature code among the one or more facial feature codes that has the greatest similarity with the facial feature code of the facial image of the person as the identity information of the person.

[0198] In another example, if the similarity of the facial feature codes in the facial feature code library corresponding to the scene and the facial image of the person is less than the second one, the server can determine that the identity information of the person is the preset identity information (such as visitor or stranger).

[0199] Based on the technical solution in Figure 9, after obtaining the image to be identified, the server can perform face detection on the image to be identified to obtain a facial image with a high quality score in the image to be identified. This can avoid the problem of mistakenly identifying an image that is not a face as a face image. After obtaining the facial image in the image to be identified, the facial recognition model can be used to obtain the facial feature code of the facial image. Based on the facial feature code of the facial image and a facial feature code library for multiple scenarios, the identity information of the person in the image to be identified can be determined.

[0200] In some embodiments, in the above S902, the server performs face alignment on the avatar including the face according to the facial key points, and the method for obtaining the facial image of the person in the image to be identified may include: adjusting the position and size of the face detection frame according to the mapping relationship between the facial key points and the facial key points of the face model image, and obtaining the facial image of the person from the image to be identified based on the adjusted face detection frame.

[0201] In one possible implementation, after acquiring the image to be recognized, the server may detect the image to be recognized and obtain face detection frames and corresponding position information of the people in the image to be recognized. Based on the position information of the face detection frames, the server may capture the image within the face detection frames from the image to be recognized, thereby obtaining facial images of each person in the image to be recognized.

[0202] In some scenarios, a person's head may be tilted, but the facial image within the face detection frame may be abnormal. For example, the person's head may be tilted in the image frame, but the face detection frame may be horizontal (that is, the centerline of the face detection frame is perpendicular to the horizontal line of the image frame). In this case, the facial image within the face detection frame may be incomplete.

[0203] In order to accurately obtain the facial image of a person, after obtaining the face detection frame and corresponding position information of the person in the image to be identified, the server can detect the image in the face detection frame again to determine the coordinate data of multiple key points of the face in the image frame.

[0204] Among them, multiple key points of a face can be used to represent the characteristic information of the face. For example, the multiple key points may include the left eye, right eye, nose, left corner of the mouth, right corner of the mouth, etc.

[0205] In one example, the server may input a facial image into a facial key point detection model to obtain multiple key points of the facial image and coordinate data of each key point.

[0206] In one possible scenario, the person in the facial image is wearing a mask. After detecting the left and right eye keypoints of the person's face, the server can determine the coordinates of the nose keypoint in the facial image based on the distance between the eyes and the nose. Similarly, the server can determine the coordinates of the left and right corners of the mouth keypoints in the facial image based on the distance between the eyes and the corners of the mouth.

[0207] The distance between the eyes and the nose, and the distance between the eyes and the corners of the mouth may be pre-configured by the server. The server may determine the distance between the eyes and the nose, and the distance between the eyes and the corners of the mouth on the face of a plurality of face image templates.

[0208] In a possible implementation, as shown in FIG11 , the method for the server to capture a facial image from an image to be identified based on position information of a face detection frame and multiple key points of the face may specifically include S1101 to S1102 .

[0209] S1101. Determine a mapping relationship between multiple key points of a person and multiple key points of a preset face template.

[0210] The preset face template can be used to correct the face image. For example, the angle between the centerline of the face and the horizontal line in the preset face template is 90 degrees. The centerline of the face can refer to the left and right dividing lines of the face. The size of the face detection frame of the preset face model (i.e., length and width) is the same as the size of the face detection frame. The preset face model can be pre-configured by the server.

[0211] In one possible implementation, the server maps multiple key points on a person's face to key points of corresponding types in a preset face template based on their types. For example, the left eye of a person's face is mapped to the left eye of a preset face template, and the right eye is mapped to the right eye of a preset face template. Similarly, the nose, left corner of the mouth, and right corner of the mouth are mapped to corresponding positions in the preset face template. Based on this, the server can obtain the mapping relationship between the person's face and the preset face template.

[0212] In one example, the server can calculate the distance (e.g., Euclidean distance) between the coordinate data of multiple key points of the facial image of the person in the image frame and multiple key points of the preset facial template to obtain one or more distance values. Based on the one or more distance values, the server can determine the transformation relationship between the face and the preset facial template, and use the transformation relationship as the mapping relationship between the facial image in the image frame and the preset facial template.

[0213] The conversion relationship may be a mean or median of the one or more distance values. Based on the conversion relationship, the server may correct the coordinate data of the four vertices of the face detection frame in the image frame to obtain a corrected face detection frame.

[0214] In another example, the server can calculate the offset vector between the coordinate data of each key point of multiple key points in the facial image of a person and the key point of the corresponding type of the preset facial template to obtain one or more offset vectors, and determine the mapping relationship between the facial image in the image to be identified and the preset facial template based on the one or more offset vectors.

[0215] S1102 . Correct the face detection frame in the image to be recognized according to the mapping relationship to obtain a corrected face detection frame, and obtain a face image of the person from the image to be recognized according to the corrected face detection frame.

[0216] The tilt angle of the corrected face detection frame is consistent with the tilt angle of the person's face, so that the corrected face detection frame can include the entire face area of ​​the person.

[0217] In one example, when the mapping relationship is determined based on one or more distances, the server can correct the position information (such as the coordinate data of four vertices) of the face detection frame of the person in the image frame based on the one or more distances, and obtain a new face detection frame based on the corrected vertex coordinate data.

[0218] In another example, when the mapping relationship is determined based on one or more offset vectors, the server may offset the coordinate data of the vertices of the face detection frame of the person in the image frame based on the one or more offset vectors to obtain a corrected face detection frame.

[0219] In a possible implementation, the server may directly capture the image within the face detection frame in the image frame based on the corrected face detection frame to obtain the face image of the person.

[0220] Based on the technical solution in Figure 11, for people whose heads are tilted in an image, the face detection frame can be corrected using multiple key points of the person. This allows the corrected face detection frame to accurately and comprehensively encompass the face image. This allows the server to accurately capture the face image from the image based on the corrected face detection frame, thus avoiding inaccurate face recognition caused by incomplete captured face images.

[0221] In some embodiments, as shown in FIG12 , which is a schematic diagram of a face recognition method provided by an embodiment of the present application, the method may include S1201 to S1212.

[0222] S1201: Adjust the acquired image.

[0223] The acquired image may include an image to be identified and an ID photo containing identity information. Adjusting the acquired image may involve adjusting the acquired image to a preset size. The preset size can be set as needed, for example, 640*640, without limitation.

[0224] S1202: Perform face detection on the acquired image.

[0225] Herein, face detection may refer to detecting a face detection frame in an image.

[0226] S1203: Map the face detection frame to the original image.

[0227] The original image may refer to an image before size adjustment.

[0228] In one example, the server may determine the size of the face detection frame in the original image based on the adjustment ratio between the original image and the adjusted image, and map the face detection frame to the original image based on the position of the face detection frame in the adjusted image. For example, the server may determine the position of the face detection frame in the original image based on the position of the center point of the face detection frame in the adjusted image, and obtain the face detection frame in the image based on the position of the center point of the face detection frame and its size in the image.

[0229] S1204: Capture a face image from the original image according to the face detection frame, and adjust the size of the face image.

[0230] Adjusting the size of the facial image may refer to adjusting the facial image to a preset size, for example, the facial image may be adjusted to 112*112.

[0231] S1105: Detect multiple key points of the adjusted facial image.

[0232] Among them, the multiple key points of the face image can refer to the above description and will not be repeated here.

[0233] S1206: Map multiple key points of the face image to the original image.

[0234] The method of mapping multiple key points to the original image can refer to the description of mapping the center point of the face detection frame to the original image in S1203 above, and will not be repeated here.

[0235] S1207: Perform face alignment on the face image according to multiple key points of the face image.

[0236] For details, please refer to the description of FIG11 above and no further details will be given.

[0237] It should be noted that in FIG12 , the description is made by taking the original image as an example of the image to be identified. The processing method of the facial image in the ID photo with identity information can refer to the processing method of the facial image in the image to be identified.

[0238] S1208: Determine the quality score and facial features of the facial image.

[0239] The quality score and determination method can refer to the description in S902 above. Facial features may include whether a person is wearing a mask, gender, and whether the facial image is a human face. If the quality score of a facial image is less than a preset value, the server may determine that the facial image is not a human face. If the quality score of a facial image is greater than or equal to a preset value, the server may determine that the facial image is a human face.

[0240] In one possible implementation, the server can obtain the facial information shown in Table 1 based on the face quality assessment model. The facial information may include the quality score of the facial image (such as face image 1 to face image 3), whether the person is wearing a mask, gender, whether the person is a human face, etc.

[0241] Table 1

[0242] It should be noted that the data in Table 1 is only exemplary and may also include other facial information without limitation.

[0243] In one example, if the facial image is determined to be non-face, the server may delete the face detection frame of the face in the original image. If the facial image is determined to be a face and the quality score of the facial image is greater than or equal to a preset value, the server may execute S1109 to S1112 below. If the quality score of the facial image is less than the preset value, the server may input a registration failure or recognition failure.

[0244] Among them, for the face image in the ID photo with identity information, the registration process (S1209~S1210) can be executed; for the face image in the image to be identified, the identification process (S1211~S1212) can be executed.

[0245] S1209: Process the facial image with identity information through multiple image processing methods to obtain facial images in multiple scenarios.

[0246] For details, please refer to the description of S202 above and no further details will be given.

[0247] S1210. Based on the face recognition model, determine the facial feature coding of the facial image in each scene, and construct a facial feature coding library corresponding to multiple scenes according to the facial feature coding of the facial images in multiple scenes and the identity information of the facial images.

[0248] For details, please refer to the description of S803 above and no further details will be given.

[0249] S1211. Determine the facial feature code of the facial image in the image to be recognized based on the facial recognition model.

[0250] S1212. Determine identity information of the person in the facial image based on the facial feature code of the facial image and a facial feature code library corresponding to multiple scenes.

[0251] For details, please refer to the description of S904 above and no further details will be given.

[0252] Furthermore, after determining the identity information of the person in the image to be identified, the server may display the identity information (such as name) of the person in the image to be identified via a display device. For example, the display device may display the image shown in FIG12 .

[0253] Based on the technical solution of FIG12 , the embodiment of the present application can quickly and accurately identify a person's facial image and determine their identity information by constructing a facial feature encoding library for multiple scenarios based on a facial recognition model. Because the facial recognition model can recognize facial images in multiple scenarios, the robustness of facial recognition is improved.

[0254] In the embodiment of the present application, the model training device and the face recognition device can be divided into functional modules or functional units according to the above method examples. For example, each functional module or functional unit can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0255] As shown in Figure 13, it is a structural diagram of a model training device provided in an embodiment of the present application. The device includes: an acquisition unit 131, a processing unit 132, and a training unit 133.

[0256] The acquiring unit 131 is configured to acquire multiple facial images of each of the multiple persons.

[0257] The processing unit 132 is configured to: process the facial images of each person based on multiple image processing methods to obtain facial images in multiple scenarios, where different image processing methods correspond to different scenarios.

[0258] The training unit 133 is configured to: train facial images of multiple persons in different scenarios according to a preset algorithm to obtain a face recognition model.

[0259] In some embodiments, the training unit 133 is specifically configured to: obtain the fused facial feature codes corresponding to the facial images of each of the multiple persons in different scenes, and train the fused facial feature codes corresponding to the facial images of the multiple persons in different scenes according to a preset algorithm to obtain a facial recognition model.

[0260] In some embodiments, the acquisition unit 131 is specifically configured to: for each of the multiple persons, input the facial images of the person in different scenes into a preset residual network model, obtain the common features of the facial images of the person in different scenes, and obtain the fused facial feature encoding of each of the multiple persons.

[0261] In some embodiments, the fused facial feature encoding has the same dimension as the facial feature encoding of the facial images in multiple scenes.

[0262] In some embodiments, the preset residual network model includes a first part, a second part, and a third part. The first part is used to extract common features from facial images of people in different scenarios. The second part is used to obtain facial feature codes for facial images of people in each scenario from the common features of facial images of people in different scenarios. The third part is used to fuse the facial feature codes of facial images of people in different scenarios to obtain a fused facial feature code.

[0263] In some embodiments, the second part includes a first branch and a second branch. The first branch may output facial feature codes for a face image of a person wearing a mask based on a coordinate attention mechanism. The second branch may output facial feature codes for a face image of a person not wearing a mask based on a channel attention mechanism.

[0264] In some embodiments, as shown in FIG13 , the device further includes a construction unit 134. The acquisition unit 131 is further configured to: acquire multiple target facial images with identity information. The processing unit is further configured to: process the target facial image based on multiple image processing methods for each target facial image to obtain facial images of the target facial image in different scenes. The processing unit is further configured to: input the facial images of the target facial image in different scenes into a facial recognition model to obtain facial feature codes of the target facial image in different scenes, so as to obtain facial feature codes of each target facial image in different scenes in the multiple target facial images. The construction unit 134 is configured to: construct multiple facial feature code libraries based on the facial feature codes of each facial image in different scenes in the multiple target facial images, wherein a facial feature code library includes facial feature codes of multiple facial images in the same scene and identity information corresponding to each facial feature code.

[0265] In some embodiments, facial images in multiple scenes include: facial images of people wearing masks in illuminated scenes, facial images of people not wearing masks in illuminated scenes, facial images of people wearing masks in infrared scenes, and facial images of people not wearing masks in infrared scenes.

[0266] In some embodiments, the multiple facial images of a person are multiple facial images with different facial postures and angles. The acquisition unit 131 is specifically configured to: acquire multiple facial images to be cleaned for each person. For each person's multiple facial images to be cleaned, a second facial image is obtained based on the similarity between the reference facial image and the first facial image. The first facial image is a facial image other than the reference facial image in the multiple facial images to be cleaned; the second facial image is a facial image in the first facial image whose similarity to the reference facial image is greater than a first threshold.

[0267] The processing unit 132 is specifically configured to: determine a fourth facial image based on the similarity between the second facial image and the third facial image, where the third facial image is a facial image in the first facial image other than the second facial image, and the fourth facial image is a facial image in the third facial image whose similarity to the second facial image is greater than a second threshold; the multiple facial images of the person include a reference facial image, a second facial image, and a third facial image.

[0268] In some embodiments, the multiple image processing methods include a first processing method, a second processing method, and a third processing method. The first processing method is used to process the shape of the facial image and the face in the facial image; the second processing method is used to process the clarity of the facial image; and the third processing method is used to process the brightness of the facial image and process the facial image into a facial image of a face wearing a mask.

[0269] In some embodiments, the first processing method includes one or more of horizontal flipping, angular flipping, and radial transformation. The second processing method includes one or more of adding Gaussian noise to the facial image, image compression, and image resizing. The third processing method includes one or more of gamma change, grayscale conversion, and mask occlusion.

[0270] As shown in FIG14 , it is a schematic structural diagram of a face recognition device provided in an embodiment of the present application. The device includes: an acquisition unit 141 , a detection unit 142 , a processing unit 143 , and a determination unit 144 .

[0271] The acquisition unit 141 is configured to acquire an image to be recognized.

[0272] The detection unit 142 is configured to perform face detection on the image to be identified, obtain a face image of the person in the image to be identified, and the quality score of the face image is greater than a preset value.

[0273] The processing unit 143 is configured to obtain facial feature codes of a person's facial image based on a facial recognition model; the facial recognition model has the function of extracting facial feature codes of facial images in multiple scenarios.

[0274] Determining unit 144 is configured to determine, based on the facial feature codes of the facial image of the person, a first facial feature code library from multiple facial feature code libraries corresponding to different scenarios, and determine the person's identity information based on the first facial feature code library. The first facial feature code library corresponds to the same scenario as the facial image, and the first facial feature code library includes multiple facial feature codes and identity information corresponding to each facial feature code.

[0275] In some embodiments, the detection unit 142 is further configured to: obtain an image containing a face from the image to be identified based on a face detection frame in the image to be identified, and perform facial key point detection on the image containing the face to obtain facial key points in the image containing the face. The processing unit 143 is further configured to: perform face alignment on the image containing the face based on the facial key points to obtain a facial image of the person in the image to be identified.

[0276] In some embodiments, as shown in FIG14 , the apparatus further includes an adjustment unit 145 configured to: adjust the image to be identified to an image of a preset size, perform face detection on the image of the preset size to obtain a first face detection frame, and map the first face detection frame onto the image to be identified to obtain a second face detection frame of the image to be identified; wherein the position of the first face detection frame in the image of the preset size is the same as the position of the second face detection frame in the image to be identified.

[0277] In some embodiments, processing unit 143 is further configured to input a facial image in the image to be recognized into a facial quality assessment model, obtain a value for each of the multiple facial feature codes in the facial image, and calculate a quality score for the facial image based on the values ​​corresponding to the multiple facial feature codes and the weight of each facial feature code.

[0278] In some embodiments, the determining unit 144 is configured to: if there are one or more first facial feature codes in the first facial feature code library whose similarity to the facial feature code of the facial image is greater than or equal to a threshold, use the identity information of the facial feature code with the greatest similarity to the facial feature code of the facial image among the one or more first facial feature codes as the identity information of the person. If the similarity between the facial feature codes in the first facial feature code library and the facial image is less than the threshold, determine that the identity information of the person is the preset identity information.

[0279] In some embodiments, the adjustment unit 145 is specifically configured to: adjust the position and size of the face detection frame according to the mapping relationship between the facial key points and the facial key points of the face template image, and obtain the facial image of the person from the image to be identified based on the adjusted face detection frame.

[0280] When implemented through hardware, the acquisition unit in the embodiment of the present application can be integrated into the communication interface, and the processing unit can be integrated into the processor. The specific implementation is shown in Figure 15.

[0281] FIG15 shows the processing device involved in the above-mentioned embodiments, which can be or be applied to the above-mentioned model training device and face recognition device. Processor 1502 and communication interface 1503. Processor 1502 is used to control and manage the actions of the device, for example, to execute the steps performed by the above-mentioned processing unit 132 or processing unit 143, and / or to execute other processes of the technology described herein. Communication interface 1503 is used to support communication between the device and other network entities, for example, to execute the steps performed by the above-mentioned acquisition unit 131 or acquisition unit 141. The device may also include a memory 1501 and a bus 1504, and the memory 1501 is used to store program code and data of the device.

[0282] Among them, the memory 1501 can be a memory in the device, etc., and the memory can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory can also include a combination of the above types of memory.

[0283] The processor 1502 may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, or a combination of a DSP and a microprocessor.

[0284] Bus 1504 may be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1504 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG15 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0285] The device in FIG15 may also be a chip, which includes one or more (including two) processors 1502 and a communication interface 1503 .

[0286] Optionally, the chip further includes a memory 1505, which may include a read-only memory and a random access memory, and provides operation instructions and data to the processor 1502. A portion of the memory 1505 may also include a non-volatile random access memory (NVRAM).

[0287] In some embodiments, the memory 1505 stores the following elements, execution modules or data structures, or a subset thereof, or an extended set thereof.

[0288] In the embodiment of the present application, the corresponding operation is performed by calling the operation instruction stored in the memory 1505 (the operation instruction may be stored in the operating system).

[0289] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer (e.g., a server), the computer executes the model training method and face recognition method as described in any of the above embodiments.

[0290] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical disks (e.g., CDs (Compact Disks), DVDs (Digital Versatile Disks), etc.), smart cards, and flash memory devices (e.g., EPROMs (Erasable Programmable Read-Only Memory), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0291] Some embodiments of the present disclosure further provide a computer program product, for example, stored on a non-transitory computer-readable storage medium. The computer program product includes computer program instructions that, when executed on a computer (e.g., a server), cause the computer to perform the model training method and face recognition method described in the above embodiments.

[0292] Some embodiments of the present disclosure further provide a computer program. When the computer program is executed on a computer (e.g., a server), the computer program causes the computer to execute the model training method and face recognition method described in the above embodiments.

[0293] The beneficial effects of the above-mentioned computer-readable storage medium, computer program product and computer program are the same as the beneficial effects of the model training method and face recognition method described in some of the above-mentioned embodiments, and will not be repeated here.

[0294] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0295] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0296] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0297] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that a person skilled in the art can conceive within the technical scope disclosed in the present disclosure should be included within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A model training method, wherein: The method comprises: Obtain one or more facial images of each of a plurality of persons; For each person's multiple facial images, the facial images are processed based on multiple image processing methods to obtain facial images in multiple scenes, where different image processing methods correspond to different scenes; The facial images of the multiple persons in different scenarios are trained according to a preset algorithm to obtain a facial recognition model.

2. The method according to claim 1, wherein: The method of training the facial images of the plurality of persons in different scenes according to a preset algorithm to obtain a facial recognition model includes: Obtaining fused facial feature codes corresponding to facial images of each of the multiple persons in different scenes; The fused facial feature encodings of the multiple persons are trained according to the preset algorithm to obtain the facial recognition model.

3. The method according to claim 2, wherein: The obtaining of the fused facial feature codes corresponding to the facial images of each of the multiple persons in different scenes includes: For each of the multiple persons, the facial images of the person in different scenes are input into a preset residual network model to obtain the fused facial feature codes corresponding to the facial images of the person in different scenes, so as to obtain the fused facial feature codes corresponding to the facial images of the multiple persons in different scenes.

4. The method according to claim 2 or 3, wherein: The fused facial feature encoding has the same dimension as the facial feature encoding of the facial images in the multiple scenes.

5. The method according to claim 3, wherein: The preset residual network model includes a first part, a second part and a third part; Wherein, the first part is used to extract common features of the facial images of the person in different scenes; The second part is used to obtain the facial feature code of the facial image in each scene from the common features of the facial images of the person in different scenes; The third part is used to fuse the facial feature codes of the facial images of the person in different scenes to obtain the fused facial feature codes.

6. The method according to claim 5, wherein: The second part includes a first branch and a second branch. The first branch outputs facial feature coding of a face image wearing a mask based on a coordinate attention mechanism, and the second branch outputs facial feature coding of a face image not wearing a mask based on a channel attention mechanism.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: Acquire multiple target face images with different identity information; For each target face image, processing the target face image based on the multiple image processing methods to obtain face images of the target face image in different scenes; Inputting the face images of the target face image in different scenes into the face recognition model to obtain the face feature codes of the target face image in different scenes, so as to obtain the face feature codes of each target face image in different scenes in the multiple target face images; According to the facial feature codes of each facial image in different scenes in the multiple target facial images, multiple facial feature code libraries are constructed, and one facial feature code library includes facial feature codes of multiple facial images in the same scene and identity information corresponding to each facial feature code.

8. The method according to any one of claims 1 to 7, wherein: The facial images in the multiple scenes include: facial images of people wearing masks in illuminated scenes, facial images of people not wearing masks in illuminated scenes, facial images of people wearing masks in infrared scenes, and facial images of people not wearing masks in infrared scenes.

9. The method according to any one of claims 1 to 8, wherein: The multiple facial images of the persons are multiple facial images with different facial postures and angles, and the acquiring of the multiple facial images of each of the multiple persons includes: Obtain multiple facial images to be cleaned for each person; For each person, a reference face image in a plurality of face images to be cleaned is used to obtain a second face image according to the similarity between the reference face image and a first face image, wherein the first face image is other face images in the plurality of face images to be cleaned except the reference face image; and the second face image is a face image in the first face image whose similarity with the reference face image is greater than a first threshold value; A fourth facial image is determined based on the similarity between the second facial image and the third facial image, wherein the third facial image is a facial image in the first facial image other than the second facial image, and the fourth facial image is a facial image in the third facial image whose similarity with the second facial image is greater than a second threshold; the multiple facial images of the person include the reference facial image, the second facial image, and the third facial image.

10. The method according to any one of claims 1 to 9, wherein: The multiple image processing methods include at least one of a first processing method, a second processing method, and a third processing method; The first processing method is used to process the shape of the face image and the face in the face image; The second processing method is used to process the clarity of the face image; The third processing method is used to adjust the brightness of the face image and process the face image to look like a mask. Human face image.

11. The method according to claim 10, wherein: The first processing method includes one or more of horizontal flipping, angular flipping, and radial transformation; The second processing method includes one or more of adding Gaussian noise to the face image, image compression, and image size adjustment; The third processing method includes one or more of gamma change, grayscale, and mask occlusion.

12. A face recognition method, wherein: The method comprises: Obtain an image to be recognized; Performing face detection on the image to be identified to obtain a face image of a person in the image to be identified, wherein the quality score of the face image is greater than a preset value; Based on the face recognition model, the face feature code of the face image of the person is obtained; the face recognition model has the function of extracting the face feature code of the face image in multiple scenes; According to the facial feature coding of the facial image of the person, a first facial feature coding library is determined from facial feature coding libraries corresponding to multiple scenes, and the identity information of the person is determined based on the first facial feature coding library; wherein the scene corresponding to the first facial feature coding library is the same as the scene corresponding to the facial image, and the first facial feature coding library includes multiple facial feature codes and identity information corresponding to each facial feature code.

13. The method according to claim 12, wherein: The step of performing face detection on the image to be identified to obtain a face image of a person in the image to be identified includes: According to the face detection frame in the image to be identified, an image including a face is acquired from the image to be identified, and face key point detection is performed on the image including the face to obtain face key points in the image including the face; According to the facial key points, face alignment is performed on the image including the face to obtain a facial image of the person in the image to be identified.

14. The method according to claim 12 or 13, wherein: The method further comprises: Adjusting the image to be recognized to an image of a preset size, and performing face detection on the image of the preset size to obtain a first face detection frame; The first face detection frame is mapped to the image to be identified to obtain a second face detection frame of the image to be identified; wherein the position of the first face detection frame in the image of the preset size is the same as the position of the second face detection frame in the image to be identified.

15. The method according to any one of claims 12 to 14, wherein: The method further comprises: Inputting the face image in the image to be recognized into a face quality assessment model to obtain the value of each face feature code in multiple face feature codes of the face image; The quality score of the face image is calculated based on the values ​​corresponding to the multiple face feature codes and the weight of each face feature code.

16. The method according to any one of claims 12 to 15, wherein: The determining the identity information of the person according to the first facial feature coding library includes: If there are one or more first facial feature codes in the first facial feature code library whose similarity with the facial feature code of the facial image is greater than or equal to a threshold, the identity information of the facial feature code with the greatest similarity with the facial feature code of the facial image among the one or more first facial feature codes is used as the identity information of the person; If the similarities between the facial feature codes in the first facial feature code library and the facial image are all less than the threshold, it is determined that the identity information of the person is the preset identity information.

17. The method according to claim 13, wherein: The step of performing face alignment on the image including the face according to the face key points to obtain the face image of the person includes: According to the mapping relationship between the facial key points and the facial key points of the facial template image, the position and size of the face detection frame are adjusted, and the facial image of the person is obtained from the image to be identified based on the adjusted face detection frame.

18. A model training device, wherein: The device comprises: An acquisition unit is configured to: acquire one or more facial images of each of the multiple persons; The processing unit is configured to: process the multiple facial images of each person based on multiple image processing methods to obtain facial images in multiple scenes, where different image processing methods correspond to different scenes; The training unit is configured to: train the facial images of the multiple persons in different scenes according to a preset algorithm to obtain a facial recognition model.

19. A face recognition device, wherein: The device comprises: The acquisition unit is configured to: acquire an image to be recognized; A detection unit is configured to: perform face detection on the image to be identified to obtain a face image of a person in the image to be identified, wherein a quality score of the face image is greater than a preset value; The processing unit is configured to: obtain the facial feature code of the facial image of the person based on the facial recognition model; the facial recognition model has the function of extracting the facial feature code of the facial image in multiple scenes; The determining unit is configured to: determine a first facial feature coding library from facial feature coding libraries corresponding to multiple scenes according to the facial feature coding of the facial image of the person, and determine the identity information of the person according to the first facial feature coding library; wherein the scene corresponding to the first facial feature coding library is the same as the scene corresponding to the facial image, and the first facial feature coding library includes Multiple facial feature codes and identity information corresponding to each facial feature code.

20. A processing device, wherein: include: Processor and memory; The memory stores computer instructions, and the processor is used to run the computer program to implement the model training method as described in any one of claims 1-11, or to implement the face recognition method as described in any one of claims 12-17.

21. A non-transitory computer-readable storage medium, wherein: The computer-readable storage medium stores instructions. When the computer executes the instructions, the computer executes the model training method described in any one of claims 1 to 11, or executes the face recognition method described in any one of claims 13 to 17.

Images