A face recognition method, device and storage medium

Abstract

This application discloses a method, apparatus, and storage medium for face recognition. A first feature is obtained by inputting a face image into a face recognition model within an integrated model. This integrated model includes a face recognition model and a deep semantic extraction branch, wherein the face recognition model comprises a shallow network and a deep network. The shallow features of the face image are then input into the deep semantic extraction branch to obtain a second feature. These features are then fused to obtain a fused feature, and feature comparison is performed for recognition. This achieves a face recognition process that fuses deep features. Because an independent branch is introduced into the deep network of the integrated model, the diversity of features is improved; and the features of the shallow network are shared, thus improving both the accuracy and efficiency of face recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a face recognition method and device and a storage medium. BACKGROUND

[0002] With the rapid development of Internet technology, people have increasingly high requirements for information security, and face recognition is an effective way to improve information security. Face recognition is applied to mobile face recognition systems in actual scenarios such as security, payment, and access control, and high requirements are placed on the running time and accuracy of face recognition models.

[0003] Generally, a large amount of face training data can be collected to train a face recognition model on a large scale, thereby improving the accuracy of face recognition.

[0004] However, collecting a large amount of face training data requires a large amount of resources and requires a large amount of face training data to be labeled. In addition, due to the difficulty of effectively labeling a large number of samples, incorrect labels are introduced, making the training process cumbersome and affecting the accuracy and efficiency of face recognition. SUMMARY

[0005] Therefore, the present application provides a face recognition method that can effectively improve the accuracy and efficiency of face recognition.

[0006] The first aspect of the present application provides a face recognition method that can be applied to a system or program containing a face recognition function in a terminal device, specifically comprising:

[0007] Obtaining a face image of a target object;

[0008] Inputting the face image into a face recognition model in an integrated model to obtain first features, the integrated model comprising the face recognition model and a deep semantic extraction branch, the face recognition model comprising a shallow network and a deep network, and the shallow network being configured to extract shallow features of the face image;

[0009] Inputting the shallow features of the face image into the deep semantic extraction branch to obtain second features, the deep semantic extraction branch using features extracted by the shallow network during training, and the network parameters of the deep semantic extraction branch being different from the network parameters of the deep network;

[0010] Inputting the first features and the second features into a fusion dimension reduction model for fusion to obtain fusion features;

[0011] Performing feature comparison based on the fusion features to obtain face-related recognition information corresponding to the target object.

[0012] Optionally, in some possible implementation manners of the present application, the method further includes:

[0013] obtaining training data for face recognition;

[0014] inputting the training data into the integrated model;

[0015] extracting shallow features of the training data based on the shallow network, and extracting training features of the shallow features of the training data according to the deep network, to train the face recognition model;

[0016] training the deep semantic extraction branch based on the training data, the deep semantic extraction branch using the shallow network for feature input during the training process, and the parameters of the shallow network being fixed;

[0017] inputting the training features and deep features output by the deep semantic extraction branch into the fusion dimension reduction model, to train the fusion dimension reduction model based on a constraint condition, and the parameters of the face recognition model and the deep semantic extraction branch being fixed during the training of the fusion dimension reduction model.

[0018] Optionally, in some possible implementation manners of the present application, the extracting shallow features of the training data based on the shallow network, and extracting training features of the shallow features of the training data according to the deep network, to train the face recognition model, includes:

[0019] extracting shallow features of the training data based on the shallow network, and extracting the training features of the shallow features of the training data according to the deep network;

[0020] determining label information corresponding to the training data;

[0021] matching the training features with the label information, to perform gradient calculation based on a first loss function;

[0022] comparing a parameter value in a gradient calculation process corresponding to the first loss function with a first training condition, and if the first training condition is met, completing the training of the face recognition model.

[0023] Optionally, in some possible implementation manners of the present application, the training the deep semantic extraction branch based on the training data includes:

[0024] determining first sampling information of the training data sampled in the training process of the face recognition model;

[0025] Different from the first sampling information, a sampling configuration is performed to determine second sampling information corresponding to the deep semantic extraction branch;

[0026] The training data is sampled based on the second sampling information to obtain training samples;

[0027] The training samples are input into the trained shallow network to obtain sample input features;

[0028] The sample input features are input into the deep semantic extraction branch to obtain sample output features;

[0029] Label information corresponding to the sample output features is determined to perform gradient calculation of a second loss function on the deep semantic extraction branch;

[0030] The parameter value in the gradient calculation process corresponding to the second loss function is compared with a second training condition, and if the second training condition is met, the training of the deep semantic extraction branch is completed.

[0031] Optionally, in some possible implementations of the present application, the sample input features are input into the deep semantic extraction branch to obtain sample output features, including:

[0032] Random parameter information of a deep network in the face recognition model is obtained;

[0033] Random seeds are configured based on the random parameter information of the deep network to perform parameter configuration on the deep semantic extraction branch;

[0034] The sample input features are input into the deep semantic extraction branch after the parameter configuration to obtain the sample output features.

[0035] Optionally, in some possible implementations of the present application, the label information corresponding to the sample output features is determined to perform gradient calculation of a second loss function on the deep semantic extraction branch, including:

[0036] The label information corresponding to the sample output features is determined to determine a quality parameter of the training samples;

[0037] The quality parameter and a constraint parameter of the second loss function are configured based on the face recognition model to perform parameter adjustment on the second loss function;

[0038] The gradient calculation is performed on the deep semantic extraction branch based on the second loss function after the parameter adjustment.

[0039] Optionally, in some possible implementation manners of the present application, the inputting the shallow feature of the face image into the deep semantic extraction branch to obtain a second feature comprises:

[0040] performing network search based on preset accuracy information to determine parameter information of the deep semantic extraction branch;

[0041] performing parameter configuration on the deep semantic extraction branch according to the parameter information obtained through network search;

[0042] inputting the shallow feature of the face image into the deep semantic extraction branch after parameter configuration to obtain the second feature.

[0043] The second aspect of the present application provides a device for face recognition, comprising:

[0044] an acquisition unit configured to acquire a face image of a target object;

[0045] an input unit configured to input the face image into a face recognition model in an integrated model to obtain a first feature, the integrated model comprising the face recognition model and a deep semantic extraction branch, the face recognition model comprising a shallow network and a deep network, and the shallow network being configured to extract a shallow feature of the face image;

[0046] the input unit is further configured to input the shallow feature into the deep semantic extraction branch to obtain a second feature, the deep semantic extraction branch using a feature extracted by the shallow network in a training process, and network parameters of the deep semantic extraction branch being different from network parameters of the deep network;

[0047] a fusion unit configured to input the first feature and the second feature into a fusion and dimension reduction model to perform fusion to obtain a fused feature;

[0048] a recognition unit configured to perform feature comparison based on the fused feature to obtain face-related recognition information corresponding to the target object.

[0049] Optionally, in some possible implementation manners of the present application, the recognition unit is specifically configured to acquire training data for face recognition.

[0050] the recognition unit is specifically configured to input the training data into the integrated model;

[0051] the recognition unit is specifically configured to extract a shallow feature of the training data based on the shallow network, and extract a training feature of the shallow feature of the training data according to the deep network to train the face recognition model;

[0052] The identification unit is specifically configured to train the deep semantic extraction branch based on the training data, and the deep semantic extraction branch adopts the shallow network for feature input in a training process, and parameters of the shallow network are fixed;

[0053] The identification unit is specifically configured to input the training feature and a deep feature output by the deep semantic extraction branch into the fusion dimension reduction model, train the fusion dimension reduction model based on a constraint condition, and fix parameters of the face recognition model and the deep semantic extraction branch in a training process.

[0054] Optionally, in some possible implementations of the present application, the identification unit is specifically configured to extract shallow features of the training data based on the shallow network, and extract the training feature from the shallow features of the training data according to the deep network.

[0055] The identification unit is specifically configured to determine label information corresponding to the training data.

[0056] The identification unit is specifically configured to match the training feature with the label information, and perform gradient calculation based on a first loss function.

[0057] The identification unit is specifically configured to compare a parameter value in a gradient calculation process of the first loss function with a first training condition, and complete training of the face recognition model if the first training condition is met.

[0058] Optionally, in some possible implementations of the present application, the identification unit is specifically configured to determine first sampling information of the training data in a training process of the face recognition model.

[0059] The identification unit is specifically configured to distinguish from the first sampling information to perform sampling configuration, and determine second sampling information corresponding to the deep semantic extraction branch.

[0060] The identification unit is specifically configured to sample the training data based on the second sampling information to obtain a training sample.

[0061] The identification unit is specifically configured to input the training sample into the trained shallow network to obtain sample input features.

[0062] The identification unit is specifically configured to input the sample input features into the deep semantic extraction branch to obtain sample output features.

[0063] The identification unit is specifically configured to determine label information corresponding to the sample output features, and perform gradient calculation of a second loss function on the deep semantic extraction branch.

[0064] The identification unit is specifically configured to compare the parameter value in the gradient calculation process corresponding to the second loss function with a second training condition, and if the second training condition is met, the training of the deep semantic extraction branch is completed.

[0065] Optionally, in some possible implementations of the present application, the identification unit is specifically configured to obtain parameter random information of a deep network in the face recognition model.

[0066] The identification unit is specifically configured to distinguish the random seed configuration from the parameter random information of the deep network, so as to configure the parameters of the deep semantic extraction branch.

[0067] The identification unit is specifically configured to input the sample input feature into the deep semantic extraction branch after the parameter configuration, so as to obtain the sample output feature.

[0068] Optionally, in some possible implementations of the present application, the identification unit is specifically configured to determine the label information corresponding to the sample output feature, so as to determine the quality parameter of the training sample.

[0069] The identification unit is specifically configured to distinguish the configuration of the quality parameter and the constraint parameter of the second loss function from the face recognition model, so as to adjust the parameters of the second loss function.

[0070] The identification unit is specifically configured to perform gradient calculation on the deep semantic extraction branch based on the second loss function after the parameter adjustment.

[0071] Optionally, in some possible implementations of the present application, the input unit is specifically configured to perform network search based on preset precision information, so as to determine the parameter information of the deep semantic extraction branch.

[0072] The input unit is specifically configured to configure the parameters of the deep semantic extraction branch according to the parameter information obtained by the network search.

[0073] The input unit is specifically configured to input the shallow feature of the face image into the deep semantic extraction branch after the parameter configuration, so as to obtain the second feature.

[0074] The third aspect of the present application provides a computer device, comprising a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the face recognition method of the first aspect or any one of the first aspect according to the instructions in the program code.

[0075] The fourth aspect of the present application provides a computer readable storage medium, wherein instructions are stored in the computer readable storage medium, and when the instructions are executed on a computer, the computer executes the face recognition method of the first aspect or any one of the implementation manners of the first aspect.

[0076] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the face recognition method of the first aspect or any one of the implementation manners of the first aspect.

[0077] From the above technical solutions, the embodiments of the present application have the following advantages:

[0078] By obtaining a face image of a target object, inputting the face image into a face recognition model of an integrated model to obtain a first feature, the integrated model including the face recognition model and a deep semantic extraction branch, wherein the face recognition model includes a shallow network and a deep network, the shallow network is used to extract a shallow feature of the face image, and the deep network is used to extract the first feature from the shallow feature, and inputting the shallow feature of the face image into the deep semantic extraction branch to obtain a second feature, wherein the deep semantic extraction branch uses the feature extracted by the shallow network in a training process, and the network parameters of the deep semantic extraction branch are different from the network parameters of the deep network, then inputting the first feature and the second feature into a fusion dimension reduction model for fusion to obtain a fusion feature, and then performing feature comparison based on the fusion feature to obtain face-related recognition information corresponding to the target object. Thus, the face recognition process of the fusion deep feature is realized. Since an independent branch is introduced in the deep network of the integrated model, the diversity of the feature is improved. Moreover, the feature of the shallow network is a shared mechanism, which improves the accuracy of face recognition and improves the efficiency of face recognition. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0080] Figure 1 A network architecture diagram for a face recognition system;

[0081] Figure 2 A flow architecture diagram of a face recognition provided by the embodiments of the present application;

[0082] Figure 3 A flowchart illustrating a face recognition method provided in this application embodiment;

[0083] Figure 4 A schematic diagram illustrating a scenario for a face recognition method provided in an embodiment of this application;

[0084] Figure 5 A flowchart illustrating another face recognition method provided in this application embodiment;

[0085] Figure 6 A flowchart illustrating another face recognition method provided in this application embodiment;

[0086] Figure 7 A schematic diagram illustrating a scenario for another face recognition method provided in an embodiment of this application;

[0087] Figure 8 A flowchart illustrating another face recognition method provided in this application embodiment;

[0088] Figure 9 A schematic diagram of the structure of a face recognition device provided in an embodiment of this application;

[0089] Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0090] Figure 11 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0091] This application provides a method and related apparatus for face recognition, which can be applied to systems or programs in terminal devices that include face recognition functionality. The method involves acquiring a face image of a target object; then inputting the face image into a face recognition model within an integrated model to obtain a first feature. This integrated model includes a face recognition model and a deep semantic extraction branch. The face recognition model comprises a shallow network and a deep network. The shallow network extracts shallow features from the face image, and the deep network extracts the first feature from the shallow features. The shallow features of the face image are then input into the deep semantic extraction branch to obtain a second feature. During training, the deep semantic extraction branch uses features extracted by the shallow network, and the network parameters of the deep semantic extraction branch differ from those of the deep network. The first and second features are then input into a fusion and dimensionality reduction model for fusion to obtain a fused feature. Finally, feature comparison is performed based on the fused feature to obtain face-related recognition information corresponding to the target object. This enables the face recognition process that integrates deep features. By introducing independent branches into the deep network of the ensemble model, the diversity of features is enhanced. Furthermore, the features of the shallow network are shared, which improves both the accuracy and efficiency of face recognition.

[0092] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0093] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0094] It should be understood that the face recognition method provided in this application can be applied to systems or programs in terminal devices that include face recognition functionality, such as face recognition applications. Specifically, the face recognition system can run on, for example,... Figure 1 In the network architecture shown, such as Figure 1The diagram shown illustrates the network architecture of a facial recognition system. As can be seen, the system can perform facial recognition processes with multiple information sources. This involves sending a facial image to the server via a trigger action on the terminal side for feature extraction and further identity verification. It can be understood that... Figure 1 The diagram shows various terminal devices, which can be computer devices. In real-world scenarios, more or fewer types of terminal devices may participate in the facial recognition process. The specific number and types depend on the actual scenario and are not limited here. Additionally, Figure 1 The example shows one server, but in real-world scenarios, multiple servers can be involved, especially in scenarios involving multi-model training and interaction. The specific number of servers depends on the actual scenario.

[0095] In this embodiment, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, smart voice interaction device, smart home appliance, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and the terminal and server can be connected to form a blockchain network; this application does not impose any restrictions.

[0096] It is understood that the aforementioned facial recognition system can run on personal mobile terminals, such as as a facial recognition application, or it can run on a server, or it can run on a third-party device to provide facial recognition and obtain the facial recognition processing results of the information source. Specifically, the facial recognition system can run as a program in the aforementioned device, or it can run as a system component in the aforementioned device, or it can run as a cloud service program. This embodiment can be applied to cloud technology, autonomous driving, and other scenarios. The specific operating mode depends on the actual scenario and is not limited here.

[0097] With the rapid development of internet technology, people have increasingly higher requirements for information security. Facial recognition is an effective way to improve information security. In mobile facial recognition systems applied to real-world scenarios such as security, payment, and access control, high demands are placed on the running time and accuracy of facial recognition models. Generally, the accuracy of facial recognition can be improved by collecting a large amount of facial training data and training the facial recognition model on a large scale.

[0098] However, collecting a large amount of face training data requires a lot of resources and labeling of the large amount of face training data. In addition, due to the difficulty in effectively labeling a large number of samples, incorrect labels are introduced, making the training process cumbersome and affecting the accuracy and efficiency of face recognition.

[0099] To address the aforementioned problems, this application proposes a face recognition method, which is applied to... Figure 2 The facial recognition process framework shown is as follows: Figure 2 The diagram shown is a flowchart of a face recognition process provided in an embodiment of this application. The user triggers the terminal to collect a face image through the terminal access operation and sends the face image to the server. Multi-dimensional features are extracted through the face recognition model and deep semantic branch, and feature fusion is performed to obtain the face recognition result.

[0100] This embodiment introduces an independent model branch structure into the deep network structure based on the face recognition model, and then trains this branch using new random parameters. Simultaneously, a feature dimensionality reduction module is added to the model output to ensure that the feature dimensions of the output model match those of the existing model, guaranteeing compatibility with existing systems. By integrating deep semantic features, the accuracy of the face recognition model is increased. In other words, only a computationally insignificant branch is added to the existing network, ensuring that the increase in time consumption is controllable. Furthermore, the two branches are trained independently during training, without increasing GPU memory or training time.

[0101] It is understood that the method provided in this application can be a program written as processing logic in a hardware system, or it can be a face recognition device, implemented in an integrated or external manner to achieve the aforementioned processing logic. As one implementation, the face recognition device triggers a target section in a first interactive video in response to a target operation; then, it determines a second interactive video based on the association relationship corresponding to the target section. The association relationship is determined based on attribute tags or mapping relationships corresponding to the first interactive video. Attribute tags indicate the content features corresponding to the first interactive video, and mapping relationships indicate the correspondence between interactive videos. The device further displays the media content indicated in the second interactive video. This achieves the association and expansion of interactive video content. Because multiple interactive videos are associated, existing interactive video content can be fully utilized, greatly improving the progress of interactive video content expansion and thus increasing the efficiency of interactive video content expansion.

[0102] The solutions provided in this application relate to computer vision technology in artificial intelligence, and are specifically illustrated through the following embodiments:

[0103] Based on the above process architecture, the face recognition method in this application will be described below. Please refer to [link / reference]. Figure 3 , Figure 3 The flowchart illustrates a face recognition method provided in this application embodiment. This management method can be executed by a terminal or a server, and this application embodiment includes at least the following steps:

[0104] 301. Obtain the face image of the target object.

[0105] In this embodiment, the target object is the object to be identified, and its facial image can be captured by an instant-activated camera device, thereby being applied to various 1:1 or 1:N facial recognition services in facial recognition applications.

[0106] Specifically, this embodiment introduces an independent feature extraction branch in the deep structure of the face recognition model, and trains the normal face recognition model and the deep face recognition branch independently. The deep feature extraction branch is used to assist in the integration of the recognition model, thereby improving the recognition accuracy of the face recognition model.

[0107] In one possible scenario, this embodiment adopts Figure 4 The architecture shown Figure 4 This diagram illustrates a scenario for a face recognition method provided in this embodiment. The diagram shows that this embodiment is divided into a network module training phase and a network module integration and deployment phase. First, a normal face is used to train the model. Then, an independent deep feature extraction branch is introduced into the face recognition network. This branch is initialized using different initialization parameters, and a shallow feature sharing mode is used to train the branch, reducing model processing time. Finally, a feature fusion and dimensionality reduction module is trained to fuse and integrate the face recognition model with auxiliary deep semantic features. In the network module deployment phase, the auxiliary face recognition branch and the feature fusion and dimensionality reduction model are integrated into the normal face recognition model, and all three are integrated into a single model for deployment.

[0108] 302. Input the face image into the face recognition model in the integrated model to obtain the first feature.

[0109] In this embodiment, the integrated model includes a face recognition model and a deep semantic extraction branch. That is, an independent feature extraction branch is introduced into the deep structure of the face recognition model, and the normal face recognition model and the deep face recognition branch are trained independently. The deep feature extraction branch is used to assist in the integration of the recognition model. Therefore, the face recognition model includes a shallow network and a deep network. The shallow network is used to extract shallow features of the face image, and the deep network is used to extract the first feature from the shallow features.

[0110] The training process of the ensemble model is explained below. This ensemble model includes a face recognition model and a deep semantic extraction branch. It can also integrate subsequent fusion and dimensionality reduction models, that is, integrate the three into a single model for deployment.

[0111] Specifically, the training process for the face recognition model, the deep semantic extraction branch, and the fusion dimensionality reduction model is carried out step by step. First, training data for face recognition is acquired. Then, the training data is input into the ensemble model. Shallow features are extracted from the training data based on the shallow network, and training features are extracted from the shallow features of the training data based on the deep network to train the face recognition model. Next, the deep semantic extraction branch is trained based on the training data. During training, the deep semantic extraction branch uses the shallow network as feature input, and the parameters of this shallow network are fixed, meaning the parameters of the shallow network in the trained face recognition model are fixed. Finally, the training features and the deep features output by the deep semantic extraction branch are input into the fusion dimensionality reduction model to train it based on constraints. During the training of this fusion dimensionality reduction model, the parameters of the face recognition model and the deep semantic extraction branch are fixed, thus achieving a step-by-step training process and reducing the computational load of training.

[0112] In one possible scenario, the training process for a face recognition model is as follows: Figure 5 As shown, Figure 5 The flowchart illustrates another face recognition method provided in this application embodiment. First, training data is acquired based on a training data preparation module. Then, feature extraction is performed using a recognition network unit model. Specifically, shallow features of the training data are extracted first based on a shallow network, and training features are extracted based on the shallow features of the training data using a deep network. Next, a face recognition objective function calculation module is executed. This involves first determining the label information corresponding to the training data; then matching the training features with the label information to perform gradient calculation based on a first loss function; comparing the parameter values ​​in the gradient calculation process corresponding to the first loss function with the first training conditions. If the first training conditions (basic model training conditions) are met, the face recognition model training is completed; otherwise, the objective function is optimized, and the model is trained again.

[0113] Specifically, the training data preparation module in the diagram involves reading the face training data during training, combining the read data into a batch, and feeding it into the deep network unit for processing. The function of the recognition network unit module is to extract spatial features from the face images, and the output feature map retains the spatial structural information of the face image. This module typically uses a Convolutional Neural Network (CNN) structure, which includes operations such as convolution calculations, ReLU nonlinear activation function calculations, and pooling calculations.

[0114] Furthermore, the face recognition objective function calculation module takes the feature *f* output by the fully connected mapping unit and the label information of the face image that generated the vector as input to calculate the objective function value. The objective function can be a classification function (such as softmax, various types of softmax with margins), or other types of objective functions (the first loss function). The face recognition objective function optimization module trains and optimizes the entire network based on gradient descent (such as stochastic gradient descent, stochastic gradient descent with a driving term, Adam, Adamard). The training process is repeated until the training result meets the training termination condition. The termination condition is generally set to the number of iterations meeting a set value, or the loss calculated by the face recognition objective function being less than a set value.

[0115] To meet the time constraints of deployment scenarios while achieving a more stable improvement in model accuracy, this embodiment proposes a method using deep semantic features to integrate the features of a face recognition model. This improves recognition accuracy while ensuring the model meets the time requirements in different application scenarios. In face training using a deep network model, integrating multiple weaker recognition models (face recognition models) to integrate recognition features (deep semantic extraction branch integration) yields a stronger recognition model (ensemble model).

[0116] Understandably, the above embodiments adjust the structure of the recognition model by introducing two recognition branches into the deep semantic feature extraction network, and training each branch independently. This allows a deep network with fewer layers to acquire independent evaluation features for the same image, and then integrates the two features to obtain a stronger model.

[0117] 303. Input the shallow features of the face image into the deep semantic extraction branch to obtain the second feature.

[0118] In this embodiment, the deep semantic extraction branch uses features extracted by the shallow network during training. The network parameters of the deep semantic extraction branch are different from those of the deep network, which means that a shallow feature sharing mechanism is adopted, reducing the amount of computation during training and improving the diversity of features through the configuration of random parameters.

[0119] Specifically, the training process for the deep semantic extraction branch can be explained in conjunction with modular configuration, such as... Figure 6 As shown, Figure 6The flowchart illustrates another face recognition method provided in this application embodiment. It can execute a training data preparation module, which first determines the first sampling information for sampling training data during the face recognition model training process; then, it performs sampling configuration distinct from the first sampling information to determine the second sampling information corresponding to the deep semantic extraction branch. This means the random seed control for data sampling is inconsistent with the face recognition model training process; finally, it samples the training data based on the second sampling information to obtain training samples.

[0120] Then, the integrated network unit module is executed, that is, the training samples are input into the trained shallow network to obtain the sample input features; and the sample input features are input into the deep semantic extraction branch to obtain the sample output features; further, the label information corresponding to the sample output features is determined to perform gradient calculation of the second loss function on the deep semantic extraction branch; then, the parameter values ​​in the gradient calculation process of the second loss function are compared with the second training conditions. If the second training conditions are met (which can be the number of iterations or the loss value), the training of the deep semantic extraction branch is completed. If the second training conditions are not met, the above training process is executed again after adjusting the parameters of the second loss function.

[0121] In one possible scenario, to enrich the dimensions of features, the sample output features can be adjusted based on the random seed control module, that is, the random information of the parameters of the deep network in the face recognition model can be obtained; then, the random seed can be configured differently from the random information of the parameters of the deep network to configure the parameters of the deep semantic extraction branch; then the sample input features can be input into the deep semantic extraction branch after the parameter configuration to obtain the sample output features.

[0122] In addition, for the face recognition objective function calculation module, the parameters can be controlled through the loss hyperparameter control module. That is, firstly, the label information corresponding to the sample output features is determined to determine the quality parameters of the training samples; then, the quality parameters and the constraint parameters of the second loss function are configured differently from those of the face recognition model to adjust the parameters of the second loss function; and then the gradient is calculated for the deep semantic extraction branch based on the parameter-adjusted second loss function.

[0123] In one possible scenario, the face recognition network is divided into four layers (stage 1, stage 2, stage 3, stage 4). The following explanation uses this scenario as an example. Figure 7 As shown, Figure 7This is a schematic diagram illustrating another face recognition method provided in this application embodiment. The diagram shows the structure of the integrated model, where the face recognition network consists of four parts (stage1, stage2, stage3, and stage4). In this embodiment, a deep semantic extraction branch is introduced, sharing the features output by the shallow network stage3 with the face recognition network stage4 branch. In this training step, stages1 to stage4 are the parts where parameters are not updated; the training parameters update the deep semantic extraction branch. The deep semantic extraction branch typically has a convolutional neural network (CNN) structure, including operations such as convolution calculation, ReLU nonlinear activation function calculation, and pooling calculation. In this step, the parameters of the face recognition network stages1 to stage4 are inherited from the model parameters trained in step 302.

[0124] Based on the above scenario, for the training data preparation module, the training data read is normal human face data. In this step, the random seed control for data sampling is different from that in step 302, increasing the complementarity between the model training process and step 302.

[0125] The integrated network unit module is composed of the face recognition network unit module obtained in step 302 and the introduced deep semantic feature extraction branch.

[0126] For the face recognition objective function calculation module, this module calculates the features output by the deep semantic feature extraction branch. The probability vectors of the image belonging to each category, obtained by matrix multiplication with the category centers, and the label information of the face image that generated these vectors are used as inputs to calculate the objective function value. The objective function can be a classification function (such as softmax, or a softmax with margins for each category), or other types of objective functions. The loss calculated by this module is... .

[0127] For the random seed control module, this module performs random initialization of the deep semantic extraction branch. The initialization random seed control is different from the face recognition model in step 1, which enriches the diversity of the network.

[0128] The loss hyperparameter control module controls the loss function used for training the complementary model. Currently, the loss constraint used in face recognition training is generally a loss function with margin constraints, as shown in the following formula.

[0129]

[0130] Where s is the distance of the face image from the origin, and m is the constraint parameter.

[0131] In this embodiment, face images are clustered around the space of the identity vector, and their distance *s* from the origin represents the quality of the image. Therefore, when training deep semantic features, adjusting *s* and *m* to ensure the diversity of constraints on the images in space can promote the model's learning of complementary knowledge. Thus, the loss hyperparameter control module controls the configuration of *s* and *m*, and this configuration is inconsistent with the face recognition model.

[0132] The function of the face recognition objective function optimization module is the same as that of the module in step 302.

[0133] In another possible scenario, network search technology is employed to search for the deep semantic extraction branch, balancing network runtime and recognition accuracy. First, a network search is performed based on preset accuracy information (preset runtime, training scale, or recognition accuracy) to determine the parameters of the deep semantic extraction branch. Then, the parameters of the deep semantic extraction branch are configured according to the parameters obtained from the network search. Finally, shallow features of the face image are input into the parameter-configured deep semantic extraction branch to obtain the second feature, thereby improving the configuration efficiency of the deep semantic extraction branch.

[0134] 304. Input the first feature and the second feature into the fusion dimensionality reduction model for fusion to obtain the fused feature.

[0135] In this embodiment, the output features of the ensemble model are fused and dimensionality reduced. Since the feature output dimension of a single face recognition model is d, assuming no fusion and dimensionality reduction processing is performed, the output dimension of the model would be 2*d when using deep semantic features for ensemble-assisted recognition. This increased feature length leads to longer comparison time during deployment, incompatibility with the feature dimensions of existing comparison modules, and redundant information within the ensemble features. Therefore, this embodiment proposes a fusion and dimensionality reduction module to execute the fusion and dimensionality reduction model.

[0136] In one possible scenario, the training process of the fusion dimensionality reduction model is as follows: Figure 8 As shown, Figure 8 The flowchart shows another face recognition method provided in the embodiments of this application; that is, the integrated network model outputs 2*d-dimensional features (stage4 outputs d-dimensional features and deep semantic extraction branch outputs d-dimensional features) as input to the fusion dimensionality reduction module matrix.

[0137] Specifically, the fusion dimensionality reduction module is an independent neural network, which includes operations such as nonlinear activation function (ReLU) calculation, pooling, and fully connected layers. The constraint function for training this module is the face recognition constraint function used in step 303. During the training of this module, the parameters of the face recognition ensemble model are not updated.

[0138] In addition, the training process of the fusion dimensionality reduction module also includes a training condition judgment process, which can be the number of training iterations or the magnitude of the loss value. If the training condition is met, the training of the fusion dimensionality reduction module is completed; if not, the objective function is optimized and training is performed again.

[0139] This embodiment introduces two independent branches into the deep network of the same model, while its shallow network features a sharing mechanism. Therefore, the dual-branch model structure does not significantly increase processing time, meeting the deployment requirements of the recognition model. Finally, this embodiment introduces ensemble feature dimensionality reduction technology to reduce the dimensionality of the ensemble features to the same feature dimension as the original model, thereby satisfying the model deployment compatibility requirements.

[0140] 305. Perform feature comparison based on fused features to obtain the face-related recognition information corresponding to the target object.

[0141] In this embodiment, since the fusion feature integrates multi-dimensional image features, the accuracy of face recognition can be improved during the comparison process, thereby obtaining the face-related recognition information corresponding to the target object; the face-related recognition information may include whether the face image is a registered image, the descriptive information of the face image (name, gender, number, etc.) or the matching degree information of the face image.

[0142] It is understood that the ensemble model in this embodiment can merge the trained deep semantic extraction branch, the normal face recognition model, and the feature fusion and dimensionality reduction module. The features output by the normal face recognition model... The features output by the deep semantic extraction branch are After the two are concatenated, they enter the feature fusion and dimensionality reduction module to output the final integrated features. The integrated recognition model retains the same feature dimensions and can be seamlessly integrated into a conventional face recognition system. Therefore, this embodiment can improve the accuracy of the face recognition system with a slight increase in model processing time, making it adaptable to various complex application scenarios.

[0143] As described in the above embodiments, the process involves acquiring a facial image of the target object; then inputting the facial image into a face recognition model within an ensemble model to obtain a first feature. This ensemble model includes a face recognition model and a deep semantic extraction branch. The face recognition model comprises a shallow network and a deep network. The shallow network extracts shallow features from the facial image, while the deep network extracts the first feature from the shallow features. The shallow features of the facial image are then input into the deep semantic extraction branch to obtain a second feature. During training, the deep semantic extraction branch uses features extracted by the shallow network, and its network parameters differ from those of the deep network. The first and second features are then input into a fusion and dimensionality reduction model for fusion to obtain a fused feature. Finally, feature comparison is performed based on the fused feature to obtain the facial recognition information corresponding to the target object. This achieves a face recognition process that integrates deep features. By introducing an independent branch into the deep network of the ensemble model, the diversity of features is improved. Furthermore, the features of the shallow network are shared, which enhances both the accuracy and efficiency of face recognition.

[0144] To better implement the above-described solutions of the embodiments of this application, related apparatus for implementing the above solutions is also provided below. Please refer to... Figure 9 , Figure 9 This is a schematic diagram of the structure of a face recognition device provided in an embodiment of this application. The recognition device 900 includes:

[0145] Acquisition unit 901 is used to acquire the face image of the target object;

[0146] Input unit 902 is used to input the face image into the face recognition model in the integrated model to obtain a first feature. The integrated model includes the face recognition model and a deep semantic extraction branch. The face recognition model includes a shallow network and a deep network. The shallow network is used to extract shallow features of the face image.

[0147] The input unit 902 is further configured to input the shallow features into the deep semantic extraction branch to obtain the second feature. The deep semantic extraction branch uses the features extracted by the shallow network during the training process. The network parameters of the deep semantic extraction branch are different from the network parameters of the deep network.

[0148] The fusion unit 903 is used to input the first feature and the second feature into the fusion dimensionality reduction model for fusion to obtain the fused feature;

[0149] The recognition unit 904 is used to perform feature comparison based on the fused features to obtain face-related recognition information corresponding to the target object.

[0150] Optionally, in some possible implementations of this application, the recognition unit 904 is specifically used to acquire training data for face recognition;

[0151] The recognition unit 904 is specifically used to input the training data into the ensemble model;

[0152] The recognition unit 904 is specifically used to extract shallow features of the training data based on the shallow network, and to extract training features from the shallow features of the training data based on the deep network, so as to train the face recognition model.

[0153] The recognition unit 904 is specifically used to train the deep semantic extraction branch based on the training data. During the training process, the deep semantic extraction branch uses the shallow network for feature input, and the parameters of the shallow network are fixed.

[0154] The recognition unit 904 is specifically used to input the training features and the deep features output by the deep semantic extraction branch into the fusion dimensionality reduction model, so as to train the fusion dimensionality reduction model based on constraints. During the training process, the parameters of the face recognition model and the deep semantic extraction branch are fixed.

[0155] Optionally, in some possible implementations of this application, the identification unit 904 is specifically used to extract shallow features of the training data based on the shallow network, and to extract the training features based on the shallow features of the training data according to the deep network.

[0156] The identification unit 904 is specifically used to determine the label information corresponding to the training data;

[0157] The identification unit 904 is specifically used to match the training features with the label information to perform gradient calculation based on the first loss function;

[0158] The recognition unit 904 is specifically used to compare the parameter values ​​in the gradient calculation process corresponding to the first loss function with the first training conditions. If the first training conditions are met, the training of the face recognition model is completed.

[0159] Optionally, in some possible implementations of this application, the recognition unit 904 is specifically used to determine the first sampling information of the training data during the training process of the face recognition model;

[0160] The identification unit 904 is specifically used to distinguish the sampling configuration from the first sampling information in order to determine the second sampling information corresponding to the deep semantic extraction branch.

[0161] The identification unit 904 is specifically used to sample the training data based on the second sampling information to obtain training samples;

[0162] The recognition unit 904 is specifically used to input the training samples into the trained shallow network to obtain sample input features;

[0163] The recognition unit 904 is specifically used to input the sample input features into the deep semantic extraction branch to obtain the sample output features;

[0164] The identification unit 904 is specifically used to determine the label information corresponding to the sample output features, so as to perform gradient calculation of the second loss function on the deep semantic extraction branch;

[0165] The identification unit 904 is specifically used to compare the parameter values ​​in the gradient calculation process corresponding to the second loss function with the second training conditions. If the second training conditions are met, the training of the deep semantic extraction branch is completed.

[0166] Optionally, in some possible implementations of this application, the recognition unit 904 is specifically used to obtain random parameter information of the deep network in the face recognition model;

[0167] The identification unit 904 is specifically used to configure random seeds that are different from the random information of parameters of the deep network, so as to configure parameters of the deep semantic extraction branch.

[0168] The recognition unit 904 is specifically used to input the sample input features into the deep semantic extraction branch after the input parameters are configured, so as to obtain the sample output features.

[0169] Optionally, in some possible implementations of this application, the identification unit 904 is specifically used to determine the label information corresponding to the sample output features in order to determine the quality parameters of the training samples;

[0170] The recognition unit 904 is specifically used to configure the quality parameters and the constraint parameters of the second loss function differently from those of the face recognition model, so as to adjust the parameters of the second loss function;

[0171] The recognition unit 904 is specifically used to perform gradient calculation on the deep semantic extraction branch based on the second loss function after parameter adjustment.

[0172] Optionally, in some possible implementations of this application, the input unit 902 is specifically used to perform a network search based on preset precision information to determine the parameter information of the deep semantic extraction branch;

[0173] The input unit 902 is specifically used to configure the parameters of the deep semantic extraction branch according to the parameter information obtained from the network search;

[0174] The input unit 902 is specifically used to input the shallow features of the face image into the deep semantic extraction branch after parameter configuration, so as to obtain the second feature.

[0175] The process involves acquiring a facial image of the target object; then inputting the facial image into a face recognition model within an ensemble model to obtain the first feature. This ensemble model includes a face recognition model and a deep semantic extraction branch. The face recognition model comprises shallow and deep networks. The shallow network extracts shallow features from the facial image, while the deep network extracts the first feature from the shallow features. The shallow features of the facial image are then input into the deep semantic extraction branch to obtain the second feature. During training, the deep semantic extraction branch uses features extracted by the shallow network, and its network parameters differ from those of the deep network. The first and second features are then input into a fusion and dimensionality reduction model for fusion to obtain the fused feature. Finally, feature comparison is performed based on the fused feature to obtain the facial recognition information corresponding to the target object. This achieves a face recognition process that integrates deep features. By introducing an independent branch into the deep network of the ensemble model, the diversity of features is improved; and the features of the shallow network are shared, thus improving both the accuracy and efficiency of face recognition.

[0176] This application also provides a terminal device, such as... Figure 10 The diagram shown is a structural schematic of another terminal device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of this application. The terminal can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0177] Figure 10 This is a block diagram illustrating a portion of the structure of a mobile phone related to the terminal provided in the embodiments of this application. (Reference) Figure 10 The mobile phone includes components such as a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that... Figure 10The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0178] The following is combined with Figure 10 A detailed introduction to each component of a mobile phone:

[0179] The RF circuit 1010 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 1080; additionally, it transmits uplink data to the base station. Typically, the RF circuit 1010 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), and a duplexer. Furthermore, the RF circuit 1010 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Message Service (SMS).

[0180] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various mobile phone functions and data processing by running the software programs and modules stored in the memory 1020. The memory 1020 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 1020 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0181] The input unit 1030 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1031, as well as air touch operations within a certain range on the touch panel 1031), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. Furthermore, the touch panel 1031 can be implemented using various types of sensors, including resistive, capacitive, infrared, and surface acoustic wave sensors. In addition to the touch panel 1031, the input unit 1030 may also include other input devices 1032. Specifically, these other input devices 1032 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0182] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 1040 may include a display panel 1041, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Further, a touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it transmits the information to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides corresponding visual output on the display panel 1041 according to the type of touch event. Although in Figure 10 In this embodiment, the touch panel 1031 and the display panel 1041 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the mobile phone.

[0183] The mobile phone may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1041 according to the ambient light level, and the proximity sensor can turn off the display panel 1041 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0184] The audio circuit 1060, speaker 1061, and microphone 1062 provide an audio interface between the user and the mobile phone. The audio circuit 1060 converts the received audio data into electrical signals and transmits them to the speaker 1061, where the speaker 1061 converts them into sound signals for output. On the other hand, the microphone 1062 converts the collected sound signals into electrical signals, which are then received by the audio circuit 1060, converted into audio data, and then processed by the processor 1080 before being transmitted via the RF circuit 1010 to, for example, another mobile phone, or the audio data can be output to the memory 1020 for further processing.

[0185] WiFi is a short-range wireless transmission technology. Through the WiFi module 1070, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 10 The WiFi module 1070 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0186] The processor 1080 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 1020, and calls data stored in the memory 1020 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 1080.

[0187] The mobile phone also includes a power supply 1090 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0188] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0189] In this embodiment of the application, the processor 1080 included in the terminal also has the function of performing the various steps of the page processing method described above.

[0190] This application also provides a server; please refer to [link / reference]. Figure 11 , Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1100 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1122 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the server 1100.

[0191] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0192] The steps performed by the management device in the above embodiments can be based on this Figure 11 The server structure shown.

[0193] This application also provides a computer-readable storage medium storing instructions for face recognition, which, when executed on a computer, cause the computer to perform the aforementioned actions. Figures 3 to 8 The steps performed by the face recognition device in the method described in the illustrated embodiment.

[0194] This application also provides a computer program product that includes instructions for face recognition, which, when run on a computer, causes the computer to perform the aforementioned actions. Figures 3 to 8 The steps performed by the face recognition device in the method described in the illustrated embodiment.

[0195] This application also provides a face recognition system, which may include... Figure 9 The face recognition device described in the embodiments, or Figure 10 The terminal device in the described embodiments, or Figure 11 The server described.

[0196] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0199] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0200] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a facial recognition device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0201] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for facial recognition, characterized in that, include: Obtain the facial image of the target object; The face image is input into the face recognition model in the ensemble model to obtain the first feature. The ensemble model includes the face recognition model and a deep semantic extraction branch. The face recognition model and the deep semantic extraction branch are trained independently. The face recognition model includes a shallow network and a deep network. The shallow network is used to extract shallow features of the face image, and the deep network is used to extract the first feature from the shallow features. A network search is performed based on preset precision information to determine the parameter information of the deep semantic extraction branch; The parameters of the deep semantic extraction branch are configured based on the parameter information obtained from the network search. The shallow features of the face image are input into the deep semantic extraction branch after parameter configuration to obtain the second feature. The deep semantic extraction branch uses the features extracted by the shallow network during the training process. The network parameters of the deep semantic extraction branch are different from the network parameters of the deep network. The first feature and the second feature are input into a fusion dimensionality reduction model for fusion to obtain fused features; Feature comparison is performed based on the fused features to obtain the face-related recognition information corresponding to the target object.

2. The method according to claim 1, characterized in that, The method further includes: Obtain training data for face recognition; The training data is input into the ensemble model; The shallow features of the training data are extracted based on the shallow network, and the training features are extracted based on the shallow features of the training data by the deep network, so as to train the face recognition model. The deep semantic extraction branch is trained based on the training data. During the training process, the deep semantic extraction branch uses the shallow network as feature input, and the parameters of the shallow network are fixed. The training features and the deep features output by the deep semantic extraction branch are input into the fusion dimensionality reduction model to train the fusion dimensionality reduction model based on constraints. During the training process, the parameters of the face recognition model and the deep semantic extraction branch are fixed.

3. The method according to claim 2, characterized in that, The step of extracting shallow features from the training data based on the shallow network and extracting training features from the shallow features of the training data based on the deep network to train the face recognition model includes: The shallow features of the training data are extracted based on the shallow network, and the training features are extracted based on the shallow features of the training data by the deep network. Determine the label information corresponding to the training data; The training features are matched with the label information to perform gradient calculation based on the first loss function; The parameter values ​​in the gradient calculation process corresponding to the first loss function are compared with the first training conditions. If the first training conditions are met, the training of the face recognition model is completed.

4. The method according to claim 2, characterized in that, The training of the deep semantic extraction branch based on the training data includes: Determine the first sampling information for sampling the training data during the training process of the face recognition model; The sampling configuration is different from the first sampling information to determine the second sampling information corresponding to the deep semantic extraction branch; The training data is sampled based on the second sampling information to obtain training samples; The training samples are input into the trained shallow network to obtain the sample input features; The sample input features are input into the deep semantic extraction branch to obtain the sample output features; Determine the label information corresponding to the output features of the sample in order to calculate the gradient of the second loss function for the deep semantic extraction branch; The parameter values ​​in the gradient calculation process corresponding to the second loss function are compared with the second training conditions. If the second training conditions are met, the training of the deep semantic extraction branch is completed.

5. The method according to claim 4, characterized in that, The step of inputting the sample input features into the deep semantic extraction branch to obtain sample output features includes: Obtain random parameter information of the deep network in the face recognition model; Unlike the random seed configuration of parameters in deep networks, this method configures parameters for the deep semantic extraction branch. The deep semantic extraction branch, after configuring the input parameters of the sample input features, is used to obtain the sample output features.

6. The method according to claim 4, characterized in that, The step of determining the label information corresponding to the sample output features, and calculating the gradient of the second loss function for the deep semantic extraction branch, includes: Determine the label information corresponding to the output features of the sample in order to determine the quality parameters of the training sample; The quality parameters and the constraint parameters of the second loss function are configured differently from those of the face recognition model in order to adjust the parameters of the second loss function; The gradient of the deep semantic extraction branch is calculated based on the second loss function after parameter adjustment.

7. A facial recognition device, characterized in that, include: The acquisition unit is used to acquire the face image of the target object; An input unit is used to input the face image into the face recognition model in the integrated model to obtain a first feature. The integrated model includes the face recognition model and a deep semantic extraction branch. The face recognition model and the deep semantic extraction branch are trained independently. The face recognition model includes a shallow network and a deep network. The shallow network is used to extract shallow features of the face image, and the deep network is used to extract the first feature from the shallow features. The input unit is further configured to input the shallow features into the deep semantic extraction branch to obtain the second feature. The deep semantic extraction branch uses the features extracted by the shallow network during the training process. The network parameters of the deep semantic extraction branch are different from the network parameters of the deep network. The fusion unit is used to input the first feature and the second feature into the fusion dimensionality reduction model for fusion to obtain the fused feature; The recognition unit is used to perform feature comparison based on the fused features to obtain face-related recognition information corresponding to the target object; The input unit is specifically used to perform a network search based on preset precision information in order to determine the parameter information of the deep semantic extraction branch; The input unit is specifically used to configure the parameters of the deep semantic extraction branch based on the parameter information obtained from the network search. The input unit is specifically used to input the shallow features of the face image into the deep semantic extraction branch after parameter configuration, so as to obtain the second feature.

8. The apparatus according to claim 7, characterized in that, The recognition unit is specifically used to acquire training data for face recognition; The recognition unit is specifically used to input the training data into the ensemble model; The recognition unit is specifically used to extract shallow features of the training data based on the shallow network, and to extract training features from the shallow features of the training data based on the deep network, so as to train the face recognition model. The recognition unit is specifically used to train the deep semantic extraction branch based on the training data. The deep semantic extraction branch uses the shallow network as feature input during the training process, and the parameters of the shallow network are fixed. The recognition unit is specifically used to input the training features and the deep features output by the deep semantic extraction branch into the fusion dimensionality reduction model, so as to train the fusion dimensionality reduction model based on constraints. During the training process, the parameters of the face recognition model and the deep semantic extraction branch are fixed.

9. The apparatus according to claim 8, characterized in that, The recognition unit is specifically used to extract shallow features of the training data based on the shallow network, and to extract training features based on the shallow features of the training data according to the deep network. The identification unit is specifically used to determine the label information corresponding to the training data; The identification unit is specifically used to match the training features with the label information to perform gradient calculation based on the first loss function; The recognition unit is specifically used to compare the parameter values ​​in the gradient calculation process corresponding to the first loss function with the first training conditions. If the first training conditions are met, the training of the face recognition model is completed.

10. The apparatus according to claim 8, characterized in that, The recognition unit is specifically used to determine the first sampling information of the training data during the training process of the face recognition model; The identification unit is specifically used to distinguish the sampling configuration from the first sampling information in order to determine the second sampling information corresponding to the deep semantic extraction branch. The identification unit is specifically used to sample the training data based on the second sampling information to obtain training samples; The recognition unit is specifically used to input the training samples into the trained shallow network to obtain sample input features; The recognition unit is specifically used to input the sample input features into the deep semantic extraction branch to obtain the sample output features; The identification unit is specifically used to determine the label information corresponding to the sample output features, so as to perform gradient calculation of the second loss function on the deep semantic extraction branch; The identification unit is specifically used to compare the parameter values ​​in the gradient calculation process corresponding to the second loss function with the second training conditions. If the second training conditions are met, the training of the deep semantic extraction branch is completed.

11. The apparatus according to claim 10, characterized in that, The recognition unit is specifically used to obtain random parameter information of the deep network in the face recognition model; The identification unit is specifically used to configure random seeds that are different from the random parameter information of the deep network, so as to configure the parameters of the deep semantic extraction branch. The recognition unit is specifically used to input the sample input features into the deep semantic extraction branch after the input parameters are configured, so as to obtain the sample output features.

12. The apparatus according to claim 10, characterized in that, The recognition unit is specifically used to determine the label information corresponding to the output features of the sample, so as to determine the quality parameters of the training sample. The recognition unit is specifically configured to configure the quality parameters and the constraint parameters of the second loss function differently from those of the face recognition model, so as to adjust the parameters of the second loss function; The recognition unit is specifically used to perform gradient calculation on the deep semantic extraction branch based on the second loss function after parameter adjustment.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store program code; the processor is used to execute the face recognition method according to any one of claims 1 to 6 according to the instructions in the program code.

14. A computer program product comprising a computer program / instructions stored in a computer-readable storage medium, characterized in that, When the computer program / instructions in the computer-readable storage medium are executed by a processor, they implement the steps of the face recognition method according to any one of claims 1 to 6.

15. A computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the face recognition method according to any one of claims 1 to 6.

Images