Face recognition method and system based on multi-model cooperation

Abstract

The application discloses a face recognition method and system based on multi-model cooperation, which comprises the following steps: obtaining face image data to be recognized and corresponding scene features; determining a plurality of feature extraction models and classification models corresponding to the scene features; the plurality of feature extraction models and classification models are trained by a plurality of different mask strategy processing training data sets corresponding to the scene features; inputting the face image data into the plurality of feature extraction models to obtain a plurality of image features corresponding to the face image; and determining a classification recognition result corresponding to the face image data according to the plurality of image features and the classification model. It can be seen that the application can improve the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and suitable for diversified environments, and improve the intelligent degree and automation degree of face recognition service.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a face recognition method and system based on multi-model cooperation. BACKGROUND

[0002] In recent years, with the widespread use of face recognition devices, "brushing faces" has become an essential action in daily life. With the rapid development of algorithm technology, how to improve the accuracy of face recognition with the help of algorithm technology has become an important technical problem. However, in the existing face recognition technology, most of the single models are trained based on a single training data set, and the advantages of multi-model algorithms trained based on multi-mask strategy training data sets are not fully combined to realize face recognition. Obviously, the accuracy and efficiency of face recognition are low, and the intelligent degree and automation degree are lacking. It can be seen that the existing technology has defects and needs to be solved. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a face recognition method and system based on multi-model cooperation, which can improve the adaptability and accuracy of face recognition in different scenarios, make the recognition result more robust and suitable for diversified environment, and improve the intelligent degree and automation degree of face recognition service.

[0004] To solve the above technical problems, the first aspect of the present application discloses a face recognition method based on multi-model cooperation, which comprises:

[0005] Obtaining face image data to be recognized and corresponding scene features;

[0006] According to the scene features, a plurality of feature extraction models and classification models corresponding to the scene features are determined; the plurality of feature extraction models and classification models are trained by a plurality of different mask strategy processing training data sets corresponding to the scene features;

[0007] The face image data is input into the plurality of feature extraction models to obtain a plurality of image features corresponding to the face image;

[0008] According to the plurality of image features and the classification model, a classification recognition result corresponding to the face image data is determined.

[0009] As an optional implementation, in the first aspect of the present application, the scene features include at least one of recognition location, environment light, scene type, user information and recognition purpose.

[0010] As an optional implementation, in the first aspect of the present application, the plurality of feature extraction models and classification models corresponding to the scene features are determined, comprising:

[0011] determining a masking prediction parameter corresponding to the scene feature;

[0012] For each candidate feature extraction model, determining a mask strategy parameter corresponding to a training data set corresponding to the candidate feature extraction model;

[0013] calculating a first similarity between the mask strategy parameter and the masking prediction parameter to obtain a first model priority corresponding to the candidate feature extraction model;

[0014] filtering out all the candidate feature models whose first model priority is greater than a preset first priority threshold to obtain a plurality of corresponding feature extraction models;

[0015] determining at least one classification model according to the mask strategy parameter corresponding to the training data set corresponding to the plurality of feature extraction models.

[0016] As an optional implementation, in the first aspect of the present application, the determination of the masking prediction parameter corresponding to the scene feature comprises:

[0017] inputting the scene feature into a trained masking parameter prediction neural network to obtain the masking prediction parameter corresponding to the scene feature; the masking parameter prediction neural network is trained by a training data set comprising a plurality of training scene images and corresponding scene feature annotations and masking condition annotations.

[0018] As an optional implementation, in the first aspect of the present application, the mask strategy parameter and the masking prediction parameter each comprise at least one of a masking image position, a masking part size, a masking part area ratio, and a masking part three-dimensional information; the first similarity is calculated by the following steps:

[0019] inputting the mask strategy parameter into a trained masking vector feature extraction neural network to obtain a first effect feature vector corresponding to the mask strategy parameter;

[0020] inputting the masking prediction parameter into the masking vector feature extraction neural network to obtain a second effect feature vector corresponding to the masking prediction parameter; the masking vector feature extraction neural network is trained by a training data set comprising a plurality of training masking parameters and corresponding pre-masking image annotations and post-masking image annotations;

[0021] calculating the inverse of the vector distance between the first effect feature vector and the second effect feature vector to obtain the first similarity.

[0022] As an optional implementation, in the first aspect of the present application, the determining of the at least one classification model according to the mask strategy parameters corresponding to the training data sets of the plurality of feature extraction models comprises:

[0023] For each candidate classification model, the second similarity between the mask strategy parameters corresponding to the training data set of the candidate classification model and the mask strategy parameters corresponding to the training data set of each feature extraction model is calculated based on the masking vector feature extraction neural network and the vector distance algorithm;

[0024] The average value of the second similarity of the candidate classification model and each feature extraction model is calculated to obtain the second model priority corresponding to the candidate classification model;

[0025] All candidate classification models with a second model priority greater than a preset second priority threshold are screened to obtain the corresponding at least one classification model.

[0026] As an optional implementation, in the first aspect of the present application, the candidate feature extraction model and the candidate classification model are obtained by training through the following steps:

[0027] A plurality of training image data sets processed by different mask strategy parameters are determined;

[0028] A preset prediction base model is trained based on each training image data set to obtain a plurality of training primary prediction models; the prediction base model and the primary prediction model comprise a feature extraction model and a classification model;

[0029] All the training image data sets are determined as common data sets;

[0030] Based on the common data sets, all the primary prediction models are commonly trained, and model parameter optimization is performed based on a common loss function related to the mask strategy parameters to obtain the trained candidate feature extraction model and the candidate classification model; the common loss function is a weighted sum value of the loss function values of each primary prediction model; wherein the weighted weight corresponding to the loss function value of each primary prediction model is proportional to the masking degree of the mask strategy parameters corresponding to the training image data set of the primary prediction model; wherein the masking degree is calculated by the following steps:

[0031] The mask strategy parameters corresponding to the training image data set of the primary prediction model are input into the masking vector feature extraction neural network to obtain the third effect feature vector corresponding to the mask strategy parameters;

[0032] calculate a first vector distance between the third effect feature vector and a first reference feature vector corresponding to a standard unoccluded image;

[0033] calculate a second vector distance between the third effect feature vector and a second reference feature vector corresponding to a standard full occlusion image; the first reference feature vector and the second reference feature vector are both obtained by pre-analyzing vector extraction results of a preset data set by the mask vector feature extraction neural network;

[0034] calculate a ratio of the first vector distance and the second vector distance to obtain the mask degree corresponding to the preliminary prediction model.

[0035] As an optional implementation, in the first aspect of the present application, the determining of the classification recognition result corresponding to the face image data according to the plurality of image features and the classification model comprises:

[0036] randomly sampling and combining the plurality of image features based on a random combination algorithm to obtain a plurality of combined feature sets;

[0037] inputting each combined feature set into the classification model to obtain a set of recognition results;

[0038] calculating the intersection of all the set of recognition results to obtain the classification recognition result corresponding to the face image data.

[0039] The second aspect of the embodiment of the present application discloses a face recognition system based on multi-model cooperation, and the system comprises:

[0040] an acquisition module configured to acquire face image data to be recognized and corresponding scene features;

[0041] a determination module configured to determine a plurality of feature extraction models and classification models corresponding to the scene features; the plurality of feature extraction models and classification models are trained by a plurality of training data sets processed by different mask strategies corresponding to the scene features;

[0042] an extraction module configured to input the face image data into the plurality of feature extraction models to obtain a plurality of image features corresponding to the face image data;

[0043] a recognition module configured to determine a classification recognition result corresponding to the face image data according to the plurality of image features and the classification model.

[0044] As an optional implementation, in the second aspect of the present application, the scene features comprise at least one of a recognition location, ambient light, a scene type, user information and a recognition purpose.

[0045] As an optional implementation, in the second aspect of the present application, the specific manner in which the determining module determines the corresponding plurality of feature extraction models and classification models according to the scene feature comprises:

[0046] determining the occlusion prediction parameter corresponding to the scene feature;

[0047] For each candidate feature extraction model, determining the mask strategy parameter corresponding to the training data set corresponding to the candidate feature extraction model;

[0048] calculating the first similarity between the mask strategy parameter and the occlusion prediction parameter to obtain the first model priority corresponding to the candidate feature extraction model;

[0049] screening all the candidate feature models whose first model priority is greater than a preset first priority threshold to obtain the corresponding plurality of feature extraction models;

[0050] determining at least one classification model according to the mask strategy parameter corresponding to the training data set corresponding to the plurality of feature extraction models.

[0051] As an optional implementation, in the second aspect of the present application, the specific manner in which the determining module determines the occlusion prediction parameter corresponding to the scene feature comprises:

[0052] inputting the scene feature into a trained occlusion parameter prediction neural network to obtain the occlusion prediction parameter corresponding to the scene feature; the occlusion parameter prediction neural network is trained by a training data set comprising a plurality of training scene images and corresponding scene feature annotations and occlusion condition annotations.

[0053] As an optional implementation, in the second aspect of the present application, the mask strategy parameter and the occlusion prediction parameter each comprise at least one of an occlusion image position, an occlusion part size, an occlusion part area ratio, and an occlusion part three-dimensional information; the first similarity is calculated by the following steps:

[0054] inputting the mask strategy parameter into a trained occlusion vector feature extraction neural network to obtain a first effect feature vector corresponding to the mask strategy parameter;

[0055] inputting the occlusion prediction parameter into the occlusion vector feature extraction neural network to obtain a second effect feature vector corresponding to the occlusion prediction parameter; the occlusion vector feature extraction neural network is trained by a training data set comprising a plurality of training occlusion parameters and corresponding pre-occlusion image annotations and post-occlusion image annotations;

[0056] Calculate the inverse of the vector distance between the first effect feature vector and the second effect feature vector to obtain the first similarity.

[0057] As an optional implementation, in the second aspect of the present application, the specific manner of determining the at least one classification model according to the mask strategy parameters corresponding to the training data sets of the plurality of feature extraction models comprises:

[0058] For each candidate classification model, based on the masking vector feature extraction neural network and the vector distance algorithm, calculate the second similarity between the mask strategy parameters corresponding to the training data set of the candidate classification model and the mask strategy parameters corresponding to the training data set of each feature extraction model;

[0059] Calculate the average of the second similarities of the candidate classification model and each feature extraction model to obtain the second model priority corresponding to the candidate classification model;

[0060] Screen all the candidate classification models whose second model priority is greater than a preset second priority threshold to obtain the corresponding at least one classification model.

[0061] As an optional implementation, in the second aspect of the present application, the candidate feature extraction model and the candidate classification model are obtained by training through the following steps:

[0062] Determine a plurality of training image data sets after mask processing using different mask strategy parameters;

[0063] Train a preset prediction base model based on each training image data set to obtain a plurality of training obtained preliminary prediction models; the prediction base model and the preliminary prediction model include a feature extraction model and a classification model;

[0064] Determine all the training image data sets as common data sets;

[0065] Based on the common data sets, commonly train all the preliminary prediction models, and perform model parameter optimization based on a common loss function related to the mask strategy parameters to obtain the trained candidate feature extraction model and the candidate classification model; the common loss function is a weighted sum value of the loss function values of each preliminary prediction model; wherein the weighted weight corresponding to the loss function value of each preliminary prediction model is proportional to the masking degree of the mask strategy parameters corresponding to the training image data set of the preliminary prediction model; wherein the masking degree is calculated by the following steps:

[0066] inputting the mask strategy parameter corresponding to the training image data set corresponding to the preliminary prediction model into the masking vector feature extraction neural network to obtain a third effect feature vector corresponding to the mask strategy parameter;

[0067] calculating a first vector distance between the third effect feature vector and a first reference feature vector corresponding to a standard non-occluded image;

[0068] calculating a second vector distance between the third effect feature vector and a second reference feature vector corresponding to a standard full-occluded image; the first reference feature vector and the second reference feature vector are both obtained by pre-analyzing the vector extraction result of a preset data set by the masking vector feature extraction neural network;

[0069] calculating the ratio of the first vector distance and the second vector distance to obtain the masking degree corresponding to the preliminary prediction model.

[0070] As an optional implementation, in the second aspect of the present application, the specific manner in which the identification module determines the classification identification result corresponding to the face image data according to the plurality of image features and the classification model comprises:

[0071] randomly sampling and combining the plurality of image features based on a random combination algorithm to obtain a plurality of combined feature sets;

[0072] inputting each combined feature set into the classification model to obtain a set identification result outputted by the classification model;

[0073] calculating the intersection of all the set identification results to obtain the classification identification result corresponding to the face image data.

[0074] The third aspect of the present application discloses another face recognition system based on multi-model cooperation, which comprises:

[0075] a memory storing executable program codes;

[0076] a processor coupled with the memory;

[0077] The processor invokes the executable program codes stored in the memory to execute part or all of the steps of the face recognition method based on multi-model cooperation disclosed in the first aspect of the present application.

[0078] The fourth aspect of the present application discloses a computer storage medium storing computer instructions, which are invoked to execute part or all of the steps of the face recognition method based on multi-model cooperation disclosed in the first aspect of the present application.

[0079] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0080] The present application can select corresponding feature extraction models and classification models trained by processing training data sets through multiple different mask strategies based on scene characteristics, extract multiple image features of a face image using multiple feature extraction models, and determine a classification recognition result in combination with a classification model, thereby improving the adaptability and accuracy of face recognition in different scenes, making the recognition result more robust and suitable for diversified environments, and improving the intelligent degree and automation degree of face recognition services. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0082] Figure 1 is a flow diagram of a face recognition method based on multi-model cooperation disclosed by the embodiment of the present application.

[0083] Figure 2 is a structural diagram of a face recognition system based on multi-model cooperation disclosed by the embodiment of the present application.

[0084] Figure 3 is a structural diagram of another face recognition system based on multi-model cooperation disclosed by the embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the personnel in the technical field better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0086] The terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or equipment.

[0087] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in one embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are they necessarily all directed to the same embodiment, nor are they necessarily all directed to a single alternative embodiment. Those skilled in the art will recognize that the embodiments described herein can be combined with other embodiments.

[0088] The application discloses a face recognition method and system based on multi-model cooperation, which can select corresponding feature extraction models and classification models trained by training data sets processed by multiple different mask strategies based on scene features, extract multiple image features of a face image using multiple feature extraction models, and determine a classification recognition result in combination with a classification model, so as to improve the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and suitable for diversified environments, and improve the intelligent degree and automation degree of face recognition services. The following will be described in detail.

[0089] Embodiment one

[0090] Please refer to Figure 1 , Figure 1 is a flowchart of a face recognition method based on multi-model cooperation disclosed by the embodiment of the application. Among them, Figure 1 The face recognition method based on multi-model cooperation described above can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 1 shown, the face recognition method based on multi-model cooperation can include the following operations:

[0091] 101, obtaining face image data to be recognized and corresponding scene features.

[0092] 102, determining multiple feature extraction models and classification models corresponding to the scene features.

[0093] Optionally, the multiple feature extraction models and classification models are trained by training data sets processed by multiple different mask strategies corresponding to the scene features.

[0094] 103, inputting the face image data to the multiple feature extraction models to obtain multiple image features corresponding to the face image.

[0095] 104, determining a classification recognition result corresponding to the face image data according to the multiple image features and the classification model.

[0096] It can be seen that the above embodiments can select the feature extraction model and the classification model trained based on the corresponding training data set processed by multiple different mask strategies according to the scene features, extract multiple image features of the face image by using the multiple feature extraction models, and determine the classification recognition result by combining the classification model, so as to improve the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and suitable for diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0097] As an optional embodiment, in the above step, the scene features include at least one of a recognition place, an ambient light, a scene type, user information, and a recognition purpose.

[0098] It can be seen that the above optional embodiments limit the content of the scene features to comprehensively represent the features related to the face recognition scene, so as to facilitate subsequent accurate face recognition, assist in improving the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and suitable for diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0099] As an optional embodiment, in the above step, according to the scene features, the multiple feature extraction models and the classification model corresponding to the scene features are determined, including:

[0100] determining the mask prediction parameter corresponding to the scene features;

[0101] for each candidate feature extraction model, determining the mask strategy parameter corresponding to the training data set corresponding to the candidate feature extraction model;

[0102] calculating the first similarity between the mask strategy parameter and the mask prediction parameter to obtain the first model priority of the candidate feature extraction model;

[0103] selecting all candidate feature models whose first model priority is greater than a preset first priority threshold to obtain the corresponding multiple feature extraction models;

[0104] determining at least one classification model according to the mask strategy parameter corresponding to the training data set corresponding to the multiple feature extraction models.

[0105] It can be seen that the above optional embodiments determine the mask prediction parameter based on the scene features, calculate the similarity between the mask strategy parameter of the candidate feature extraction model and the mask prediction parameter, select the feature extraction model whose priority is higher than the threshold, and determine the classification model according to the mask strategy parameter of the selected feature extraction model, so as to determine the feature extraction model and the classification model that are more adaptive and accurate, assist in improving the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and suitable for diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0106] As an optional embodiment, in the above step, the determining of the occlusion prediction parameter corresponding to the scene feature comprises:

[0107] inputting the scene feature into the trained occlusion parameter prediction neural network to obtain the occlusion prediction parameter corresponding to the scene feature; the occlusion parameter prediction neural network is trained by a training data set comprising a plurality of training scene images and corresponding scene feature labels and occlusion condition labels.

[0108] As can be seen, through the above optional embodiment, the corresponding occlusion prediction parameter can be obtained by inputting the scene feature into the occlusion parameter prediction neural network, and the correspondence between the scene feature and the occlusion condition in the training data set is learned to improve the accuracy and adaptability of the occlusion prediction parameter, thereby assisting in improving the adaptability and accuracy of face recognition in different scenes, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition service.

[0109] As an optional embodiment, in the above step, the mask strategy parameter and the occlusion prediction parameter each comprise at least one of an occlusion image position, an occlusion part size, an occlusion part area ratio, and occlusion part three-dimensional information; the first similarity is calculated by the following steps:

[0110] inputting the mask strategy parameter into the trained occlusion vector feature extraction neural network to obtain a first effect feature vector corresponding to the mask strategy parameter;

[0111] inputting the occlusion prediction parameter into the occlusion vector feature extraction neural network to obtain a second effect feature vector corresponding to the occlusion prediction parameter; optionally, the occlusion vector feature extraction neural network is trained by a training data set comprising a plurality of training occlusion parameters and corresponding pre-occlusion image labels and post-occlusion image labels;

[0112] calculating the reciprocal of the vector distance between the first effect feature vector and the second effect feature vector to obtain the first similarity.

[0113] As can be seen, through the above optional embodiment, the mask strategy parameter and the occlusion prediction parameter are respectively converted into effect feature vectors based on the occlusion vector feature extraction neural network, and the similarity is calculated by the vector distance to quantify the matching degree between different occlusion strategies and predicted occlusion conditions, thereby improving the adaptability and accuracy of the feature extraction model selection, assisting in improving the adaptability and accuracy of face recognition in different scenes, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition service.

[0114] As an optional embodiment, in the above step, the determining the at least one classification model according to the mask strategy parameters corresponding to the training data set of each feature extraction model comprises:

[0115] For each candidate classification model, the second similarity between the mask strategy parameters corresponding to the training data set of the candidate classification model and the mask strategy parameters corresponding to the training data set of each feature extraction model is calculated based on the masking vector feature extraction neural network and the vector distance algorithm.

[0116] The average value of the second similarity between the candidate classification model and each feature extraction model is calculated to obtain the second model priority corresponding to the candidate classification model.

[0117] All candidate classification models with a second model priority greater than a preset second priority threshold are screened to obtain the corresponding at least one classification model.

[0118] As can be seen, through the above optional embodiment, the similarity between the mask strategy parameters of the candidate classification model and the mask strategy parameters of the feature extraction model is calculated based on the masking vector feature extraction neural network and the vector distance algorithm, and the priority of the classification model is determined by the average similarity, so as to screen the classification model that matches the feature extraction model more, thereby improving the overall recognition accuracy, assisting in improving the adaptability and accuracy of face recognition in different scenarios, making the recognition result more robust and suitable for diversified environments, and improving the intelligent degree and automation degree of face recognition service.

[0119] As an optional embodiment, in the above step, the candidate feature extraction model and the candidate classification model are trained by the following steps:

[0120] A plurality of training image data sets processed by different mask strategy parameters are determined;

[0121] Each training image data set is used to train a preset prediction base model to obtain a plurality of preliminary prediction models trained, and optionally, the prediction base model and the preliminary prediction model comprise a feature extraction model and a classification model.

[0122] All training image data sets are determined as a common data set.

[0123] The common data set is used to jointly train all the preliminary prediction models, and the model parameters are optimized based on a common loss function related to the mask strategy parameters, to obtain a trained candidate feature extraction model and a candidate classification model; optionally, the common loss function is a weighted sum of loss function values of each preliminary prediction model; wherein the weighted weight corresponding to the loss function value of each preliminary prediction model is proportional to the masking degree of the mask strategy parameter corresponding to the training image data set of the preliminary prediction model; wherein the masking degree is calculated by the following steps:

[0124] The mask strategy parameter corresponding to the training image data set of the preliminary prediction model is input into the masking vector feature extraction neural network to obtain a third effect feature vector corresponding to the mask strategy parameter;

[0125] The first vector distance between the third effect feature vector and the first reference feature vector corresponding to the standard unoccluded image is calculated;

[0126] The second vector distance between the third effect feature vector and the second reference feature vector corresponding to the standard fully occluded image is calculated; optionally, the first reference feature vector and the second reference feature vector are both obtained by pre-analyzing the vector extraction results of the preset data set based on the masking vector feature extraction neural network;

[0127] The ratio of the first vector distance and the second vector distance is calculated to obtain the masking degree corresponding to the preliminary prediction model.

[0128] As can be seen, through the above optional embodiment, multiple training image data sets are generated based on different mask strategy parameters, and all preliminary prediction models are jointly trained based on a common data set, and the model parameters are optimized using a common loss function related to the mask strategy parameters, to train a feature extraction model and a classification model that are more suitable for masking conditions, thereby assisting in improving the adaptability and accuracy of face recognition in different scenarios, making the recognition result more robust and suitable for diversified environments, and improving the intelligent and automated level of face recognition services.

[0129] As an optional embodiment, in the above steps, determining the classification recognition result corresponding to the face image data according to the plurality of image features and the classification model, comprises:

[0130] Randomly sampling and combining the plurality of image features based on a random combination algorithm to obtain a plurality of combined feature sets;

[0131] Each combined feature set is input into the classification model to obtain an output set recognition result;

[0132] The intersection of all set recognition results is calculated to obtain the classification recognition result corresponding to the face image data.

[0133] It can be seen that, through the above optional embodiments, the multiple image features are sampled and combined based on a random combination algorithm, and the recognition results of each combined feature set are calculated through a classification model, and the intersection of all recognition results is taken to reduce feature interference and misclassification, improve the adaptability and accuracy of face recognition in different scenarios, make the recognition result more robust and applicable to diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0134] Embodiment Two

[0135] Please refer to Figure 2 , Figure 2 is a structural schematic diagram of a face recognition system based on multi-model cooperation disclosed by the embodiments of the present application. Among them, Figure 2 The face recognition system based on multi-model cooperation described can be applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 2 indicated, the face recognition system based on multi-model cooperation can include:

[0136] The acquisition module 201 is configured to acquire face image data to be recognized and corresponding scene features.

[0137] The determination module 202 is configured to determine a plurality of feature extraction models and classification models corresponding to the scene features according to the scene features.

[0138] Optionally, the plurality of feature extraction models and classification models are trained by a plurality of training data sets processed by different mask strategies corresponding to the scene features.

[0139] The extraction module 203 is configured to input the face image data into the plurality of feature extraction models to obtain a plurality of image features corresponding to the face image.

[0140] The recognition module 204 is configured to determine a classification recognition result corresponding to the face image data according to the plurality of image features and the classification model.

[0141] It can be seen that the above embodiments of the present application can select corresponding feature extraction models and classification models trained by a plurality of training data sets processed by different mask strategies based on scene features, and extract a plurality of image features of a face image using a plurality of feature extraction models, and determine a classification recognition result in combination with a classification model, thereby improving the adaptability and accuracy of face recognition in different scenarios, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition service.

[0142] As an optional embodiment, the scene features include at least one of recognition location, environment light, scene type, user information and recognition purpose.

[0143] It can be seen that through the above optional embodiments, the content of the scene feature is defined to comprehensively represent the features related to the face recognition scene, so as to facilitate subsequent accurate face recognition, assist in improving the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and applicable to diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0144] As an optional embodiment, the specific manner in which the determination module determines the corresponding multiple feature extraction models and classification models according to the scene feature includes:

[0145] determining the occlusion prediction parameter corresponding to the scene feature;

[0146] For each candidate feature extraction model, determining the mask strategy parameter corresponding to the training data set corresponding to the candidate feature extraction model;

[0147] calculating the first similarity between the mask strategy parameter and the occlusion prediction parameter to obtain the first model priority of the candidate feature extraction model;

[0148] filtering out all candidate feature models whose first model priority is greater than a preset first priority threshold to obtain the corresponding multiple feature extraction models;

[0149] determining at least one classification model according to the mask strategy parameter corresponding to the training data set corresponding to the multiple feature extraction models.

[0150] It can be seen that through the above optional embodiments, the occlusion prediction parameter is determined based on the scene feature, and the similarity between the mask strategy parameter of the candidate feature extraction model and the occlusion prediction parameter is calculated, the feature extraction model with a priority higher than the threshold is filtered out, and the classification model is determined according to the mask strategy parameter of the selected feature extraction model, so as to determine the feature extraction model and the classification model that are more adaptive and accurate, assist in improving the adaptability and accuracy of face recognition in different scenes, make the recognition result more robust and applicable to diversified environments, and improve the intelligent degree and automation degree of face recognition service.

[0151] As an optional embodiment, the specific manner in which the determination module determines the occlusion prediction parameter corresponding to the scene feature includes:

[0152] inputting the scene feature into the trained occlusion parameter prediction neural network to obtain the occlusion prediction parameter corresponding to the scene feature; the occlusion parameter prediction neural network is trained by a training data set including multiple training scene images and corresponding scene feature labels and occlusion condition labels.

[0153] It can be seen that, through the above optional embodiments, the scene feature can be input to the masking parameter prediction neural network to obtain the corresponding masking prediction parameter, and the corresponding relationship between the scene feature and the masking condition in the training data set is learned to improve the accuracy and adaptability of the masking prediction parameter, thereby assisting in improving the adaptability and accuracy of face recognition in different scenes, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition services.

[0154] As an optional embodiment, the mask strategy parameter and the masking prediction parameter each include at least one of a masking image position, a masking part size, a masking part area ratio, and a masking part three-dimensional information; the first similarity is calculated by the following steps:

[0155] The mask strategy parameter is input to the trained masking vector feature extraction neural network to obtain a first effect feature vector corresponding to the mask strategy parameter;

[0156] The masking prediction parameter is input to the masking vector feature extraction neural network to obtain a second effect feature vector corresponding to the masking prediction parameter; optionally, the masking vector feature extraction neural network is trained by a training data set including a plurality of training masking parameters and corresponding pre-masking image labels and post-masking image labels;

[0157] The reciprocal of the vector distance between the first effect feature vector and the second effect feature vector is calculated to obtain the first similarity.

[0158] It can be seen that, through the above optional embodiments, the mask strategy parameter and the masking prediction parameter are respectively converted into effect feature vectors based on the masking vector feature extraction neural network, and the similarity is calculated by the vector distance to quantify the matching degree between different masking strategies and predicted masking conditions, thereby improving the adaptability and accuracy of the feature extraction model selection, assisting in improving the adaptability and accuracy of face recognition in different scenes, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition services.

[0159] As an optional embodiment, the determining module determines the specific manner of the at least one classification model according to the mask strategy parameters corresponding to the training data sets of the plurality of feature extraction models, including:

[0160] For each candidate classification model, the second similarity between the mask strategy parameters corresponding to the training data set of the candidate classification model and the mask strategy parameters corresponding to the training data set of each feature extraction model is calculated based on the masking vector feature extraction neural network and the vector distance algorithm;

[0161] calculate an average value of the second similarity corresponding to each feature extraction model of the candidate classification model, to obtain a second model priority corresponding to the candidate classification model;

[0162] screen all candidate classification models with the second model priority greater than a preset second priority threshold, to obtain at least one corresponding classification model.

[0163] It can be seen that, through the above optional embodiments, the similarity between the mask strategy parameters of the candidate classification model and the mask strategy parameters of the feature extraction model is calculated based on the masking vector feature extraction neural network and the vector distance algorithm, and the priority of the classification model is determined by the average similarity, so as to screen the classification model that matches the feature extraction model more, thereby improving the overall recognition accuracy, assisting in improving the adaptability and accuracy of face recognition in different scenarios, making the recognition result more robust and suitable for diversified environments, and improving the intelligent degree and automation degree of face recognition services.

[0164] As an optional embodiment, the candidate feature extraction model and the candidate classification model are trained by the following steps:

[0165] determine a plurality of training image datasets subjected to mask processing by using different mask strategy parameters;

[0166] train the preset prediction base model based on each training image dataset to obtain a plurality of preliminary prediction models trained, and optionally, the prediction base model and the preliminary prediction model include the feature extraction model and the classification model;

[0167] determine all the training image datasets as a common dataset;

[0168] based on the common dataset, commonly train all the preliminary prediction models, and perform model parameter optimization based on a common loss function related to the mask strategy parameter, to obtain the trained candidate feature extraction model and the candidate classification model; optionally, the common loss function is a weighted sum value of the loss function values of each preliminary prediction model; wherein the weighted weight corresponding to the loss function value of each preliminary prediction model is proportional to the masking degree of the mask strategy parameter corresponding to the training image dataset of the preliminary prediction model; wherein the masking degree is calculated by the following steps:

[0169] input the mask strategy parameter corresponding to the training image dataset of the preliminary prediction model into the masking vector feature extraction neural network, to obtain a third effect feature vector corresponding to the mask strategy parameter;

[0170] calculate a first vector distance between the third effect feature vector and a first reference feature vector corresponding to a standard unoccluded image;

[0171] calculate a second vector distance between the third effect feature vector and a second reference feature vector corresponding to a standard full occlusion image; optionally, the first reference feature vector and the second reference feature vector are both obtained by pre-analysis of vector extraction results of a preset data set based on a masking vector feature extraction neural network;

[0172] calculate a ratio of the first vector distance and the second vector distance to obtain a preliminary prediction model corresponding to the masking degree.

[0173] As can be seen, through the above optional embodiments, multiple training image data sets are generated based on different mask strategy parameters, and all preliminary prediction models are jointly trained through a common data set, and model parameters are optimized using a common loss function related to the mask strategy parameter, so as to train a feature extraction model and a classification model that are more suitable for masking conditions, thereby assisting in improving the adaptability and accuracy of face recognition in different scenarios, making the recognition result more robust and applicable to diversified environments, and improving the intelligent degree and automation degree of face recognition services.

[0174] As an optional embodiment, the specific manner in which the recognition module determines the classification recognition result corresponding to the face image data based on the multiple image features and the classification model includes:

[0175] randomly sample and combine the multiple image features based on a random combination algorithm to obtain multiple combined feature sets;

[0176] input each combined feature set into the classification model to obtain an output set recognition result;

[0177] calculate the intersection of all set recognition results to obtain the classification recognition result corresponding to the face image data.

[0178] As can be seen, through the above optional embodiments, the multiple image features are sampled and combined based on a random combination algorithm, and the recognition results of each combined feature set are calculated through the classification model, and then the intersection of all recognition results is taken, so as to reduce feature interference and misclassification, improve the adaptability and accuracy of face recognition in different scenarios, make the recognition result more robust and applicable to diversified environments, and improve the intelligent degree and automation degree of face recognition services.

[0179] Embodiment Three

[0180] Please refer to Figure 3 , Figure 3 is another face recognition system based on multi-model cooperation disclosed by the embodiments of the present application. Figure 3 The face recognition system based on multi-model cooperation described is applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 3As shown, the multi-model collaborative face recognition system can include:

[0181] a memory 301 storing executable program code;

[0182] a processor 302 coupled with the memory 301;

[0183] The processor 302 invokes the executable program code stored in the memory 301 to perform the steps of the multi-model collaborative face recognition method described in Embodiment One.

[0184] Embodiment Four

[0185] The embodiments of the present application disclose a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the steps of the multi-model collaborative face recognition method described in Embodiment One.

[0186] Embodiment Five

[0187] The embodiments of the present application disclose a computer program product comprising a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps of the multi-model collaborative face recognition method described in Embodiment One.

[0188] The above describes specific embodiments of the present application, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described in the embodiments, and still achieve the desired results. Also, the processes depicted in the figures do not necessarily have to be performed in the specific order described or in sequential order, but can be performed in other orders or concurrently. In certain implementations, multitasking and parallel processing can be advantageous.

[0189] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by a computer chip or entity, or by a product with a certain function. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0190] For the convenience of description, the above apparatuses are described in various units by functions respectively when described. Of course, the functions of each unit can be implemented in the same or more software and / or hardware when implementing the present specification.

[0191] Those skilled in the art will appreciate that embodiments of the present description can be readily used as a method, an apparatus (system) or a computer program product. Accordingly, embodiments of the present description can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer readable program code.

[0192] The present description is described in reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present description. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0193] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 Figure 1

[0195] In one typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0196] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), or electrically erasable programmable read only memory (EEPROM), for the storage of software that is read during runtime. The memory is an example of computer readable media.​​​​​​

[0197] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0198] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0199] The specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0200] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0201] It should be finally pointed out that: the face recognition method and system based on multi-model cooperation disclosed in the embodiments of the present application are only the preferred embodiments of the present application, and are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that; it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-model collaborative based face recognition method, characterized in that, The method comprises: obtaining face image data to be identified and corresponding scene features; determining corresponding multiple feature extraction models and classification models according to the scene features, comprising: inputting the scene features into a trained mask parameter prediction neural network to obtain mask prediction parameters corresponding to the scene features; the mask parameter prediction neural network is trained by a training data set comprising multiple training scene images and corresponding scene feature annotations and mask condition annotations; for each candidate feature extraction model, determining mask strategy parameters corresponding to a training data set of the candidate feature extraction model; calculating a first similarity between the mask strategy parameters and the mask prediction parameters to obtain a first model priority of the candidate feature extraction model; the mask strategy parameters and the mask prediction parameters each comprise at least one of a mask image position, a mask part size, a mask part area ratio, and a mask part three-dimensional information; the first similarity is calculated by the following steps: inputting the mask strategy parameters into a trained mask vector feature extraction neural network to obtain a first effect feature vector corresponding to the mask strategy parameters; inputting the mask prediction parameters into the mask vector feature extraction neural network to obtain a second effect feature vector corresponding to the mask prediction parameters; the mask vector feature extraction neural network is trained by a training data set comprising multiple training mask parameters and corresponding mask pre-image annotations and mask post-image annotations; calculating the inverse of the vector distance between the first effect feature vector and the second effect feature vector to obtain the first similarity; selecting all the candidate feature extraction models whose first model priority is greater than a preset first priority threshold to obtain corresponding multiple feature extraction models; for each candidate classification model, calculating a second similarity between mask strategy parameters corresponding to a training data set of the candidate classification model and mask strategy parameters corresponding to a training data set of each feature extraction model based on the mask vector feature extraction neural network and a vector distance algorithm; calculating an average value of the second similarity between the candidate classification model and each feature extraction model to obtain a second model priority of the candidate classification model; selecting all the candidate classification models whose second model priority is greater than a preset second priority threshold to obtain at least one corresponding classification model; the candidate feature extraction models and the candidate classification models are trained by multiple training data sets processed by different mask strategy parameters; inputting the face image data into the multiple feature extraction models to obtain multiple image features corresponding to the face image; determining a classification recognition result corresponding to the face image data according to the multiple image features and the classification model.

2. The multi-model based collaborative face recognition method of claim 1, wherein, The scene features comprise at least one of an identification location, an ambient light, a scene type, user information, and an identification purpose.

3. The multi-model based collaborative face recognition method of claim 1, wherein, The candidate feature extraction models and the candidate classification models are trained by the following steps: determine a plurality of training image data sets after mask processing with different mask strategy parameters; train a preset prediction base model based on each of the training image data sets to obtain a plurality of preliminary prediction models; the prediction base model and the preliminary prediction models include a feature extraction model and a classification model; determine all the training image data sets as a common data set; train all the preliminary prediction models based on the common data set, and perform model parameter optimization based on a common loss function related to the mask strategy parameters to obtain the trained candidate feature extraction model and the candidate classification model; the common loss function is a weighted sum of loss function values of each of the preliminary prediction models; wherein the weighted weight corresponding to the loss function value of each of the preliminary prediction models is proportional to the masking degree of the mask strategy parameter corresponding to the training image data set of the preliminary prediction model; wherein the masking degree is calculated by the following steps: input the mask strategy parameter corresponding to the training image data set of the preliminary prediction model into the masking vector feature extraction neural network to obtain a third effect feature vector corresponding to the mask strategy parameter; calculate a first vector distance between the third effect feature vector and a first reference feature vector corresponding to a standard unoccluded image; calculate a second vector distance between the third effect feature vector and a second reference feature vector corresponding to a standard fully occluded image; the first reference feature vector and the second reference feature vector are both obtained by pre-analysis based on vector extraction results of a preset data set by the masking vector feature extraction neural network; calculate the ratio of the first vector distance and the second vector distance to obtain the masking degree corresponding to the preliminary prediction model.

4. The multi-model based collaborative face recognition method of claim 1, wherein, determining a classification recognition result corresponding to the face image data according to the plurality of image features and the classification model, including: randomly sampling and combining the plurality of image features based on a random combination algorithm to obtain a plurality of combined feature sets; inputting each of the combined feature sets into the classification model to obtain an output set recognition result; calculating the intersection of all the set recognition results to obtain the classification recognition result corresponding to the face image data.

5. A multi-model collaborative based face recognition system characterized in that, The system comprises: an acquisition module configured to acquire face image data to be recognized and corresponding scene features; a determination module configured to determine a plurality of feature extraction models and classification models corresponding to the scene features, including: inputting the scene features into a trained masking parameter prediction neural network to obtain masking prediction parameters corresponding to the scene features; the masking parameter prediction neural network is trained by a training data set including a plurality of training scene images and corresponding scene feature labels and masking condition labels; for each candidate feature extraction model, determining mask strategy parameters corresponding to the training data set of the candidate feature extraction model; calculate a first similarity between the mask strategy parameter and the masking prediction parameter, to obtain a first model priority corresponding to the candidate feature extraction model; the mask strategy parameter and the masking prediction parameter both include at least one of a masking image position, a masking part size, a masking part area ratio, and a masking part three-dimensional information; the first similarity is calculated by the following steps: input the mask strategy parameter into the trained masking vector feature extraction neural network, to obtain a first effect feature vector corresponding to the mask strategy parameter; input the masking prediction parameter into the masking vector feature extraction neural network, to obtain a second effect feature vector corresponding to the masking prediction parameter; the masking vector feature extraction neural network is trained by a training data set including a plurality of training masking parameters and corresponding masking pre-image labels and masking post-image labels; calculate the reciprocal of the vector distance between the first effect feature vector and the second effect feature vector, to obtain the first similarity; screen all the candidate feature extraction models with a first priority threshold greater than a preset first priority threshold, to obtain a plurality of corresponding feature extraction models; for each candidate classification model, based on the masking vector feature extraction neural network and a vector distance algorithm, calculate a second similarity between the mask strategy parameter corresponding to the training data set of the candidate classification model and the mask strategy parameter corresponding to the training data set of each feature extraction model; calculate the average value of the second similarity of the candidate classification model and each feature extraction model, to obtain a second model priority corresponding to the candidate classification model; screen all the candidate classification models with a second priority threshold greater than a preset second priority threshold, to obtain at least one corresponding classification model; the candidate feature extraction model and the candidate classification model are trained by a plurality of training data sets processed by different mask strategy parameters; an extraction module configured to input the face image data into the plurality of feature extraction models, to obtain a plurality of image features corresponding to the face image; an identification module configured to determine a classification recognition result corresponding to the face image data according to the plurality of image features and the classification model.

6. A multi-model collaborative based face recognition system characterized in that, The system comprises: a memory storing executable program codes; a processor coupled with the memory; the processor invokes the executable program codes stored in the memory to execute the multi-model collaborative face recognition method according to any one of claims 1-4.

Images