Neural Network Training Method for Image Processing, Face Recognition Method and Device

Abstract

The present disclosure provides a neural network training method, a face recognition method and apparatus for image processing. Among them, the neural network training method for image processing includes: obtaining multiple groups of sample data groups; for a target sample data group among the multiple groups of sample data groups, respectively determining a first feature similarity and a second feature similarity between a first sample image and a second sample image based on a teacher neural network and a student neural network to be trained; and for the same sample image in the target sample data group, respectively determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained, so as to obtain a third feature similarity between the two feature extraction results; determining a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity and the third feature similarity, and training the student neural network based on the target loss value.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a neural network training method, a face recognition method and a device for image processing. Background Art

[0002] Knowledge distillation refers to a training strategy that uses a teacher neural network with better test performance to train the student neural network to be trained.

[0003] In the training process related to image processing, the teacher neural network and the student neural network extract features from the same image respectively, then calculate the feature similarity between the two feature extraction results, and adjust the network parameters of the student neural network with the goal of improving the feature similarity. However, if the teacher network itself has low feature extraction accuracy for some easily confused images, the network accuracy of the student neural network trained in the above way will also be relatively low. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a neural network training method, a face recognition method and a device for image processing.

[0005] In a first aspect, an embodiment of the present disclosure provides a neural network training method for image processing, comprising:

[0006] Acquire multiple groups of sample data groups; wherein the sample data groups include a first sample image of a first user and a second sample image of a second user, and the first user and the second user are similar users;

[0007] For a target sample data group among the multiple sample data groups, determining a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on a teacher neural network and a student neural network to be trained respectively; and for the same sample image in the target sample data group, determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results;

[0008] Based on the first feature similarity, the second feature similarity, and the third feature similarity, a target loss value for training the student neural network to be trained is determined, and the student neural network is trained based on the target loss value.

[0009] In this way, by introducing the first feature similarity and the second feature similarity into the loss value, additional supervision is introduced into the process of knowledge distillation, which can specifically improve the student neural network's ability to distinguish similar sample images, thereby solving the hidden dangers of traditional knowledge distillation methods in student neural network training and effectively improving the network accuracy of student neural networks.

[0010] In a possible implementation manner, the method further includes determining the sample data set according to the following method:

[0011] Perform feature extraction on the facial image of each sample user in the data set to determine the first facial features corresponding to each sample user;

[0012] Determine similar users corresponding to each sample user based on the first facial features corresponding to each sample user;

[0013] For a sample user among the sample users, a facial image of the sample user and facial images of similar users corresponding to the sample user are taken as a set of sample data groups.

[0014] In this way, by setting facial images of similar users in the sample data group, the ability of the student neural network to distinguish similar users can be improved in a targeted manner.

[0015] In a possible implementation manner, determining similar users corresponding to each sample user based on the first facial features corresponding to each sample user includes:

[0016] For a first target sample user, determining a fourth feature similarity between a first facial feature corresponding to the first target sample user and first facial features corresponding to other sample users in the data set except the first target sample user;

[0017] Based on the fourth feature similarity, similar users corresponding to the first target sample user are determined.

[0018] In this way, through the fourth feature similarity between the first facial feature corresponding to the first target sample user and the first facial feature corresponding to the remaining sample users in the data set except the first target sample user, similar users in the data set with similar features corresponding to the first target sample user can be accurately determined, so as to subsequently improve the student neural network's ability to distinguish similar users.

[0019] In a possible implementation, the data set includes multiple face images corresponding to each sample user;

[0020] The step of extracting features from the facial images of each sample user in the data set to determine first facial features corresponding to each sample user includes:

[0021] For a second target sample user, based on the teacher neural network, feature extraction is performed on a plurality of face images corresponding to the second target sample user to obtain a plurality of second facial features;

[0022] Based on the plurality of second facial features corresponding to the second target sample user, a first facial feature corresponding to the second target sample user is determined.

[0023] In this way, by respectively identifying a plurality of facial images corresponding to the sample user, the first facial feature corresponding to the sample user is obtained, so that the first facial feature can more comprehensively represent the features of the sample user.

[0024] In a possible implementation manner, the student neural network is used to determine the second feature similarity between the first sample image and the second sample image in the target sample data group according to the following method:

[0025] Based on the student neural network, feature extraction is performed on the first sample image in the target sample data group to obtain a third facial feature; and based on the teacher neural network, feature extraction is performed on the second sample image in the target sample data group to obtain a fourth facial feature;

[0026] Obtain the second feature similarity between the third facial feature and the fourth facial feature.

[0027] In this way, by using the teacher neural network to replace the student neural network to extract features from the second sample image in the target sample data group, and using the obtained fourth facial features to calculate the second feature similarity, the training speed of the neural network can be improved.

[0028] In a possible implementation manner, determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively to obtain a third feature similarity between the two feature extraction results includes:

[0029] Based on the teacher neural network and the student neural network to be trained, respectively, feature extraction is performed on the target sample image in the target sample data group to obtain a fifth facial feature and a sixth facial feature, wherein the target sample image is the first sample image or the second sample image;

[0030] Obtain a third feature similarity between the fifth facial feature and the sixth facial feature.

[0031] In this way, based on the teacher neural network and the student neural network to be trained, feature extraction is performed on the target sample image in the target sample data group to obtain the fifth facial feature and the sixth facial feature, which can make the final third feature similarity more accurate. Further, the student neural network can be distilled and trained through the teacher neural network to improve the accuracy of the student neural network.

[0032] In a possible implementation, determining a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity includes:

[0033] Based on the first feature similarity and the second feature similarity, determining a first loss value in this training process, and using the third feature similarity as a second loss value;

[0034] The target loss value is determined based on the first loss value and the second loss value.

[0035] In this way, determining the target loss value based on the first loss value and the second loss value can make the supervision data in the training process richer, thereby improving the training effect on the student neural network.

[0036] In a possible implementation manner, determining the first loss value in the current training process based on the first feature similarity and the second feature similarity includes:

[0037] Determine the difference between the second feature similarity and the first feature similarity corresponding to the same sample data group;

[0038] The average of the feature similarity differences of the sample data groups that meet the preset conditions in the feature similarity differences is used as the first loss value.

[0039] In this way, by introducing the difference between the second feature similarity and the first feature similarity into the loss value, additional supervision can be introduced into the process of knowledge distillation, and the student neural network's ability to distinguish sample images can be specifically improved, thereby improving the network accuracy of the student neural network.

[0040] In a possible implementation manner, the sample images in the sample data group carry user tags for identifying users in the sample images;

[0041] The method further comprises:

[0042] Extracting facial features of sample images in the sample data group based on the student neural network;

[0043] Performing face recognition based on the face features and determining a face recognition result;

[0044] Determining a third loss value based on the face recognition result and the user tag;

[0045] The determining the target loss value based on the first loss value and the second loss value includes:

[0046] The target loss value is determined based on the first loss value, the second loss value, and the third loss value.

[0047] In this way, by introducing face recognition results and user labels into the loss value, additional supervision can be introduced into the process of knowledge distillation, thereby improving the network accuracy of the student neural network.

[0048] In a second aspect, the present disclosure also provides a face recognition method, including:

[0049] Obtaining a face image to be recognized;

[0050] The face image to be identified is identified using a student neural network trained based on any one of the neural network training methods for image processing described in the first aspect to determine a face recognition result corresponding to the face image to be identified.

[0051] In a third aspect, the present disclosure also provides a neural network training device for image processing, including:

[0052] A first acquisition module is used to acquire multiple groups of sample data groups; wherein the sample data groups include a first sample image of a first user and a second sample image of a second user, and the first user and the second user are similar users;

[0053] A first determination module is used to determine, for a target sample data group among the multiple sample data groups, a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on a teacher neural network and a student neural network to be trained respectively; and, for the same sample image in the target sample data group, determine two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results;

[0054] The second determination module is used to determine a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity and the third feature similarity, and train the student neural network based on the target loss value.

[0055] In a possible implementation manner, the first acquisition module is further configured to determine the sample data set according to the following method:

[0056] Perform feature extraction on the facial image of each sample user in the data set to determine the first facial features corresponding to each sample user;

[0057] Determine similar users corresponding to each sample user based on the first facial features corresponding to each sample user;

[0058] For a sample user among the sample users, a facial image of the sample user and facial images of similar users corresponding to the sample user are taken as a set of sample data groups.

[0059] In a possible implementation manner, when determining similar users corresponding to each sample user based on the first facial features corresponding to each sample user, the first acquisition module is configured to:

[0060] For a first target sample user, determining a fourth feature similarity between a first facial feature corresponding to the first target sample user and first facial features corresponding to other sample users in the data set except the first target sample user;

[0061] Based on the fourth feature similarity, similar users corresponding to the first target sample user are determined.

[0062] In a possible implementation, the data set includes multiple face images corresponding to each sample user;

[0063] The first acquisition module, when extracting features from the facial images of each sample user in the data set to determine the first facial features corresponding to each sample user, is used to:

[0064] For a second target sample user, based on the teacher neural network, feature extraction is performed on a plurality of face images corresponding to the second target sample user to obtain a plurality of second facial features;

[0065] Based on the plurality of second facial features corresponding to the second target sample user, a first facial feature corresponding to the second target sample user is determined.

[0066] In a possible implementation manner, the first determination module, when determining the second feature similarity between the first sample image and the second sample image in the target sample data group, is configured to:

[0067] Based on the student neural network, feature extraction is performed on the first sample image in the target sample data group to obtain a third facial feature; and based on the teacher neural network, feature extraction is performed on the second sample image in the target sample data group to obtain a fourth facial feature;

[0068] Obtain the second feature similarity between the third facial feature and the fourth facial feature.

[0069] In a possible implementation manner, the first determination module, when determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively to obtain a third feature similarity between the two feature extraction results, is used to:

[0070] Based on the teacher neural network and the student neural network to be trained, respectively, feature extraction is performed on the target sample image in the target sample data group to obtain a fifth facial feature and a sixth facial feature, wherein the target sample image is the first sample image or the second sample image;

[0071] Obtain a third feature similarity between the fifth facial feature and the sixth facial feature.

[0072] In a possible implementation manner, the second determination module, when determining a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity, is used to:

[0073] Based on the first feature similarity and the second feature similarity, determining a first loss value in this training process, and using the third feature similarity as a second loss value;

[0074] The target loss value is determined based on the first loss value and the second loss value.

[0075] In a possible implementation manner, when determining the first loss value in the current training process based on the first feature similarity and the second feature similarity, the second determination module is used to:

[0076] Determine the difference between the second feature similarity and the first feature similarity corresponding to the same sample data group;

[0077] The average of the feature similarity differences of the sample data groups that meet the preset conditions in the feature similarity differences is used as the first loss value.

[0078] In a possible implementation manner, the sample images in the sample data group carry user tags for identifying users in the sample images;

[0079] The first determining module is further used for:

[0080] Extracting facial features of sample images in the sample data group based on the student neural network;

[0081] Performing face recognition based on the face features and determining a face recognition result;

[0082] Determining a third loss value based on the face recognition result and the user tag;

[0083] The second determination module, when determining the target loss value based on the first loss value and the second loss value, is used to:

[0084] The target loss value is determined based on the first loss value, the second loss value, and the third loss value.

[0085] In a fourth aspect, the present disclosure also provides a face recognition device, including:

[0086] The second acquisition module is used to acquire the face image to be recognized;

[0087] The third determination module is used to identify the face image to be identified by using the student neural network trained based on any neural network training method for image processing described in the first aspect, and determine the face recognition result corresponding to the face image to be identified.

[0088] In a fifth aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps in any possible implementation of the first aspect or the second aspect are performed.

[0089] In a sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in any possible implementation of the first aspect or the second aspect are executed.

[0090] For a description of the effects of the above-mentioned face recognition method, face recognition device, neural network training device, computer equipment and storage medium, please refer to the description of the above-mentioned neural network training method, which will not be repeated here.

[0091] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0093] Figure 1 A flowchart of a neural network training method for image processing provided by an embodiment of the present disclosure is shown;

[0094] Figure 2 A flowchart showing a specific method for determining a sample data group in a neural network training method for image processing provided by an embodiment of the present disclosure;

[0095] Figure 3 A flowchart showing a specific method for determining a first facial feature in a neural network training method for image processing provided by an embodiment of the present disclosure;

[0096] Figure 4 A flowchart showing a specific method for determining similar users corresponding to each sample user in the neural network training method for image processing provided by an embodiment of the present disclosure;

[0097] Figure 5 A flowchart showing a specific method for determining the second feature similarity in the neural network training method for image processing provided by an embodiment of the present disclosure;

[0098] Figure 6 A flowchart showing a specific method for determining the third feature similarity in the neural network training method for image processing provided by an embodiment of the present disclosure;

[0099] Figure 7 A flowchart showing a specific method for determining a target loss value in a neural network training method for image processing provided by an embodiment of the present disclosure;

[0100] Figure 8 A flowchart showing a specific method for determining a first loss value in a neural network training method for image processing provided by an embodiment of the present disclosure;

[0101] Fig. 9 A flowchart showing a specific method for determining a third loss value in a neural network training method for image processing provided by an embodiment of the present disclosure;

[0102] Fig.10A flow chart of a face recognition method provided by an embodiment of the present disclosure is shown;

[0103] Fig.11 A schematic diagram of the architecture of a neural network training device for image processing provided by an embodiment of the present disclosure is shown;

[0104] Fig.12 A schematic diagram of the architecture of a face recognition device provided by an embodiment of the present disclosure is shown;

[0105] Fig.13 A schematic diagram of the structure of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0106] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0107] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0108] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0109] Research has found that in the training process related to image processing, the teacher neural network and the student neural network respectively extract features for the same picture, then calculate the feature similarity between the two feature extraction results, and adjust the network parameters of the student neural network with the goal of improving the feature similarity. However, if the teacher network itself has low feature extraction accuracy for some confusing pictures, the adjustment effect of using the feature similarity between the two feature extraction results is poor, and the network accuracy of the student neural network after training is also relatively low.

[0110] Based on the above research, the present disclosure provides a neural network training scheme for image processing, for a target sample data group among multiple sample data groups, based on the teacher neural network and the student neural network to be trained, respectively, the first feature similarity and the second feature similarity between the first sample image and the second sample image in the target sample data group are determined; and the third feature similarity of the feature extraction results of the teacher neural network and the student neural network to be trained for the same sample image in the target sample data group is determined; then based on the first feature similarity, the second feature similarity and the third feature similarity, the target loss value for training the student neural network to be trained is determined. In this way, by introducing the first feature similarity and the second feature similarity into the loss value, additional supervision is introduced into the process of knowledge distillation, which can improve the student neural network's ability to distinguish similar sample images in a targeted manner, thereby solving the hidden dangers brought to the student neural network training by the traditional knowledge distillation method and effectively improving the network accuracy of the student neural network.

[0111] To facilitate understanding of this embodiment, a neural network training method for image processing disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the neural network training method for image processing provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example, a server or other processing device. In some possible implementations, the neural network training method for image processing can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0112] See also Figure 1 FIG. 1 is a flowchart of a neural network training method for image processing provided by an embodiment of the present disclosure, wherein the method comprises steps S101 to S103, wherein:

[0113] S101: Acquire multiple groups of sample data groups; wherein the sample data groups include a first sample image of a first user and a second sample image of a second user, and the first user and the second user are similar users.

[0114] S102: For a target sample data group among the multiple sample data groups, determine a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on the teacher neural network and the student neural network to be trained respectively; and, for the same sample image in the target sample data group, determine two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results.

[0115] S103: Determine a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity, and train the student neural network based on the target loss value.

[0116] The following is a detailed description of the above steps.

[0117] With respect to S101, the multiple groups of sample data sets may be obtained from a preset difficult sample data set, and the difficult sample data set may include difficult positive sample pairs and difficult negative sample pairs (i.e., the sample data sets). The difficult positive sample pairs include facial images of two sample users whose feature similarity is lower than a first threshold, and the difficult negative sample pairs include facial images of two sample users whose feature similarity is higher than a second threshold. By using the sample data in the difficult sample data set for training, the recognition ability of the student neural network for difficult samples can be specifically improved. The second threshold is greater than or equal to the first threshold.

[0118] Specifically, the first user and the second user in the difficult negative sample pair are similar users to each other; for any of the first users, the similar users of the first user are users whose feature similarity (such as similarity of facial features) with the first user is higher than a second threshold.

[0119] In a possible implementation, Figure 2 As shown, the sample data set can be determined by the following steps:

[0120] S201: extracting features from the facial images of each sample user in the data set to determine first facial features corresponding to each sample user.

[0121] Here, the data set may include multiple facial images corresponding to each sample user, such as different facial images of the sample user under different shooting conditions such as shooting angles, shooting backgrounds, and makeup.

[0122] In a possible implementation, Figure 3As shown, the first facial features corresponding to each sample user can be determined by the following steps:

[0123] S2011: For a second target sample user, based on the teacher neural network, feature extraction is performed on a plurality of face images corresponding to the second target sample user to obtain a plurality of second facial features.

[0124] Here, the second target sample user may be a sample user in the data set corresponding to a plurality of face images; the second facial feature may be a facial feature vector used to represent the facial feature of the second target sample user.

[0125] S2012: Determine a first facial feature corresponding to the second target sample user based on a plurality of second facial features corresponding to the second target sample user.

[0126] Specifically, the obtained multiple facial feature vectors may be averaged, and the obtained average value may be used as the first facial feature corresponding to the second target sample user.

[0127] Exemplarily, taking the second facial feature as a 256-dimensional vector and the number of facial images corresponding to the second target sample user as an example, the first facial feature corresponding to the second target sample user is a 256-dimensional vector, each dimension of which is the average value of the facial feature vectors corresponding to the above three facial images in the corresponding dimension.

[0128] S202: Determine similar users corresponding to each sample user based on the first facial features corresponding to each sample user.

[0129] In a possible implementation, Figure 4 As shown, similar users corresponding to each sample user can be determined by the following steps:

[0130] S2021: For a first target sample user, determine a fourth feature similarity between a first facial feature corresponding to the first target sample user and first facial features corresponding to remaining sample users in the data set except the first target sample user.

[0131] Here, the fourth feature similarity can be cosine similarity, which can be obtained according to the cosine distance between the feature vectors, and the calculation formula is: 1-cosine distance = cosine similarity. For example, after calculating that the cosine distance between feature vector A and feature vector B is 0.6, it can be determined according to 1-0.6=0.4 that the cosine similarity between feature vector A and feature vector B is 0.4.

[0132] S2022: Determine similar users corresponding to the first target sample user based on the fourth feature similarity.

[0133] Specifically, when determining similar users corresponding to the first target sample user, for any first target sample user, the remaining sample users in the data set can be sorted from high to low according to the fourth feature similarity corresponding to the first target sample user, and the first N sample users in the sorting result whose corresponding fourth feature similarities exceed a preset similarity threshold are taken as similar users corresponding to the first target sample user, where N is a preset positive integer.

[0134] For example, the data set includes sample user 1, sample user 2, sample user 3, sample user 4 and sample user 5, N is 1, and the preset similarity threshold is 0.6. Among them, the facial features corresponding to sample user 2, sample user 3, sample user 4 and sample user 5 have fourth feature similarities with sample user 1 of 0.8, 0.6, 0.7 and 0.9 respectively. Then, according to N=1, it can be determined that sample user 5 whose corresponding fourth similarity is 0.9 (the fourth feature similarity 0.9 has the largest value in the sorting result and exceeds the similarity threshold) is the similar user corresponding to sample user 1.

[0135] In this way, through the fourth feature similarity between the first facial feature corresponding to the first target sample user and the first facial feature corresponding to the remaining sample users in the data set except the first target sample user, similar users in the data set with similar features corresponding to the first target sample user can be accurately determined, so as to subsequently improve the student neural network's ability to distinguish similar users.

[0136] S203: taking the facial image of each sample user and the facial image of a similar user corresponding to each sample user as a set of sample data groups.

[0137] Here, a set of the sample data groups may include a facial image of a sample user and a facial image of a similar user corresponding to the sample user.

[0138] Specifically, the facial image of the sample user and a facial image of a similar user corresponding to the sample user can be randomly extracted from the data set; or, can be filtered from the data set according to preset filtering conditions, and the preset filtering conditions include one or more of an image quality score used to characterize the clarity of the image, the size of the storage space occupied by the image, and the image size information.

[0139] S102: For a target sample data group among the multiple sample data groups, determine a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on the teacher neural network and the student neural network to be trained respectively; and, for the same sample image in the target sample data group, determine two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results.

[0140] Here, the target sample data group may be each sample data group in the multiple groups of sample data, the first feature similarity, the second feature similarity, and the third feature similarity may all be the cosine similarity, and the method for determining the cosine similarity is described in the above related content and will not be repeated here.

[0141] When determining the first feature similarity, the first sample image and the second sample image in the target sample data group can be respectively input into the teacher neural network, and after obtaining the feature vectors corresponding to the first sample image and the second sample image output by the teacher neural network, the cosine similarity between the feature vectors is calculated to obtain the first feature similarity; when determining the second feature similarity, the first sample image and the second sample image in the target sample data group can be respectively input into the student neural network, and after obtaining the feature vectors corresponding to the first sample image and the second sample image output by the student neural network, the cosine similarity between the feature vectors is calculated to obtain the second feature similarity.

[0142] In practical applications, due to factors such as hardware limitations, the student neural network may not be able to quickly extract features from the first sample image and the second sample image, thereby affecting the training speed of the student neural network.

[0143] In a possible implementation, Figure 5 As shown, the student neural network can determine the second feature similarity according to the following steps:

[0144] S501: Based on the student neural network, extract features from the first sample image in the target sample data group to obtain a third facial feature; and based on the teacher neural network, extract features from the second sample image in the target sample data group to obtain a fourth facial feature.

[0145] S502: Obtain the second feature similarity between the third facial feature and the fourth facial feature.

[0146] Here, the third facial feature and the fourth facial feature can be the facial feature vectors. The relevant description of obtaining the second feature similarity can refer to the relevant content of determining the fourth similarity above (for example, the cosine similarity between the third facial feature and the fourth facial feature can be calculated), which will not be repeated here.

[0147] In this way, by using the teacher neural network to replace the student neural network to extract features from the second sample image in the target sample data group, and using the obtained fourth facial features to calculate the second feature similarity, the training efficiency of the student neural network can be improved.

[0148] In a possible implementation, Figure 6 As shown, the third feature similarity can be determined by the following steps:

[0149] S601: Based on the teacher neural network and the student neural network to be trained, respectively, feature extraction is performed on the target sample image in the target sample data group to obtain a fifth facial feature and a sixth facial feature, wherein the target sample image is the first sample image or the second sample image.

[0150] S602: Obtain a third feature similarity between the fifth facial feature and the sixth facial feature.

[0151] Here, the fifth facial feature and the sixth facial feature can be the facial feature vectors. The relevant description of obtaining the third feature similarity can refer to the relevant content of determining the fourth similarity above (for example, the cosine similarity between the fifth facial feature and the sixth facial feature can be calculated), which will not be repeated here.

[0152] In this way, based on the teacher neural network and the student neural network to be trained, feature extraction is performed on the target sample image in the target sample data group to obtain the fifth facial feature and the sixth facial feature, which can make the final third feature similarity more accurate. Further, the student neural network can be distilled and trained through the teacher neural network to improve the accuracy of the student neural network.

[0153] S103: Determine a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity, and train the student neural network based on the target loss value.

[0154] Here, after determining the target loss value, the network parameters of the student neural network can be adjusted based on the target loss value, so as to train the student neural network.

[0155] In a possible implementation, Figure 7 As shown, the target loss value can be determined according to the following steps:

[0156] S701: Based on the first feature similarity and the second feature similarity, determine a first loss value in this training process, and use the third feature similarity as a second loss value.

[0157] In a possible implementation, Figure 8 As shown, the first loss value can be determined by the following steps:

[0158] S7011: Determine the difference between the second feature similarity and the first feature similarity corresponding to the same sample data group.

[0159] S7012: taking the average of the feature similarity differences of the sample data groups that meet the preset conditions in the feature similarity differences as the first loss value.

[0160] Here, when determining the sample data groups whose feature similarities meet the preset conditions, the sample data groups can be sorted from high to low according to the difference in feature similarities, and the first M sample data groups in the sorting result are used as the sample data groups that meet the preset conditions, where M is a preset positive integer.

[0161] S702: Determine the target loss value based on the first loss value and the second loss value.

[0162] Here, a first weight corresponding to the preset first loss value and a second weight corresponding to the second loss value may be obtained, and the first loss value and the second loss value may be weightedly summed according to the first weight and the second weight to determine the target loss value.

[0163] Exemplarily, the first loss value is L 1 The second loss value is L 2 , the first weight is α, the second weight is β, for example, according to the objective loss function L = αL 1 +βL 2 , calculate the target loss value.

[0164] In this way, determining the target loss value based on the first loss value and the second loss value can make the supervision data in the training process richer, thereby improving the training effect on the student neural network.

[0165] In practical applications, the sample images in the sample data group may also carry user tags for identifying the identities of users in the sample images, and a corresponding third loss value may be set for the user identities.

[0166] In a possible implementation, Fig. 9 As shown, the third loss value can be determined by the following steps:

[0167] S901: Extracting facial features of sample images in the sample data group based on the student neural network.

[0168] Here, the sample image in the sample data group may be the first sample image and / or the second sample image.

[0169] S902: Perform face recognition based on the face features and determine a face recognition result.

[0170] Here, the face recognition result may be an identity recognition result of a sample user in the sample image.

[0171] S903: Determine a third loss value based on the face recognition result and the user tag.

[0172] Here, when determining the third loss value, the cross entropy loss value can be calculated based on the face recognition result and the user tag to obtain the third loss value.

[0173] Furthermore, the target loss value may be determined based on the first loss value, the second loss value, and the third loss value.

[0174] Specifically, the third weight corresponding to the preset third loss value can be obtained, and the first loss value, the second loss value, and the third loss value are weighted and summed according to the first weight, the second weight, and the third weight to determine the target loss value.

[0175] Exemplarily, the first loss value is L 1 The second loss value is L 2 The third loss value is L 3 , the first weight is α, the second weight is β, and the third weight is χ as an example, according to the objective loss function L=αL 1 +βL 2 +xL 3 , calculate the target loss value.

[0176] It should be noted that the first weight, the second weight, and the third weight are respectively used to characterize the importance of the first loss value, the second loss value, and the third loss value to the target loss value. In order to facilitate subsequent weight adjustment, the sum of the values ​​of the first weight, the second weight, and the third weight can be set to a preset value, such as 1.

[0177] In this way, by introducing face recognition results and user labels into the loss value, additional supervision can be introduced into the process of knowledge distillation, thereby improving the network accuracy of the student neural network.

[0178] Based on the same concept, the present disclosure also provides a method for face recognition, see Fig.10 FIG. 1 is a flowchart of a face recognition method provided by an embodiment of the present disclosure, wherein the face recognition method includes S1001 to S1002, wherein:

[0179] S1001: Obtain a face image to be recognized.

[0180] S1002: Using a student neural network trained by any of the neural network training methods for image processing provided by the embodiments of the present disclosure to recognize the face image to be recognized, and determining a face recognition result corresponding to the face image to be recognized.

[0181] Here, the face image to be recognized may include multiple images, such as multiple images of the same user, or images corresponding to different users.

[0182] Since the student neural network trained by any of the neural network training methods for image processing provided by the embodiments of the present disclosure has a strong ability to distinguish similar sample images and a high network accuracy, the student neural network trained by any of the neural network training methods for image processing provided by the embodiments of the present disclosure has a more accurate face recognition result when identifying a face image to be identified.

[0183] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0184] Based on the same inventive concept, the embodiments of the present disclosure also provide a neural network training device for image processing corresponding to the neural network training method for image processing. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the above-mentioned neural network training method for image processing in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0185] Reference Fig.11 , which is a schematic diagram of the architecture of a neural network training device for image processing provided by an embodiment of the present disclosure, the device comprises: a first acquisition module 1101, a first determination module 1102, and a second determination module 1103; wherein,

[0186] A first acquisition module 1101 is used to acquire multiple groups of sample data groups; wherein the sample data groups include a first sample image of a first user and a second sample image of a second user, and the first user and the second user are similar users;

[0187] A first determination module 1102 is used to determine, for a target sample data group among the multiple sample data groups, a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on the teacher neural network and the student neural network to be trained respectively; and, for the same sample image in the target sample data group, determine two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results;

[0188] The second determination module 1103 is used to determine a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity and the third feature similarity, and train the student neural network based on the target loss value.

[0189] In a possible implementation manner, the first acquisition module 1101 is further configured to determine the sample data group according to the following method:

[0190] Perform feature extraction on the facial image of each sample user in the data set to determine the first facial features corresponding to each sample user;

[0191] Determine similar users corresponding to each sample user based on the first facial features corresponding to each sample user;

[0192] For a sample user among the sample users, a facial image of the sample user and facial images of similar users corresponding to the sample user are taken as a set of sample data groups.

[0193] In a possible implementation manner, when determining similar users corresponding to each sample user based on the first facial features corresponding to each sample user, the first acquisition module 1101 is configured to:

[0194] For a first target sample user, determining a fourth feature similarity between a first facial feature corresponding to the first target sample user and first facial features corresponding to other sample users in the data set except the first target sample user;

[0195] Based on the fourth feature similarity, similar users corresponding to the first target sample user are determined.

[0196] In a possible implementation, the data set includes multiple face images corresponding to each sample user;

[0197] The first acquisition module 1101, when extracting features from the facial images of each sample user in the data set to determine the first facial features corresponding to each sample user, is used to:

[0198] For a second target sample user, based on the teacher neural network, feature extraction is performed on a plurality of face images corresponding to the second target sample user to obtain a plurality of second facial features;

[0199] Based on the plurality of second facial features corresponding to the second target sample user, a first facial feature corresponding to the second target sample user is determined.

[0200] In a possible implementation manner, the first determining module 1102, when determining the second feature similarity between the first sample image and the second sample image in the target sample data group, is configured to:

[0201] Based on the student neural network, feature extraction is performed on the first sample image in the target sample data group to obtain a third facial feature; and based on the teacher neural network, feature extraction is performed on the second sample image in the target sample data group to obtain a fourth facial feature;

[0202] Obtain the second feature similarity between the third facial feature and the fourth facial feature.

[0203] In a possible implementation manner, the first determination module 1102, when determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively to obtain a third feature similarity between the two feature extraction results, is used to:

[0204] Based on the teacher neural network and the student neural network to be trained, respectively, feature extraction is performed on the target sample image in the target sample data group to obtain a fifth facial feature and a sixth facial feature, wherein the target sample image is the first sample image or the second sample image;

[0205] Obtain a third feature similarity between the fifth facial feature and the sixth facial feature.

[0206] In a possible implementation manner, the second determination module 1103, when determining a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity, is used to:

[0207] Based on the first feature similarity and the second feature similarity, determining a first loss value in this training process, and using the third feature similarity as a second loss value;

[0208] The target loss value is determined based on the first loss value and the second loss value.

[0209] In a possible implementation manner, when determining the first loss value in the current training process based on the first feature similarity and the second feature similarity, the second determination module 1103 is used to:

[0210] Determine the difference between the second feature similarity and the first feature similarity corresponding to the same sample data group;

[0211] The average of the feature similarity differences of the sample data groups that meet the preset conditions in the feature similarity differences is used as the first loss value.

[0212] In a possible implementation manner, the sample images in the sample data group carry user tags for identifying users in the sample images;

[0213] The first determining module 1102 is further configured to:

[0214] Extracting facial features of sample images in the sample data group based on the student neural network;

[0215] Performing face recognition based on the face features and determining a face recognition result;

[0216] Determining a third loss value based on the face recognition result and the user tag;

[0217] The second determination module 1103, when determining the target loss value based on the first loss value and the second loss value, is used to:

[0218] The target loss value is determined based on the first loss value, the second loss value, and the third loss value.

[0219] Reference Fig.12 , which is a schematic diagram of the architecture of a face recognition device provided by an embodiment of the present disclosure, the device comprises: a second acquisition module 1201 and a third determination module 1202; wherein,

[0220] The second acquisition module 1201 is used to acquire a face image to be recognized;

[0221] The third determination module 1202 is used to identify the face image to be identified by using the student neural network trained by any of the neural network training methods for image processing provided by the embodiments of the present disclosure, and determine the face recognition result corresponding to the face image to be identified.

[0222] The neural network training device provided by the disclosed embodiment determines, for a target sample data group among multiple sample data groups, the first feature similarity and the second feature similarity between the first sample image and the second sample image in the target sample data group based on the teacher neural network and the student neural network to be trained respectively; and determines the third feature similarity of the feature extraction results of the teacher neural network and the student neural network to be trained for the same sample image in the target sample data group; and then determines the target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity and the third feature similarity. In this way, by introducing the first feature similarity and the second feature similarity into the loss value, additional supervision is introduced into the process of knowledge distillation, which can improve the student neural network's ability to distinguish similar sample images in a targeted manner, thereby solving the hidden dangers brought by the traditional knowledge distillation method to the student neural network training and effectively improving the network accuracy of the student neural network.

[0223] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0224] Based on the same technical concept, the embodiment of the present disclosure also provides a computer device. Fig.13 As shown, it is a schematic diagram of the structure of a computer device 1300 provided in an embodiment of the present disclosure, including a processor 1301, a memory 1302, and a bus 1303. Among them, the memory 1302 is used to store execution instructions, including a memory 13021 and an external memory 13022; the memory 13021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 1301, and the data exchanged with the external memory 13022 such as a hard disk. The processor 1301 exchanges data with the external memory 13022 through the memory 13021. When the computer device 1300 is running, the processor 1301 communicates with the memory 1302 through the bus 1303, so that the processor 1301 executes the following instructions:

[0225] Acquire multiple groups of sample data groups; wherein the sample data groups include a first sample image of a first user and a second sample image of a second user, and the first user and the second user are similar users;

[0226] For a target sample data group among the multiple sample data groups, determining a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on a teacher neural network and a student neural network to be trained respectively; and for the same sample image in the target sample data group, determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained respectively, so as to obtain a third feature similarity between the two feature extraction results;

[0227] Determine a target loss value for training the student neural network to be trained based on the first feature similarity, the second feature similarity, and the third feature similarity, and train the student neural network based on the target loss value; or

[0228] The processor 1301 executes the following instructions:

[0229] Obtaining a face image to be recognized;

[0230] The face image to be identified is identified using a student neural network trained by any of the neural network training methods for image processing provided based on the embodiments of the present disclosure, and a face recognition result corresponding to the face image to be identified is determined.

[0231] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the neural network training method for image processing described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0232] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the neural network training method for image processing described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0233] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0234] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0235] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0236] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0237] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0238] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A neural network training method for image processing, characterized in that, it includes: Obtain multiple groups of sample data sets; wherein, the sample data set contains the first sample image of the first user and the second sample image of the second user, and the first user and the second user are similar users; For the target sample data set in the multiple groups of sample data sets, respectively based on the teacher neural network and the student neural network to be trained, determine the first feature similarity and the second feature similarity between the first sample image and the second sample image in the target sample data set; and, for the same sample image in the target sample data set, respectively based on the teacher neural network and the student neural network to be trained, determine two feature extraction results of the same sample image to obtain the third feature similarity between the two feature extraction results; Based on the first feature similarity and the second feature similarity, determine the first loss value in the current training process, and use the third feature similarity as the second loss value; Based on the first loss value and the second loss value, determine the target loss value, and train the student neural network based on the target loss value.

2. The method according to claim 1, characterized in that, the method further includes determining the sample data set according to the following method: Extract features from the face images of each sample user in the data set to determine the first facial features corresponding to each sample user; Based on the first facial features corresponding to each sample user, determine the similar users corresponding to each sample user; For a sample user among the sample users, use the face image of the sample user and the face image of the similar user corresponding to the sample user as a group of sample data sets.

3. The method according to claim 2, characterized in that, the determining the similar users corresponding to each sample user based on the first facial features corresponding to each sample user includes: For the first target sample user, determine the fourth feature similarity between the first facial features corresponding to the first target sample user and the first facial features corresponding to the remaining sample users in the data set other than the first target sample user; Based on the fourth feature similarity, determine the similar users corresponding to the first target sample user.

4. The method according to claim 2 or 3, characterized in that, the data set includes multiple face images corresponding to each sample user; the extracting features from the face images of each sample user in the data set to determine the first facial features corresponding to each sample user includes: For the second target sample user, based on the teacher neural network, extract features from multiple face images corresponding to the second target sample user to obtain multiple second facial features; Based on the multiple second facial features corresponding to the second target sample user, determine the first facial features corresponding to the second target sample user.

5. The method according to claim 1, characterized in that, the student neural network is used to determine the second feature similarity between the first sample image and the second sample image in the target sample data set according to the following method: Extract the third face feature from the first sample image in the target sample data group based on the student neural network; and extract the fourth face feature from the second sample image in the target sample data group based on the teacher neural network; Obtain the second feature similarity between the third face feature and the fourth face feature.

6. The method according to claim 1, wherein, The step of respectively determining two feature extraction results of the same sample image based on the teacher neural network and the student neural network to obtain the third feature similarity between the two feature extraction results includes: Respectively based on the teacher neural network and the student neural network to be trained, extract the fifth face feature and the sixth face feature from the target sample image in the target sample data group, wherein the target sample image is the first sample image or the second sample image; Obtain the third feature similarity between the fifth face feature and the sixth face feature.

7. The method according to claim 1, wherein, The step of determining the first loss value in the current training process based on the first feature similarity and the second feature similarity includes: Determine the difference between the feature similarity of the second feature similarity corresponding to the same sample data group and the first feature similarity; Take the mean of the feature similarity differences of each sample data group that meets the preset conditions in the feature similarity differences as the first loss value.

8. The method according to claim 1 or 7, wherein, The sample image in the sample data group carries a user label for identifying the user in the sample image; The method further includes: Extract the face feature of the sample image in the sample data group based on the student neural network; Perform face recognition based on the face feature to determine the face recognition result; Determine the third loss value based on the face recognition result and the user label; The step of determining the target loss value based on the first loss value and the second loss value includes: Determine the target loss value based on the first loss value, the second loss value and the third loss value.

9. A face recognition method, wherein, includes: Obtain the face image to be recognized; Use the student neural network trained by the neural network training method for image processing according to any one of claims 1 to 8 to recognize the face image to be recognized, and determine the face recognition result corresponding to the face image to be recognized.

10. A neural network training device for image processing, wherein, includes: A first acquisition module for acquiring multiple groups of sample data groups; wherein, the sample data group contains the first sample image of the first user and the second sample image of the second user, and the first user and the second user are similar users; A first determination module, configured to, for a target sample data group among the multiple groups of sample data groups, respectively determine a first feature similarity and a second feature similarity between a first sample image and a second sample image in the target sample data group based on a teacher neural network and a student neural network to be trained; and, for the same sample image in the target sample data group, respectively determine two feature extraction results of the same sample image based on the teacher neural network and the student neural network to be trained, so as to obtain a third feature similarity between the two feature extraction results. A second determination module, configured to determine a first loss value in the current training process based on the first feature similarity and the second feature similarity, and use the third feature similarity as a second loss value; determine a target loss value based on the first loss value and the second loss value, and train the student neural network based on the target loss value.

11. A neural network training device for image processing Characterized in that It includes: A second acquisition module, configured to acquire a face image to be recognized; A third determination module, configured to use the student neural network trained by the neural network training method for image processing according to any one of claims 1 to 8 to recognize the face image to be recognized, and determine a face recognition result corresponding to the face image to be recognized.

12. A computer device Characterized in that It includes: A processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the neural network training method for image processing according to any one of claims 1 to 8 are executed; Or, the steps of the face recognition method according to claim 9 are executed.

13. A computer-readable storage medium Characterized in that A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the neural network training method for image processing according to any one of claims 1 to 8 are executed; or, the steps of the face recognition method according to claim 9 are executed.

Images