Training method and device for face recognition model

Abstract

The embodiment of the present application provides a method and device for training a face recognition model, wherein the method includes: obtaining image data; extracting image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; normalizing the image feature information through an improved normalization layer to obtain normalized image feature information; obtaining a feature vector from the normalized image feature information through an improved embedding layer; and training the face recognition model based on the feature vector. Since the scheme specifically uses the improved mobilenetV2 model as a feature extraction model, and correspondingly modifies the normalization layer and the embedding layer to simplify the model structure and take into account the processing accuracy; and then uses the improved model to train the face recognition model, the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing methods are solved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to a training method and device for a face recognition model. Background Art

[0002] In the field of face recognition technology, deep learning is usually used to train a neural network capable of face recognition, that is, a face recognition model.

[0003] At present, in order to train the above-mentioned face recognition model, most of the existing methods are to train and learn the neural network based on the FaceNet algorithm to obtain a face recognition model with high recognition accuracy. However, the above-mentioned FaceNet algorithm is limited by the implementation mechanism. Although the feature extraction model (DEEP ARCHITECTURE) contained therein has high processing accuracy, the structure of the above-mentioned model is relatively cumbersome, resulting in a relatively low processing rate during specific operation. In addition, other layers involved in the FaceNet algorithm, such as the normalization layer (L2 layer) and the embedding layer (EMBEDDING layer), also have some structural defects. For embedded ARM (Advance RISC Machines) devices, it is often required to ensure a relatively high processing accuracy while also requiring a relatively high processing rate. Therefore, the existing training methods for face recognition models based on the FaceNet algorithm are often not suitable for embedded ARM devices. In summary, when the existing methods are implemented, there are often technical problems such as low processing efficiency and poor applicability to embedded ARM devices.

[0004] To address the above problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for training a face recognition model to solve the technical problems of low processing efficiency and poor applicability to embedded ARM devices in existing methods, thereby achieving the technical effect of effectively improving processing efficiency while taking into account processing accuracy.

[0006] The present application provides a method for training a face recognition model, including:

[0007] Get image data;

[0008] Extracting image feature information from the image data by using an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model;

[0009] Normalizing the image feature information through an improved normalization layer to obtain normalized image feature information;

[0010] Obtaining a feature vector from the normalized image feature information through an improved embedding layer;

[0011] A face recognition model is trained based on the feature vector.

[0012] In one embodiment, the improved mobilenetV2 model is obtained in the following manner:

[0013] Delete the Linear layer in the mobilenetV2 model.

[0014] In one embodiment, the improved normalization layer is obtained in the following manner:

[0015] The normalization range of the normalization layer is modified to obtain the improved normalization layer, wherein the normalization range of the improved normalization layer is greater than or equal to 0 and less than or equal to 255.

[0016] In one embodiment, the improved embedding layer is obtained as follows:

[0017] The feature vector dimension of the embedding layer is expanded to obtain the improved embedding layer, wherein the feature vector dimension of the improved embedding layer is 256.

[0018] In one embodiment, training a face recognition model according to the feature vector includes:

[0019] Based on the feature vector, a face recognition model is trained using an improved multivariate loss layer.

[0020] In one embodiment, the improved multivariate loss layer is obtained as follows:

[0021] The loss function in the multivariate loss layer is modified to obtain the improved multivariate loss layer, wherein the loss function in the improved multivariate loss layer is a ternary loss function.

[0022] In one embodiment, the ternary loss function is a loss function generated by taking the first facial data of the target person, the second facial data of the target person, and the facial data of the non-target person as input, wherein the first facial data of the target person and the second facial data of the target person are facial data of the target person in different states.

[0023] The present application also provides a training device for a face recognition model, including:

[0024] A first acquisition module, used for acquiring image data;

[0025] An extraction module, used for extracting image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model;

[0026] A normalization module, used for normalizing the image feature information through an improved normalization layer to obtain normalized image feature information;

[0027] A second acquisition module, used for acquiring a feature vector from the normalized image feature information through an improved embedding layer;

[0028] The training module is used to train the face recognition model according to the feature vector.

[0029] In one embodiment, the apparatus is applied in an embedded ARM device.

[0030] The embodiment of the present application also provides a computer-readable storage medium having computer instructions stored thereon, which, when executed, realizes acquiring image data; extracting image feature information in the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; normalizing the image feature information through an improved normalization layer to obtain normalized image feature information; acquiring a feature vector from the normalized image feature information through an improved embedding layer; and training a face recognition model based on the feature vector.

[0031] In the embodiment of the present application, an improved mobilenetV2 model that is easy to converge and has a fast processing speed is used as a feature extraction model, and the normalization layer and the embedding layer are modified accordingly to simplify the model structure and take into account the processing accuracy; the improved model is then used to train the face recognition model, thereby solving the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing method, and achieving the technical effect of effectively improving processing efficiency while taking into account the processing accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 is a processing flow chart of a training method for a face recognition model provided in accordance with an embodiment of the present application;

[0034] Figure 2 It is a structural diagram of the FaceNet algorithm in the existing method;

[0035] Figure 3 It is a schematic diagram of a partial principle structure of an improved mobilenetV2 model in a training method for a face recognition model provided in an embodiment of the present application;

[0036] Figure 4 It is a structural diagram of a training device for a face recognition model provided in accordance with an embodiment of the present application;

[0037] Figure 5 It is a schematic diagram of the structure of an electronic device based on the training method of the face recognition model provided in the embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0039] Considering that most of the existing training methods of face recognition models are based on the FaceNet algorithm, and the model structure in the FaceNet algorithm is relatively cumbersome and complex, the FaceNet algorithm can obtain good processing accuracy, but the processing efficiency is relatively low. For example, the feature extraction model in the FaceNet algorithm often uses a neural network model such as inception_resNet. This type of neural network model is deep and wide in overall structure, and has the advantage of rich extracted features; but it is precisely because of the above structural characteristics that when the above neural network model is used as a feature extraction model to train the face recognition model, it is often not easy to converge and the processing speed is relatively slow. In addition, other models involved in the FaceNet algorithm, such as the normalization layer and the embedding layer, also have certain structural shortcomings. Due to the above shortcomings in the structure of the model involved in the FaceNet algorithm, the existing face recognition algorithm based on the FaceNet algorithm often has technical problems such as low processing efficiency and poor applicability to embedded ARM devices when it is implemented. In view of the root cause of the above technical problems, this application considers that the models involved in the FaceNet algorithm, such as the model structures of the feature extraction model, normalization layer, and embedding layer, can be improved respectively, so as to simplify the model structure and improve the processing efficiency on the basis of taking into account the processing accuracy, so that it can be applied to embedded ARM devices. This solves the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing methods, and achieves the technical effect of effectively improving processing efficiency while taking into account the processing accuracy.

[0040] Based on the above thinking, the present application embodiment provides a method for training a face recognition model. Figure 1 The processing flow chart of the training method of the face recognition model provided in the embodiment of the present application is shown. The training method of the face recognition model provided in the embodiment of the present application may include the following steps when implemented.

[0041] S11: Acquire image data.

[0042] In this embodiment, the above image data may specifically be image data containing a human face, and accordingly, the above image data may be understood as training sample data for training a face recognition model. The face recognition model is a neural network model for face recognition to be trained by the face recognition model training method provided in the embodiment of the present application.

[0043] In this embodiment, the acquisition of image data may specifically include: collecting photos containing human faces as the image data; and / or intercepting pictures containing human faces from video data as the image data. Of course, it should be noted that the above-listed methods for acquiring image data are only for better illustrating the implementation of the present application. In specific implementation, other methods may be selected to acquire the image data according to specific application scenarios, and the present application does not limit this.

[0044] S12: extracting image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model.

[0045] In this embodiment, the feature extraction model (ie, DEEP ARCHITECTURE) can be specifically understood as a network model used to extract image feature information of training sample data, ie, input image data, during the process of training a face recognition model.

[0046] In this embodiment, please refer to Figure 2 The structural diagram of the FaceNet algorithm in the existing method shown in the figure shows that, through in-depth analysis of the model structure involved in the FaceNet algorithm used in the existing method, it is found that in order to ensure processing accuracy, the existing method usually chooses to use a network model such as inception_resNet as the above-mentioned feature extraction model. The above-mentioned type of network model has certain advantages in extracting image feature information due to its deep and wide characteristics in the model structure, and can extract relatively rich image feature information. However, due to the above-mentioned model structure characteristics, when training the face recognition model, there are also problems such as training is not easy to converge and the calculation speed is slow. It is precisely because of the above-mentioned problems that the overall efficiency of the FaceNet algorithm is relatively low during operation, and it cannot even be well promoted and applied to operating environments such as embedded ARM devices.

[0047] In this embodiment, it should be noted that the above-mentioned embedded ARM device can specifically refer to a non-computer electronic device or device that contains an ARM (Advance RISC Machines) chip and has a certain data processing capability. Commonly used embedded ARM devices can be specifically applied to a large number of electrical devices in life, such as handheld PDAs, mobile computing devices, TV set-top boxes, mobile Internet access, digital TV, multimedia, cars, microwave ovens, digital cameras, home automation systems, elevators, air conditioners, security systems, vending machines, etc. Among them, the above-mentioned embedded ARM devices usually need to have the characteristics of small software code, high automation, and fast response speed during operation due to the needs of application scenarios. However, the existing training method based on the FaceNet face recognition model is not suitable for operating environments such as embedded ARM devices due to the relatively complex model structure itself and relatively slow processing efficiency.

[0048] Considering the above-mentioned problems existing in the existing methods using network models such as inception_resNet as the above-mentioned feature extraction models, and considering the specific application scenarios: for embedded ARM devices, the most important concern is processing efficiency first, and then processing accuracy. Therefore, in the implementation mode of the present application, an improved mobilenetV2 model is used to replace the network models such as inception_resNet used in the existing method as the feature extraction model to effectively improve the overall processing efficiency, so as to be better suitable for operating environments such as embedded ARM devices.

[0049] In this embodiment, the mobilenetV2 model can be specifically understood as a neural network with relatively higher processing efficiency, which is a network model obtained by modifying the mobilenetV1 model accordingly. The network structure thereof can be specifically referred to in the Mobilenet V2 network structure parameter table shown in Table 1.

[0050] MobileNet V2

[0051]

[0052] In one embodiment, combined with a specific application scenario, it is considered that in the process of training a face recognition model, it is not actually necessary to perform regression processing on the acquired image feature information. Therefore, the original Linear layer in the mobilenetV2 model, as an activation layer for regression processing, is actually redundant in the process of training a face recognition model. Based on the above considerations, in order to further improve the processing rate and simplify the model, in a specific implementation, the mobilenetV2 model can be improved in the following manner to obtain an improved mobilenetV2 model: Delete the Linear layer in the mobilenetV2 model.

[0053] In this embodiment, please refer to Figure 3 The diagram shows a partial schematic structure diagram of the improved mobilenetV2 model in the training method of the face recognition model provided in the embodiment of the present application. It should be noted that what is shown in the figure is a schematic diagram of the principle structure of the core part of the improved mobilenetV2 model. Among them, the PW part is specifically used to expand the dimension of the input image data (corresponding to expansion). The DW (depthWise) part is specifically used to independently process the image data of each channel through convolution processing to obtain separated feature information, thereby reducing the calculation amount of the model and shortening the convergence time of training (corresponding to convolution). This is also one of the reasons why the improved mobilenetV2 model has high processing efficiency compared to network models such as inception_resNet and is suitable for fast operation in environments such as embedded (ARM) devices. The PW (PointWise) part is specifically used to link the feature information separated from each channel to obtain the image feature information of the image data (corresponding to compression).

[0054] It should be noted that the Linear part of the mobilenetV2 model is used to regress the feature information output by the PW part using a linear regression activation function to obtain the result data. However, considering the specific application scenario of training face recognition models, it is not necessary to apply the result data obtained by the regression of the above Linear part. In order to further improve the processing efficiency of the model, the improved mobilenetV2 model deletes the Linear part (i.e., the Linear layer) in the mobilenetV2 model and simplifies the model structure. Figure 3 As shown in the figure, the improved mobilenetV2 model does not have a Linear layer compared to the ordinary mobilenetV2 model, but is directly connected to the normalization layer (i.e., L2 layer). This can further improve the efficiency of extracting image feature information using the feature extraction model.

[0055] S13: performing normalization processing on the image feature information through an improved normalization layer to obtain normalized image feature information.

[0056] In this embodiment, the image feature information is normalized, which can be specifically understood as unifying the image feature information of different dimensions into a numerical range.

[0057] In one embodiment, based on the premise that the processing efficiency is effectively improved by using the improved mobilenetV2 model as the feature extraction model, the normalization layer can be improved and the image feature information can be normalized using the improved normalization layer to retain the differences in feature information of different dimensions, so that the feature layer distribution of the model is more uniform, which can help improve the processing accuracy.

[0058] In this embodiment, the existing normalization layer based on the FaceNet algorithm is often used to normalize the input image feature information to the numerical range of [0, 1]. This normalization method will make the image feature information of different dimensions tend to be more concentrated, resulting in the difference between the image feature information of different dimensions is not obvious enough, and it is difficult to converge, thereby affecting the processing accuracy.

[0059] In one embodiment, considering the above-mentioned problems existing in the normalization layer of the existing method when performing normalization processing, the normalization layer (ie, L2 layer) is modified and adjusted accordingly to obtain an improved normalization layer (ie, improved L2 layer).

[0060] In one embodiment, during specific implementation, the normalization layer may be modified accordingly in the following manner to obtain an improved normalization layer: the normalization range of the normalization layer is modified to obtain the improved normalization layer, wherein the normalization range of the improved normalization layer is greater than or equal to 0 and less than or equal to 255. That is, the normalization range is expanded from the original [0, 1] to [0, 255], so that the normalized image feature information is more evenly distributed, the mixed interference of feature information of different dimensions is avoided, the robustness of the model is improved, and the processing accuracy is further improved.

[0061] S14: Obtaining a feature vector from the normalized image feature information through an improved embedding layer.

[0062] In this embodiment, the analysis of the existing embedding layer (i.e., EMBEDDING layer) based on the FaceNet algorithm is often to determine the corresponding feature vector based on the normalized input image feature information, so as to subsequently use the above feature vector for specific model training. However, since the vector dimension of the embedding layer used in the existing method is usually 128, the dimension of the feature vector obtained based on the above embedding layer is often limited, and thus it is impossible to obtain comprehensive and rich feature vectors of various dimensions, which in turn affects the processing accuracy.

[0063] In one embodiment, taking into account the above-mentioned problems existing in the embedding layer, based on the premise of effectively improving the processing efficiency by using the improved mobilenetV2 model as the feature extraction model, in order to ensure that more comprehensive and rich feature vectors are obtained from the normalized image feature information and further improve the processing accuracy, in the specific implementation, the embedding layer used in the existing method can be improved in a targeted manner, and then the improved embedding layer (i.e., the improved EMBEDDING layer) can be used to obtain richer feature vectors from the normalized image feature information.

[0064] In one embodiment, during specific implementation, the embedding layer can be modified and adjusted in the following manner to obtain an improved embedding layer: the feature vector dimension of the embedding layer is expanded to obtain the improved embedding layer, wherein the feature vector dimension of the improved embedding layer is 256. In this way, the embedding layer with the expanded vector dimension can be used in conjunction with the normalization layer with the expanded normalization range to obtain feature vectors with more dimensions and richer features in the image data, making the model more expressive and further improving the processing accuracy.

[0065] S15: Training a face recognition model according to the feature vector.

[0066] In this embodiment, during specific implementation, the above-mentioned feature vector can be used as input, and the neural network model can be trained using a loss function through a multivariate loss layer (i.e., a Triplet Loss layer) to obtain the above-mentioned face recognition model. That is, according to the feature vector, the face recognition model is trained using a multivariate loss layer.

[0067] In this embodiment, considering that the processing efficiency is effectively improved by the improved feature extraction model, the dimension and expressiveness of the feature vector are effectively improved by the improved normalization layer and the improved embedding layer, so that the feature vector with better effect can be used through the multivariate loss layer to train a face recognition model with higher accuracy. In addition, since the improved feature extraction model effectively shortens the training convergence time and improves the processing efficiency, the overall speed of training the face recognition model is also significantly improved compared with the existing methods.

[0068] In this embodiment, the existing embedding layer based on the FaceNet algorithm is analyzed to optimize the face recognition model using a binary loss function. However, the binary loss function usually maps the facial features of the same individual to the same point in space. It can be understood that the result of the above mapping may make the intra-class distance greater than the inter-class distance, which may easily lead to errors in processing and affect the processing accuracy.

[0069] In one embodiment, taking into account the above-mentioned problems existing in the existing embedding layer, in order to further improve the processing accuracy, the loss function used by the multi-source loss layer can be adjusted and modified in a targeted manner to obtain an improved multivariate loss layer (i.e., an improved Triplet Loss layer). The improved multivariate loss layer can then be used to more effectively and accurately optimize the face recognition model, thereby obtaining a more accurate face recognition model and improving the processing accuracy.

[0070] In one embodiment, during specific implementation, the loss function used by the multi-source loss layer can be modified in the following manner to obtain an improved multivariate loss layer: modify the loss function in the multivariate loss layer to obtain the improved multivariate loss layer, wherein the loss function in the improved multivariate loss layer is a ternary loss function.

[0071] In this embodiment, the above-mentioned ternary loss function is different from the binary loss function and can be specifically a loss function generated by taking the first facial data of the target person, the second facial data of the target person, and the facial data of the non-target person as input. Among them, the first facial data of the target person and the second facial data of the target person are facial data of the target person in different states. Non-target persons can be specifically understood as other people different from the target person. For example, the first facial data of the target person and the second facial data of the target person can be two photos of the same target person taken under different lighting environments, containing the face of the same target person. Of course, the above-mentioned different states can also specifically be different distance states, different angle states, and so on.

[0072] In this embodiment, it should be noted that the above ternary loss function can be understood in principle as mapping any bright feature of the same individual to the same area in space, so that the intra-class distance is smaller than the inter-class distance, that is, the distance between the faces of the same person is relatively minimum, and the distance between the faces of different people is relatively maximum. In this way, using the ternary loss function to replace the conventional binary loss function to train the face recognition model can effectively reduce errors and further improve processing accuracy.

[0073] In one embodiment, the above ternary loss function can be specifically expressed as follows:

[0074]

[0075] By deduction, we can get the following form:

[0076]

[0077] Among them, x represents the input face picture (or the feature information of the image), the superscript a is used to identify the first face of the target person, the superscript n is used to represent the face of the non-target person, the superscript p is used to identify the second face of the target person, and f represents the feature vector obtained based on the picture (or the feature information of the image).

[0078] In this way, by utilizing the improved multivariate loss layer and using a ternary loss function with smaller error and higher accuracy to train and optimize the face recognition model, the processing accuracy can be improved and the error can be reduced.

[0079] In one embodiment, the image data may specifically be image data containing a human face. Of course, the image data listed above is only a schematic illustration. During specific implementation, image data of other contents may also be obtained according to specific application scenarios. This application does not limit this.

[0080] From the above description, it can be seen that the training method of the face recognition model provided in the embodiment of the present application uses the improved mobilenetV2 model that is easy to converge and has a fast processing speed as a feature extraction model in a targeted manner, and modifies the normalization layer and the embedding layer accordingly to simplify the model structure and take into account the processing accuracy; the improved model is then used to train the face recognition model, thereby solving the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing method, and achieving the technical effect of effectively improving processing efficiency while taking into account processing accuracy; by expanding the normalization range of the normalization layer, the difference of feature information of different dimensions is made more obvious, and the mixed interference of feature information of different dimensions is avoided, thereby improving the robustness of the model and the processing accuracy; by expanding the feature vector dimension of the embedding layer, the dimension of the acquired feature vector is richer and more expressive, and the processing accuracy is further improved; by modifying the loss function in the multivariate loss layer to a ternary loss function with better effect, the face recognition model can be optimized and trained more accurately, thereby further improving the processing accuracy.

[0081] Based on the same inventive concept, an embodiment of the present invention further provides a training device for a face recognition model, as described in the following embodiments. Since the principle of solving the problem by the training device for a face recognition model is similar to that of the training method for a face recognition model, the implementation of the training device for a face recognition model can refer to the implementation of the training method for a face recognition model, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable. Please refer to Figure 4 , is a composition structure diagram of a training device for a face recognition model provided in an embodiment of the present application. The device may specifically include: a first acquisition module 41, an extraction module 42, a normalization module 43, a second acquisition module 44, and a training module 45. The structure is described in detail below.

[0082] A first acquisition module 41, which can be specifically used to acquire image data;

[0083] The extraction module 42 may be specifically used to extract image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model;

[0084] The normalization module 43 may be specifically configured to perform normalization processing on the image feature information through an improved normalization layer to obtain normalized image feature information;

[0085] A second acquisition module 44, which may be specifically configured to acquire a feature vector from the normalized image feature information through an improved embedding layer;

[0086] The training module 45 can be specifically used to train a face recognition model according to the feature vector.

[0087] In this embodiment, the device may further include a first improvement module, wherein the first improvement module may be used to improve the mobilenetV2 model to obtain an improved mobilenetV2 model. In specific implementation, the first improvement module may improve the mobilenetV2 model according to the following procedure: deleting the Linear layer in the mobilenetV2 model.

[0088] In one embodiment, the device may further include a second improved model, wherein the second improved module may be specifically used to improve the normalization layer to obtain an improved normalization layer. In specific implementation, the second improved module may improve the normalization layer according to the following procedure: modify the normalization range of the normalization layer to obtain the improved normalization layer, wherein the normalization range of the improved normalization layer is greater than or equal to 0 and less than or equal to 255.

[0089] In one embodiment, the apparatus may further include a third improvement module, wherein the third improvement module may be specifically used to improve the embedding layer to obtain an improved embedding layer. In specific implementation, the third improvement module may improve the embedding layer according to the following procedure: expanding the feature vector dimension of the embedding layer to obtain the improved embedding layer, wherein the feature vector dimension of the improved embedding layer is 256.

[0090] In one embodiment, in order to train a face recognition model according to the feature vector, the above training module can be trained according to the following procedure when it is implemented: according to the feature vector, the face recognition model is trained using an improved multivariate loss layer.

[0091] In one embodiment, the device may further include a fourth improvement module, wherein the fourth improvement module may be specifically used to improve the multivariate loss layer. In specific implementation, the fourth improvement module may improve the multivariate loss layer according to the following procedure: modify the loss function in the multivariate loss layer to obtain the improved multivariate loss layer, wherein the loss function in the improved multivariate loss layer is a ternary loss function.

[0092] In one embodiment, the ternary loss function can specifically be a loss function generated by taking the first facial data of the target person, the second facial data of the target person, and the facial data of the non-target person as input, wherein the first facial data of the target person and the second facial data of the target person are facial data of the target person in different states.

[0093] In one embodiment, the image data may specifically be image data containing a human face. Of course, the image data listed above is only a schematic illustration. During specific implementation, image data of other contents may also be obtained according to specific application scenarios. This application does not limit this.

[0094] In one embodiment, the above apparatus can be specifically applied to an embedded ARM device.

[0095] In this embodiment, it should be noted that the above-mentioned embedded ARM device can specifically refer to a non-computer electronic device or device that contains an ARM chip and has a certain data processing capability. Commonly used embedded ARM devices can be specifically applied to a large number of electrical devices in life, such as handheld PDAs, mobile computing devices, TV set-top boxes, mobile Internet access, digital TV, multimedia, cars, microwave ovens, digital cameras, home automation systems, elevators, air conditioners, security systems, vending machines, etc. Among them, the above-mentioned embedded ARM devices usually need to have the characteristics of small software code, high automation, and fast response speed during operation due to the needs of application scenarios. However, the existing training method based on the FaceNet face recognition model is not suitable for operating environments such as embedded ARM devices due to the relatively complex model structure itself and relatively slow processing efficiency.

[0096] In this embodiment, since the above-mentioned device has high processing efficiency while taking into account processing accuracy, the above-mentioned device is suitable for deployment in operating environments such as embedded ARM devices. Of course, the embedded ARM devices listed above are only a schematic illustration. During specific implementation, the training device of the face recognition model provided by the embodiment of the present application can also be applied to other types of operating environments according to specific application scenarios and implementation requirements. This application is not limited to this.

[0097] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0098] It should be noted that the systems, devices, modules or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, in this specification, the above devices are described in various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0099] In addition, in this specification, adjectives such as first and second may only be used to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be interpreted as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0100] From the above description, it can be seen that the training device for the face recognition model provided in the embodiment of the present application uses the improved mobilenetV2 model that is easy to converge and has a fast processing speed as a feature extraction model, and correspondingly modifies the normalization layer and the embedding layer to simplify the model structure and take into account the processing accuracy; and then uses the improved model to train the face recognition model through the extraction module, the normalization module, the second acquisition module, and the training module, thereby solving the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing methods, achieving the goal of taking into account the processing accuracy while effectively improving the performance. The technical effect of processing efficiency is improved; the normalization range of the normalization layer is expanded through the second improvement module, so that the difference of feature information of different dimensions is more obvious, the mixed interference of feature information of different dimensions is avoided, the robustness of the model is improved, and the processing accuracy is improved; the feature vector dimension of the embedding layer is expanded through the third improvement module, so that the dimension of the acquired feature vector is richer and more expressive, and the processing accuracy is further improved; the loss function in the multivariate loss layer is modified into a ternary loss function with better effect through the fourth improvement module, so that the face recognition model can be optimized and trained more accurately, and the processing accuracy is further improved.

[0101] The present application also provides an electronic device. Figure 5 The schematic diagram of the structure of the electronic device based on the training method of the face recognition model provided in the embodiment of the present application is shown, and the electronic device may specifically include an input device 51, a processor 52, and a memory 53. Among them, the input device 51 can be specifically used to input image data. The processor 52 can be specifically used to extract image feature information in the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; the image feature information is normalized by an improved normalization layer to obtain normalized image feature information; a feature vector is obtained from the normalized image feature information by an improved embedding layer; and a face recognition model is trained according to the feature vector. The memory 53 can be specifically used to store image data input through the input device 51, intermediate data generated by the processor 52, and related program instructions.

[0102] In this embodiment, the input device may specifically be one of the main devices for information exchange between the user and the computer system. The input device may include a keyboard, a mouse, a camera, a scanner, a light pen, a handwriting input board, a voice input device, etc.; the input device is used to input the original data and the program for processing these numbers into the computer. The input device can also obtain and receive data transmitted from other modules, units, and devices. The processor can be implemented in any appropriate manner. For example, the processor can take the form of a computer-readable medium, a logic gate, a switch, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable logic controller, and an embedded microcontroller, etc., such as a microprocessor or a processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor. The memory may specifically be a memory device used to store information in modern information technology. The memory may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function without a physical form is also called a memory, such as a RAM, a FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, a TF card, etc.

[0103] In this embodiment, the functions and effects specifically realized by the electronic device can be explained in comparison with other embodiments and will not be described in detail here.

[0104] The present application also provides a computer storage medium for a training method based on a face recognition model, wherein the computer storage medium stores computer program instructions, which when executed implement the following: acquiring image data; extracting image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; normalizing the image feature information through an improved normalization layer to obtain normalized image feature information; acquiring a feature vector from the normalized image feature information through an improved embedding layer; and training a face recognition model based on the feature vector.

[0105] In this embodiment, the storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD) or a memory card. The memory may be used to store computer program instructions. The network communication unit may be an interface for network connection communication set in accordance with the standard specified by the communication protocol.

[0106] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other embodiments and will not be repeated here.

[0107] In a specific implementation scenario example, the face recognition model training method and device provided in the embodiment of the present application is used to train the face recognition model applicable to the embedded ARM device. The specific implementation process can be referred to as follows.

[0108] The training method of the face recognition model used in this example is based on the FaceNet algorithm used in the existing method and is improved accordingly. Therefore, here we first briefly analyze and explain the implementation process of the FaceNet algorithm used in the existing method and the existing problems.

[0109] Among them, the FaceNet algorithm structure can be found in Figure 2 Content shown.

[0110] The specific implementation can include the following five steps:

[0111] Step 1: Get a set of images of the input source of batch size (i.e. get image data).

[0112] Step 2: The input source obtains feature information through the feature extraction neural network model (equivalent to the feature extraction model). The feature extraction neural network model in the original FaceNet algorithm uses a deep and wide neural network model such as inception_resNet. The advantage of this model is that it extracts rich features, but the disadvantage is that the training is not easy to converge and the calculation speed is slow.

[0113] Step 3: Normalize the extracted features accordingly. The normalization interval (i.e., normalization range) of the normalization layer in the original FaceNet algorithm is between [0,1]. However, for this process, different feature dimensions tend to be more clustered, and the differences between different features of the input source tend to be more difficult to distinguish.

[0114] Step 4: This step is the FaceNet training optimization process, that is, the process of defining the loss function and using the loss function to optimize the model training. In this step, the original loss function used was a binary loss function. The goal of the binary loss function is to map the facial features of the same individual to the same point in space, which makes it easy to generate errors in the training optimization process.

[0115] Based on the implementation process of the FaceNet algorithm used in the existing methods and the existing problems, this example makes targeted improvements to the above steps and obtains the training method of the face recognition model used in this example to improve the processing efficiency. Specific improvements may include the following.

[0116] For the improvement of the second step, in order to improve the processing efficiency on embedded (ARM) devices, mobileNetV2 is selected as the feature extraction model to extract the corresponding features. That is, mobileNetV2 is used as the basic model in the feature extraction model represented by DEEP ARCHITECTURE, and the model is modified accordingly (to obtain the improved mobileNetV2 model).

[0117] See also Figure 3 As shown in the figure, the PW part on the left in the mobileNetV2 model is used to expand the dimension of the number of image channels. The middle DW (depthWise) part is used for convolution processing to process each channel independently, which can greatly reduce the corresponding amount of calculation. This is one of the reasons why mobileNet can run quickly on embedded devices. The PW (PointWise) part on the right is used to connect the feature dimensions separated by each channel, unify the features and finally use the linear regression activation function to regress and get the results. It should be noted that the last layer of mobileNetV2 in the figure uses Linear (i.e., Linear layer) to regress the features accordingly, which can be understood as the activation layer of mobileNet. However, in the application scenario of training face recognition models, this layer is not needed, so it can be removed (i.e., delete the Linear layer in the mobilenetV2 model).

[0118] For the improvement of the third step, if the Linear layer is removed, the next layer is the L2 layer (i.e. the normalization layer). In order to avoid the different feature dimensions in the existing method tending to be more clustered, making the differences between different features of the input source more difficult to distinguish. This step changes the normalization interval to [0,255]. This can expand the feature difference dimension range to enhance the model robustness, and can also enhance the model robustness.

[0119] In addition, the embedding layer (i.e., embedding layer) after the L2 layer has been modified accordingly. The original embedding layer feature dimension is 128 bits. Considering that for the original network model, if the dimensions are set to be too large, the convergence of the model will be slower and the training difficulty will increase. However, in this example, since mobileNetV2 is used as the deep learning network model, the amount of training calculations will be greatly reduced and the convergence time will be shortened, so the dimension of the final Euclidean vector is expanded here. Finally, it is expanded to 256 dimensions. The vector range obtained in this way is wider, because the normalization range of the L2 layer is also larger, making the final feature distribution of the model more uniform and less prone to mixed interference, which improves the robustness of the model.

[0120] For the improvement of the fourth step, the original binary loss function is replaced by the ternary loss function for training optimization. The goal of the ternary loss function is to map to the same area so that the intra-class distance is smaller than the inter-class distance.

[0121] Specifically, assuming that x is the input image and f is the feature vector after CNN representation, the ternary loss function can be expressed as follows:

[0122]

[0123] Furthermore, the final ternary loss function can be obtained as:

[0124]

[0125] From the above function, we can see that the above formula requires three input portraits. How to choose this triplet for training? In order to ensure the convergence speed of training, we choose the same portrait with the farthest distance and the different portrait with the closest distance for training.

[0126] Then, the face recognition model applicable to the embedded ARM device can be trained by the above-mentioned improved steps, that is, using the face recognition model training method provided by the embodiment of the present application. That is: acquiring image data;

[0127] The image feature information in the image data is extracted by an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; the image feature information is normalized by an improved normalization layer to obtain normalized image feature information; a feature vector is obtained from the normalized image feature information by an improved embedding layer; and a face recognition model is trained according to the feature vector.

[0128] By comparing with the existing methods, we can see that: in order to solve the problem of computing speed of face recognition on embedded ARM devices, the DEEP ARCHITECTURE network in FaceNet is replaced with mobileNetV2, and some details of mobileNetV2 and FaceNet are modified accordingly. Specifically, the mobileNetV2 model is used as the basic network model of the feature extraction model, and the corresponding Linear layer is removed. The number of lightweight layers obtained in this way is conducive to the training convergence of the model and improves processing efficiency. Expanding the normalization range of the L2 layer is conducive to the distribution of the model feature layer, which is conducive to improving the model accuracy. Expanding the dimension of the Euclidean vector of the Embedding layer is conducive to expanding the breadth of model features. This model has stronger expressiveness and is conducive to improving the model accuracy.

[0129] Through the above scenario examples, the training method and device of the face recognition model provided by the embodiments of the present application are verified. By using the improved mobilenetV2 model that is easy to converge and has a fast processing speed as the feature extraction model, and modifying the normalization layer and the embedding layer accordingly to simplify the model structure and take into account the processing accuracy; the improved model is then used to train the face recognition model, which indeed solves the technical problems of low processing efficiency and poor applicability to embedded ARM devices in the existing methods, and achieves the technical effect of effectively improving the processing efficiency while taking into account the processing accuracy.

[0130] Although different specific embodiments are mentioned in the content of this application, this application is not limited to the situations described in the industry standards or embodiments, etc. Some industry standards or slightly modified implementation plans based on the implementation of the custom methods or embodiments can also achieve the same, equivalent or similar, or predictable implementation effects after deformation of the above embodiments. The embodiments of data acquisition, processing, output, judgment methods, etc. after these modifications or deformations can still fall within the scope of the optional implementation plans of this application.

[0131] Although the present application provides method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent a unique execution order. When the device or client product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "includes", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements.

[0132] The devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by dividing them into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules. The device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0133] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0134] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0135] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0136] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0137] Although the present application has been described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present application without departing from the spirit of the present application. It is intended that the attached embodiments include these modifications and changes without departing from the present application.

Claims

1. A training method for a face recognition model, characterized in that: include: Get image data; Extracting image feature information from the image data by using an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; Normalizing the image feature information through an improved normalization layer to obtain normalized image feature information; Obtaining a feature vector from the normalized image feature information through an improved embedding layer; Training a face recognition model according to the feature vector; The improved mobilenetV2 model is obtained in the following manner: Delete the Linear layer in the improved mobilenetV2 model; The improved normalization layer is obtained in the following manner: Modifying a normalization range of a normalization layer to obtain the improved normalization layer, wherein the normalization range of the improved normalization layer is greater than or equal to 0 and less than or equal to 255; The improved embedding layer is obtained in the following manner: The feature vector dimension of the embedding layer is expanded to obtain the improved embedding layer, wherein the feature vector dimension of the improved embedding layer is 256.

2. The method according to claim 1, characterized in that: According to the feature vector, a face recognition model is trained, including: Based on the feature vector, a face recognition model is trained using an improved multivariate loss layer.

3. The method according to claim 2, characterized in that The improved multivariate loss layer is obtained as follows: The loss function in the multivariate loss layer is modified to obtain the improved multivariate loss layer, wherein the loss function in the improved multivariate loss layer is a ternary loss function.

4. The method according to claim 3, characterized in that: The ternary loss function is a loss function generated by taking the first facial data of the target person, the second facial data of the target person, and the facial data of the non-target person as input, wherein the first facial data of the target person and the second facial data of the target person are facial data of the target person in different states.

5. A training device for a face recognition model, characterized in that: include: A first acquisition module, used for acquiring image data; An extraction module, used for extracting image feature information from the image data through an improved feature extraction model, wherein the improved feature extraction model is an improved mobilenetV2 model; A normalization module, used for normalizing the image feature information through an improved normalization layer to obtain normalized image feature information; A second acquisition module, used for acquiring a feature vector from the normalized image feature information through an improved embedding layer; A training module, used for training a face recognition model according to the feature vector; The improved mobilenetV2 model is obtained in the following manner: Delete the Linear layer in the improved mobilenetV2 model; The improved normalization layer is obtained in the following manner: Modifying a normalization range of a normalization layer to obtain the improved normalization layer, wherein the normalization range of the improved normalization layer is greater than or equal to 0 and less than or equal to 255; The improved embedding layer is obtained in the following manner: The feature vector dimension of the embedding layer is expanded to obtain the improved embedding layer, wherein the feature vector dimension of the improved embedding layer is 256.

6. The device according to claim 5, characterized in that The device is applied to embedded ARM equipment.

7. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the instructions are executed, the steps of the method according to any one of claims 1 to 4 are implemented.

Images