Face recognition model optimization method, device and equipment
By adding parallel Conv-LoRA layers to the key layer of the convolutional neural network, fine-tuning its output, the problem of poor fine-tuning flexibility of convolutional neural networks on different scenario data sets is solved, and efficient model fine-tuning and resource conservation are achieved .
Patent Information
- Application Number
- CN202411288111.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2025-01-17
AI Technical Summary
Existing convolutional neural networks have poor training flexibility when fine-tuning data sets adapted to different scenarios and require high computing resources, making it difficult to efficiently fine-tune the model on limited resource devices.
By obtaining the pre-trained convolutional neural network model, select the first n convolutional layer modules with the largest differences in iterative training as key layers, and add parallel Conv-LoRA layers at these key layers to fine-tune their outputs. The image data set in practical application scenarios trains the Conv-LoRA layer to reduce training parameters and memory usage.
It realizes efficient fine-tuning of the face recognition model in actual use scenarios, improves the model's migration and application capabilities under different data distributions, and reduces the training resource requirements.
Smart Images
Figure CN119323812A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, in particular to technical fields such as computer vision, deep learning, and face recognition, and more specifically to an optimization method, device, and equipment for a face recognition model. Background Art
[0002] Face recognition technology is widely used in security monitoring, identity authentication, and human-computer interaction. Among them, convolutional neural network (CNN) plays a very important role in face recognition technology. CNN is a deep learning algorithm that automatically learns the features of an image by simulating the working principle of the human visual system. In the field of face recognition, CNN can effectively extract features from face images and use them to distinguish different faces.
[0003] In the prior art, although the deep learning model of the convolutional neural network performs well on large-scale face datasets, when it is fine-tuned on differentiated datasets to adapt to different scenarios or trained on devices with limited computing resources, there are problems such as poor flexibility in training fine-tuning and high requirements for computing resources for fine-tuning training. Summary of the invention
[0004] In order to overcome at least one of the above-mentioned defects of the prior art, the present invention provides a method, device, equipment and storage medium for optimizing a face recognition model. The technical solution adopted by the present invention is as follows.
[0005] In a first aspect, the present invention provides a method for optimizing a face recognition model, comprising:
[0006] Obtaining a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition;
[0007] Selecting a key layer in a pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer;
[0008] Adding a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the output of the key layer;
[0009] Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer;
[0010] Y_new=Y_o+Y_delta;
[0011] In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change;
[0012] Y_delta = W_delta * X;
[0013] Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers;
[0014] Obtain a face image dataset in an actual application scenario, and use the face image dataset as training data to train the Conv-LoRA layer.
[0015] In one embodiment, the process of selecting a key layer in a pre-trained convolutional neural network model includes:
[0016] Use face image datasets from actual application scenarios for iterative training;
[0017] Compare the weight parameters of each convolutional layer module after iterative training with the weight parameters without iterative training to obtain the difference comparison results;
[0018] The first n convolutional layer modules with the largest differences in the difference comparison results are selected as the key layers.
[0019] In one embodiment, the process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes:
[0020] Subtract the corresponding position parameters of the weight parameters after iterative training from the weight parameters without iterative training for each convolutional layer module to obtain the difference value of the weight parameters;
[0021] According to the difference value of the weight parameter, the cumulative value of the parameter difference and the standard deviation of the parameter difference of each convolution layer module are counted;
[0022] A difference comparison result is obtained according to the cumulative value of the parameter difference and the standard deviation of the parameter difference.
[0023] In one embodiment, the process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes:
[0024] Normalize the weight parameters of each convolutional layer module after iterative training and the weight parameters without iterative training to obtain the processed weights, where CW_new' is the processed weight after iterative training and CW_o' is the processed weight without iterative training.
[0025] Calculate the distance L2 of each convolutional layer module, where L2 = ||CW_new'-CW_o'||^2;
[0026] A difference comparison result is obtained according to the distance L2.
[0027] In one embodiment, the two low channel convolutional layers are respectively a low channel convolutional layer A and a low channel convolutional layer B;
[0028] The low channel convolution layer A is used to reduce the dimension of the input feature, and the low channel convolution layer B is used to restore the dimension of the processed feature to the same dimension as the original output feature.
[0029] In one embodiment, the dimension of W_delta is (C_out*C_in*k*k);
[0030] The dimension of the channel convolution layer A is (C_r*C_in*1*1), and the dimension of the low channel convolution layer B is (C_r*C_out*k*k), where C_r is much smaller than C_in and C_out.
[0031] In one embodiment, it further includes:
[0032] Evaluate the performance of the fine-tuned model on the validation set, and take the model weight with the best evaluation result as the model weight actually used in the subsequent use.
[0033] In a second aspect, the present invention provides an optimization device for a face recognition model, comprising:
[0034] An acquisition module, used to acquire a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition;
[0035] A selection module is used to select a key layer in a pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer;
[0036] An insertion module is used to add a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the output of the key layer;
[0037] Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer;
[0038] Y_new=Y_o+Y_delta;
[0039] In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change;
[0040] Y_delta = W_delta * X;
[0041] Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers;
[0042] The training module is used to obtain a face image dataset in an actual application scenario, and use the face image dataset as training data to train the Conv-LoRA layer.
[0043] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods when executing the program.
[0044] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of any of the above-mentioned embodiments.
[0045] In the present invention, by finding out the convolutional layer modules with large differences in weight parameters of each convolutional layer module, they are defined as key layers, and these key layers are considered to have insufficient coverage scenarios. Then, by adding parallel Conv-LoRA layers, the weights of these key layers are fine-tuned to obtain an optimized face recognition model. The present invention can reduce training parameters and video memory usage, achieve efficient model fine-tuning, perform better in actual data scenarios, and can be easily migrated and applied to usage scenarios under different data distributions. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a schematic diagram of the overall process of an implementation method of Example 1 of the present invention.
[0047] Figure 2 This is a schematic diagram of the structure of the IResNet-100 network.
[0048] Figure 3 It is a schematic diagram of the structure of an implementation method of the first embodiment of the present invention in which a parallel Conv-LoRA layer is added.
[0049] Figure 4 It is a schematic diagram of the Conv-LoRA layer of another implementation manner of Example 1 of the present invention.
[0050] Figure 5 It is a schematic diagram of the overall structure of the second embodiment of the present invention. DETAILED DESCRIPTION
[0051] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0052] It should be noted that the terms "first\second\..." involved in the embodiments of the present invention are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\..." can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\..." can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.
[0053] Embodiment 1
[0054] See also Figure 1 , Figure 1 1 is a flow chart of a method for optimizing a face recognition model provided in Embodiment 1 of the present invention, the method comprising step S110, step S120, step S130 and step S140. It should be noted that step S110, step S120, step S130 and step S140 are merely reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps in this embodiment.
[0055] Step S110, obtaining a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition;
[0056] Step S120, selecting a key layer in the pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer;
[0057] Step S130, adding a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the weight of the key layer;
[0058] Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer;
[0059] Y_new=Y_o+Y_delta;
[0060] In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change;
[0061] Y delta = W delta * X;
[0062] Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers;
[0063] Step S140, obtaining a face image dataset in an actual application scenario, and using the face image dataset as training data to train the Conv-LoRA layer.
[0064] In step S110, a pre-trained convolutional neural network model is obtained, where the pre-trained convolutional neural network model is a convolutional neural network model for face recognition.
[0065] The basic structure of a convolutional neural network (CNN) includes a convolutional layer, a pooling layer, a fully connected layer, etc. The convolutional layer is responsible for extracting image features, and this method is mainly used on the convolutional layer.
[0066] Take the IResNet-100 network as an example. Figure 2 As shown, Figure 2 This is a schematic diagram of the IResNet-100 network structure, which includes: input downsampling layer, backbone feature extraction network, classification layer; the backbone feature extraction network contains 4 feature extraction sub-networks with different feature sizes, and each feature extraction sub-network consists of several IConv_block convolutional layer modules. The entire backbone feature extraction network has a total of 49 IConv_block convolutional layer modules.
[0067] The pre-trained convolutional neural network model has poor recognition performance in actual use scenarios. A big reason is that the scenes covered by its data set are insufficient, resulting in insufficient generalization of the trained model. In order for the model to perform well in actual use scenarios, it is necessary to fine-tune the training based on the images of the actual scene and change the parameters of the model to adapt to the data distribution characteristics of the new scene. Steps S120-140 are the process of fine-tuning the training based on the images of the actual scene.
[0068] Continuing with the previous example of the IResNet-100 network, its entire backbone feature extraction network has a total of 49 IConv_block convolutional layer modules. This method is to allow Conv-LoRA to act on the IConv_block convolutional layer module at any position in the backbone feature extraction network and fine-tune it to obtain a model that adapts to the new scene data distribution characteristics.
[0069] The backbone feature extraction network of the IResNet-100 network has a total of 49 IConv_block convolutional layer modules, and fine-tuning does not need to be performed on all layer modules, but only on some of the layer modules. Therefore, in step S120, the layer modules that need to be fine-tuned are selected.
[0070] In step S120, a number value n is preset, and the number value n can be obtained based on actual experience, and the convolutional layer modules ranked in the top n are selected. The ranking is based on the difference in iterative training, and the top n convolutional layer modules with the largest difference are the selected convolutional layer modules.
[0071] There are many specific methods for step S120. In one implementation, the process of step S120 includes: step S121, step S122 and step S123.
[0072] Step S121, using a face image dataset in an actual application scenario to perform iterative training;
[0073] Step S122, comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain a difference comparison result;
[0074] Step S123, selecting the first n convolutional layer modules with the largest differences in the difference comparison results as the key layers.
[0075] In this implementation, through iterative training, the differences between the weight parameters of each convolutional layer module are found. The larger the difference, the more it means that the data set does not cover enough scenarios and needs to be optimized in a targeted manner.
[0076] Specifically, in one implementation, the process of step S122 includes: step S210, step S220 and step S230.
[0077] Step S210, subtracting the corresponding position parameters of the weight parameters after iterative training from the weight parameters without iterative training of each convolutional layer module to obtain the difference value of the weight parameters;
[0078] Step S220, according to the difference value of the weight parameter, counting the cumulative value of the parameter difference and the standard deviation of the parameter difference of each convolutional layer module;
[0079] Step S230, obtaining a difference comparison result according to the cumulative value of the parameter difference and the standard deviation of the parameter difference.
[0080] This is the implementation of the difference method. The two layer parameters are directly subtracted and evaluated from the difference accumulation, difference mean, and difference variance.
[0081] After iterative training, the corresponding position parameters of the model weight W_new after iterative training are subtracted from the weight W_o of the original model to obtain the weight parameter difference value W_delta; the cumulative value SW_delta of the parameter difference of each convolutional layer module and the standard deviation STDWdelta of the parameter difference of each convolutional layer module are counted, and the two factors are comprehensively considered. For example, according to the preset weights of the two factors, the first n layers with the largest change range are selected for optimization and fine-tuning, and the frozen parameters of other layers with small cumulative differences are not adjusted.
[0082] Specifically, in another implementation, the process of step S122 includes: step S310, step S320 and step S330.
[0083] Step S310, normalizing the weight parameters of each convolutional layer module after iterative training and the weight parameters without iterative training to obtain the processed weights, where CW_new' is the processed weight after iterative training, and CW_o' is the processed weight without iterative training;
[0084] Step S320, calculating the distance L2 of each convolutional layer module, where L2 = ||CW_new'-CW_o'||^2;
[0085] Step S330, obtaining a difference comparison result according to the distance L2.
[0086] This is an implementation of the normalized distance metric. Correlation level of the convolution kernels of two layers, matrix similarity metric algorithm: normalized Euclidean distance.
[0087] After iterative training, the parameter weight CW_new of each convolutional layer module after iterative training and the weight CW_o of each convolutional layer module of the original model are normalized to obtain the processed weights CW_new', CW_o';
[0088] Calculate the distance L2 of the parameters of each convolutional layer module, where L2 = ||CW_new'-CW_o'||^2.
[0089] The first n convolutional layer modules with the largest changes are selected for optimization and fine-tuning, and the frozen parameters of other layers with small cumulative differences are not adjusted.
[0090] Of course, based on the distance L2 as one of the metrics, other metric factors can be ranked and processed comprehensively to select the appropriate layer to be fine-tuned.
[0091] After step S120, the first n convolutional layer modules are selected, step S130 is the process of fine-tuning these n convolutional layer modules, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the structure with the parallel Conv-LoRA layer added.
[0092] For convolutional deep learning models commonly used for face recognition, such as ResNet, VGG, etc., Conv-LoRA layers can be added to the key layers of the model. By adding variable convolution layers to the existing convolutional extraction blocks, adaptive adjustments can be made. This allows the generalization learned by the original pre-trained model on large-scale face data to be utilized, while adapting to the characteristics of the distribution of collected face data in specific actual situations.
[0093] LoRA (low rank adaptation) is a fine-tuning method commonly used for generative large models. Its essence is an approximate numerical decomposition technology for low-rank decomposition of feature change matrices, which can greatly reduce the number of parameters of feature change matrices. For the fine-tuning training of the original model of the key layer, it is not necessary to adjust all its parameters. Instead, a weight residual change is added. By training the weight residual, the optimization process of the original model weight is completed, and the adjusted fusion layer of the key layer is obtained:
[0094] like Figure 4 As shown, Figure 4 This is a schematic diagram of the Conv-LoRA layer. Conv-LoRA is an improvement made to the convolution structure based on the idea of LoRA to adapt to the feature input of the image shape. A variable convolution layer operation is used to approximate the change of the original convolution block. The number of input and output channels of the variable convolution layer is consistent with the number of channels of the original convolution block. The variable convolution layer can be replaced by two convolution layers with a lower number of channels to greatly reduce the number of training parameters.
[0095] Assuming that the number of input and output channels of Conv_block is C_in and C_out, the number of channels of the low-channel convolution A is (C_r*C_in), and the dimension of B is (C_out*C_r), where C_r is required to be much smaller than C_r and C_out; the smaller the value of C_r, the smaller the number of parameters that need to be trained in the Conv-LoRA model, but the effect of changing the convolution layer to approximate the change amount of the original convolution block will also deteriorate.
[0096] In order to ensure the stability of training, at least one of the convolution kernel parameters of the low-channel convolution A or B needs to be initialized to all zeros to avoid falling into a saddle point at the beginning of training.
[0097] The following uses IResNet-100 as an example to illustrate how to insert the Conv-LoRA layer into the convolutional model. IResNet-100 contains 4 feature size processing layers, a total of 49 IConv_block convolutional layer modules. Assuming that the key layer to be selected is a certain IConV_block convolutional layer module, its input and output dimensions are:
[0098] Input: B x C_in x H x W;
[0099] Output: B x C_out x H x W;
[0100] The number of parameters of the variable convolutional layer module is 2*C_in*C_out*kernel_size*kernel_size.
[0101] like Figure 3 As shown, the Conv-LoRA layer is used to assist fine-tuning training.
[0102] The low-channel convolution layer A has a similar function to the low-rank matrix A of LoRA, which is used to reduce the dimension of the input features. Its shape is C_r x C_in x 1x 1, and it performs dimensionality reduction on the single features corresponding to all channels;
[0103] The low channel convolution layer B has a similar function to the restoration matrix B of LoRA. It is used to restore the processing feature dimension to the same as the original output feature dimension, with a shape of C_out x C_r x kernel_size x kernel_size;
[0104] The low-rank dimension C_r is much smaller than min(C_in, C_out);
[0105] The number of parameters of the Conv-LoRA layer is C_in*C_r+C_r*C_out*kernel_size*kernel_size.
[0106] The parameters of IConV_block are frozen, and only the parameters of A and B layers are trained and adjusted.
[0107] A parallel Conv-LoRA layer is added, and step S140 is to train the parallel Conv-LoRA layer. First, freeze all parameters of the pre-trained convolutional neural network model; then, select a suitable optimizer, such as Adam or SGD, set the learning rate and batch size; then, select a suitable target loss function Loss, such as ArcFace_loss, CosFace_loss; finally, input the training data to calculate the target loss function, update the Conv-LoRA layer parameters through back propagation, reduce the target loss function value, and gradually improve the performance of the model in face recognition tasks.
[0108] In one embodiment, the method for optimizing the face recognition model further includes step S150.
[0109] Step S150, evaluate the performance of the fine-tuned model on the validation set, and take the model weight with the best evaluation result as the model weight actually used in the subsequent step.
[0110] In this method, we identify the convolutional layer modules with large differences in weight parameters of each convolutional layer module and define them as key layers. These key layers are considered to have insufficient coverage of the scene. Then, we add parallel Conv-LoRA layers to fine-tune the weights of these key layers, thereby obtaining an optimized face recognition model. This method can reduce training parameters and video memory usage, achieve efficient model fine-tuning, perform better in actual data scenarios, and can be easily migrated and applied to scenarios with different data distributions.
[0111] Embodiment 2
[0112] Corresponding to the method of embodiment 1, as Figure 5 As shown, the present invention also provides an optimization device for a face recognition model, comprising: an acquisition module 510, a selection module 520, an insertion module 530 and a training module 540;
[0113] An acquisition module 510 is used to acquire a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition;
[0114] A selection module 520 is used to select a key layer in the pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer;
[0115] Inserting module 530, used to add a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the output of the key layer;
[0116] Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer;
[0117] Y_new=Y_o+Y_delta;
[0118] In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change;
[0119] Y_delta = W_delta * X;
[0120] Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers;
[0121] The training module 540 is used to obtain a face image dataset in an actual application scenario, and use the face image dataset as training data to train the Conv-LoRA layer.
[0122] In one embodiment, the process of executing the key layer in the selected pre-trained convolutional neural network model by the selected module includes:
[0123] Use face image datasets from actual application scenarios for iterative training;
[0124] Compare the weight parameters of each convolutional layer module after iterative training with the weight parameters without iterative training to obtain the difference comparison results;
[0125] The first n convolutional layer modules with the largest differences in the difference comparison results are selected as the key layers.
[0126] In one embodiment, the process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes:
[0127] Subtract the corresponding position parameters of the weight parameters after iterative training from the weight parameters without iterative training for each convolutional layer module to obtain the difference value of the weight parameters;
[0128] According to the difference value of the weight parameter, the cumulative value of the parameter difference and the standard deviation of the parameter difference of each convolution layer module are counted;
[0129] A difference comparison result is obtained according to the cumulative value of the parameter difference and the standard deviation of the parameter difference.
[0130] In one embodiment, the process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes:
[0131] Normalize the weight parameters of each convolutional layer module after iterative training and the weight parameters without iterative training to obtain the processed weights, where CW_new' is the processed weight after iterative training and CW_o' is the processed weight without iterative training.
[0132] Calculate the distance L2 of each convolutional layer module, where L2 = | | CW_new'-CW_o'| |^2;
[0133] A difference comparison result is obtained according to the distance L2.
[0134] In one embodiment, the two low channel convolutional layers are respectively a low channel convolutional layer A and a low channel convolutional layer B;
[0135] The low channel convolution layer A is used to reduce the dimension of the input feature, and the low channel convolution layer B is used to restore the dimension of the processed feature to the same dimension as the original output feature.
[0136] In one embodiment, the dimension of W_delta is (C_out*C_in*k*k);
[0137] The dimension of the channel convolution layer A is (C_r*C_in*1*1), and the dimension of the low channel convolution layer B is (C_r*C_out*k*k), where C_r is much smaller than C_in and C_out.
[0138] In one embodiment, it further includes: an evaluation module.
[0139] The evaluation module is used to evaluate the performance of the fine-tuned model on the validation set and take the model weight with the best evaluation result as the model weight actually used in the subsequent test.
[0140] In this device, the convolutional layer modules with large differences in weight parameters of each convolutional layer module are identified as key layers. These key layers are considered to have insufficient coverage of the scene. Then, the weights of these key layers are fine-tuned by adding parallel Conv-LoRA layers to obtain an optimized face recognition model. This device can reduce training parameters and video memory usage, achieve efficient model fine-tuning, perform better in actual data scenarios, and can be easily migrated and applied in scenarios with different data distributions.
[0141] Embodiment 3
[0142] An embodiment of the present invention further provides a storage medium on which computer instructions are stored. When the instructions are executed by a processor, the optimization method of the face recognition model of any of the above embodiments is implemented.
[0143] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, random access memories (RAM), read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.
[0144] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.
[0145] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, an encoder, and a computer program stored in the memory and run on the encoder, wherein when the encoder executes the program, an optimization method for a face recognition model such as any one of the above-mentioned embodiments is implemented.
[0146] The above-mentioned computer device finds out the convolutional layer modules with large differences in weight parameters of each convolutional layer module, and defines them as key layers. These key layers are considered to have insufficient coverage scenarios, and then the weights of these key layers are fine-tuned by adding parallel Conv-LoRA layers, thereby obtaining an optimized face recognition model. The above-mentioned computer device can reduce training parameters and video memory usage, achieve efficient model fine-tuning, perform better in actual data scenarios, and can be easily migrated and applied in usage scenarios under different data distributions.
[0147] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0148] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.
Claims
1. A method for optimizing a face recognition model, characterized in that: include: Obtaining a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition; Selecting a key layer in a pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer; Adding a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the output of the key layer; Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer; Y_new=Y_o+Y_delta; In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change; Y_delta=W_delta*X: Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers; Obtain a face image dataset in an actual application scenario, and use the face image dataset as training data to train the Conv-LoRA layer.
2. The optimization method of the face recognition model according to claim 1, characterized in that: The process of selecting the key layers in the pre-trained convolutional neural network model includes: Use face image datasets from actual application scenarios for iterative training; Compare the weight parameters of each convolutional layer module after iterative training with the weight parameters without iterative training to obtain the difference comparison results; The first n convolutional layer modules with the largest differences in the difference comparison results are selected as the key layers.
3. The optimization method of the face recognition model according to claim 2, characterized in that: The process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes: Subtract the corresponding position parameters of the weight parameters after iterative training from the weight parameters without iterative training for each convolutional layer module to obtain the difference value of the weight parameters; According to the difference value of the weight parameter, the cumulative value of the parameter difference and the standard deviation of the parameter difference of each convolution layer module are counted; A difference comparison result is obtained according to the cumulative value of the parameter difference and the standard deviation of the parameter difference.
4. The optimization method of the face recognition model according to claim 2, characterized in that: The process of comparing the weight parameters of each convolutional layer module after iterative training with the weight parameters of the non-iterative training to obtain the difference comparison result includes: Normalize the weight parameters of each convolutional layer module after iterative training and the weight parameters without iterative training to obtain the processed weights, where CW_new' is the processed weight after iterative training and CW_o' is the processed weight without iterative training. Calculate the distance L2 of each convolutional layer module, where L2 = ||CW_new'-CW_o'||^2; A difference comparison result is obtained according to the distance L2.
5. The optimization method of the face recognition model according to claim 1, characterized in that: The two low channel convolution layers are respectively a low channel convolution layer A and a low channel convolution layer B; The low channel convolution layer A is used to reduce the dimension of the input feature, and the low channel convolution layer B is used to restore the dimension of the processed feature to the same dimension as the original output feature.
6. The optimization method of the face recognition model according to claim 5, characterized in that: The dimension of W_delta is (C_out*C_in*k*k); The dimension of the channel convolution layer A is (C_r*C_in*1*1), and the dimension of the low channel convolution layer B is (C_r*C_out*k*k), where C_r is much smaller than C_in and C_out.
7. The method for optimizing a face recognition model according to any one of claims 1 to 6, characterized in that: Also includes: Evaluate the performance of the fine-tuned model on the validation set, and take the model weight with the best evaluation result as the model weight actually used in the subsequent use.
8. An optimization device for a face recognition model, characterized in that: include: An acquisition module, used to acquire a pre-trained convolutional neural network model, wherein the pre-trained convolutional neural network model is a convolutional neural network model for face recognition; A selection module is used to select a key layer in a pre-trained convolutional neural network model, wherein the key layer is the first n convolutional layer modules with the largest difference in iterative training, and n is a preset positive integer; An insertion module is used to add a parallel Conv-LoRA layer at the position of the key layer of the pre-trained convolutional neural network model to fine-tune the output of the key layer; Wherein, in the key layer, the original model output Y_o of the key layer, the output Y_delta of the Conv-LoRA layer and the adjusted model output Y_new of the key layer; Y_new=Y_o+Y_delta; In the Conv-LoRA layer, X is the input of the key layer, and W_delta is the weight of the convolution kernel of the Conv-LoRA layer change; Y_delta = W_delta * X; Among them, the variation convolution W_delta is decomposed into two low-channel convolution layers; The training module is used to obtain a face image dataset in an actual application scenario, and use the face image dataset as training data to train the Conv-LoRA layer.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.