Image processing method, face recognition model training method, device and equipment

By pruning the network inputs of each layer of the Transformer model, using the importance information of image blocks to prune, the problem of high computing power consumption of the Transformer model in the field of face recognition is solved, and the efficiency of image processing is improved.

CN113901904BInactive Publication Date: 2024-12-03BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111157086.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-29
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The Transformer model consumes a lot of computing power in the fields of image processing and face recognition, and requires miniaturized compression processing to improve efficiency.

Method used

By cutting the face image to be processed into multiple image blocks, the importance information of each image block is determined, and the input of each layer of network is pruned according to the pruning rate of the preset visual conversion model is pruned to reduce non-important images block calculation, thereby reducing the computing power consumption of the model.

Benefits of technology

While not affecting the extraction of facial image features, the calculation amount of visual conversion model is reduced and the efficiency of image processing is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113901904B_ABST
    Figure CN113901904B_ABST
Patent Text Reader

Abstract

The present disclosure provides an image processing method, a face recognition model training method, a device and equipment, which relate to the field of artificial intelligence technology, specifically to the field of computer vision and deep learning technology, and can be applied to scenes such as image processing and image recognition. The specific implementation scheme is: obtaining a face image to be processed, and cutting the face image to be processed into multiple image blocks; determining the importance information of each image block in the face image to be processed; obtaining the pruning rate of a preset visual conversion model; inputting multiple image blocks into the visual conversion model, and pruning the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block, to obtain the output result of the visual conversion model; determining the feature vector of the face image to be processed according to the output result of the visual conversion model. The technical solution disclosed in the present disclosure can reduce the computing power consumption in the image processing process and improve the efficiency of image processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of computer vision and deep learning technology, and can be applied to scenes such as image processing and image recognition, and in particular to an image processing method, a face recognition model training method, a device and equipment. Background Art

[0002] Recently, the Vision Transformer (ViT) model has made great progress, and the Transformer model has achieved excellent results in various visual field competitions. However, compared with the convolutional neural network model, the Transformer model generally requires huge computing power for inference and deployment, and there is an urgent need to miniaturize and compress the Transformer model. Summary of the invention

[0003] The present invention provides an image processing method, a face recognition model training method, a device and equipment.

[0004] According to a first aspect of the present disclosure, there is provided an image processing method, comprising:

[0005] Acquire a face image to be processed, and cut the face image to be processed into multiple image blocks;

[0006] Determining importance information of each image block in the face image to be processed;

[0007] Get the pruning rate of the preset visual transformation model;

[0008] Inputting the multiple image blocks into the visual conversion model, and performing pruning processing on the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block, to obtain an output result of the visual conversion model;

[0009] According to the output result of the visual conversion model, a feature vector of the face image to be processed is determined.

[0010] According to a second aspect of the present disclosure, a method for training a face recognition model is provided, comprising:

[0011] Acquire a face image sample, and cut the face image sample into a plurality of image blocks;

[0012] Determining importance information of each image block in the face image sample;

[0013] Get the pruning rate of the visual transformation model;

[0014] Inputting the multiple image blocks into the visual conversion model, and performing pruning processing on the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block, to obtain an output result of the visual conversion model;

[0015] Determining a feature vector of the face image sample according to an output result of the visual conversion model, and obtaining a face recognition result according to the feature vector;

[0016] The visual conversion model is trained according to the face recognition result.

[0017] According to a third aspect of the present disclosure, there is provided an image processing apparatus, comprising:

[0018] A first acquisition module, used for acquiring a face image to be processed and cutting the face image to be processed into a plurality of image blocks;

[0019] A first determination module, used to determine the importance information of each image block in the face image to be processed;

[0020] The second acquisition module is used to obtain the pruning rate of the visual conversion model;

[0021] A pruning module, used to input the multiple image blocks into the visual conversion model, and perform pruning processing on the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block, so as to obtain an output result of the visual conversion model;

[0022] The second determination module is used to determine the feature vector of the face image to be processed according to the output result of the visual conversion model.

[0023] According to a fourth aspect of the present disclosure, a training device for a face recognition model is provided, comprising:

[0024] A first acquisition module, used for acquiring a face image sample and cutting the face image sample into a plurality of image blocks;

[0025] A first determination module, used to determine the importance information of each image block in the face image sample;

[0026] The second acquisition module is used to obtain the pruning rate of the visual conversion model;

[0027] A pruning module, used to input the multiple image blocks into the visual conversion model, and perform pruning processing on the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block, so as to obtain an output result of the visual conversion model;

[0028] A second determination module is used to determine a feature vector of the face image sample according to an output result of the visual conversion model, and obtain a face recognition result according to the feature vector;

[0029] A training module is used to train the visual conversion model according to the face recognition result.

[0030] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0031] at least one processor; and

[0032] a memory communicatively connected to the at least one processor; wherein,

[0033] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect above and / or execute the method described in the second aspect above.

[0034] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect above, and / or execute the method described in the second aspect above.

[0035] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the method described in the first aspect above, and / or implements the method described in the second aspect above.

[0036] According to the technical solution disclosed in the present invention, multiple image blocks of the face image to be processed are input into a preset visual transformation model, and the input of each layer of the network in the visual transformation model is pruned according to the pruning rate of the model and the important information of each image block. Therefore, by reducing the input features of each layer of the network in the visual transformation model, the computing power consumption of the visual transformation model can be reduced without affecting the feature extraction of the face image, thereby improving the efficiency of image processing.

[0037] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0039] Figure 1 is a schematic diagram of the structure of a visual conversion model according to an embodiment of the present disclosure;

[0040] Figure 2 is a flowchart of an image processing method provided according to an embodiment of the present disclosure;

[0041] Figure 3 is a flowchart of a pruning process for each layer of network input according to an embodiment of the present disclosure

[0042] Figure 4 is a flowchart of another pruning process for each layer of network input according to an embodiment of the present disclosure;

[0043] Figure 5 is another flowchart of pruning processing for each layer of network input provided according to an embodiment of the present disclosure;

[0044] Figure 6 is an example diagram of pruning processing for each layer of network input according to an embodiment of the present disclosure;

[0045] Figure 7 A method for training a face recognition model is provided according to an embodiment of the present disclosure;

[0046] Figure 8 is a structural block diagram of an image processing device provided according to an embodiment of the present disclosure;

[0047] Fig. 9 is a structural block diagram of another image processing device provided according to an embodiment of the present disclosure;

[0048] Fig.10 is a block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0050] In the technical solution disclosed in this disclosure, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs. The user personal information involved is acquired, stored and applied with the user's consent.

[0051] It should be noted that in some embodiments of the present disclosure, the visual transformation model refers to the VisionTransformer (ViT) model. The visual transformation model has been greatly developed recently, and the transformation model (Transformer) has achieved excellent results in various visual field competitions. However, compared with the convolutional neural network model, the Transformer model generally requires huge computing power for inference and deployment, and there is an urgent need to miniaturize and compress the Transformer model.

[0052] Among them, the structure of the visual conversion model is as follows Figure 1 As shown in the figure, in Transformer, an image is divided into multiple patches, each of which corresponds to an input position of the network. Multitransformerencoder (encoder in the multi-layer transformation model) stacks multiple layers of TransformerEncoder (encoder in the transformation model) modules, which have two norm modules, an MHA (MultiHeadAttention) module, and an MLP (Multilayer Perceptron) module.

[0053] At present, the relevant pruning technologies mainly prune the number of layers and heads of the visual transformation model. These solutions only prune some dimensions in the calculation process. In the calculation process, it can be seen that the number of image blocks also affects the amount of model calculation.

[0054] However, image block pruning has great limitations in common classification tasks. Objects in the image may appear at any position, so image block pruning may require special aggregation operations to converge the information transmission between layers. Although such operations increase the amount of calculation, they do not necessarily integrate and converge the information.

[0055] However, for face recognition models, before images are input into the model, they are detected, aligned, and other operations are performed to maximize accuracy. After these operations, face images have roughly the same structure, so the importance of face image blocks also has roughly the same order. Therefore, image blocks can be pruned according to their importance to reduce the model's calculations on non-important image blocks, thereby reducing the model's computing power consumption.

[0056] Based on the above problems and findings, the present disclosure provides an image processing method, which reduces the computational consumption in the image processing process by pruning the input of each layer of the network in the model.

[0057] Figure 2is a flowchart of an image processing method provided by an embodiment of the present disclosure. The image processing method is mainly used for processing face images, and the face recognition model in the processing process is trained. The face recognition model includes a visual conversion model, that is, the visual conversion model has also been trained. It should be noted that the image processing method in the embodiment of the present disclosure can be applied to the image processing device in the embodiment of the present disclosure, and the device can be configured in an electronic device. Figure 2 As shown, the image processing method may include the following steps:

[0058] Step 201: obtain a face image to be processed, and cut the face image to be processed into multiple image blocks.

[0059] It can be understood that in order to enable the model to fully extract the features of the facial image to be processed, the facial image to be processed can be cut into multiple image blocks, and the sizes of the multiple image blocks after cutting are equal, and the number of image blocks after cutting is consistent with the number of image block inputs of the preset visual transformation model.

[0060] Step 202: Determine the importance information of each image block in the face image to be processed.

[0061] It is understandable that not all image blocks in the face image to be processed contain important features of the face. Some image blocks may only be the background of the face image, which is not very useful for extracting facial features. Therefore, if the visual transformation model extracts features from each image block in the face to be processed through learning, a certain amount of computing power will be wasted on some unimportant image blocks.

[0062] At the same time, for the face recognition model, detection, alignment and other operations will be performed before the image is input into the model. Therefore, after these operations, each face image has roughly the same pattern, that is, the importance distribution of each image block in each face image is roughly the same. Therefore, statistical analysis based on a large number of face images can be performed to determine the importance information of each image block.

[0063] In some embodiments of the present disclosure, multiple face images can be acquired in advance, and each face image can be cut into a preset number of image blocks, and the feature information contained in each image block can be determined by using a trained face feature extraction model. The feature information of each image block in each image is combined. If the image block at position 1 of each face image contains a large amount of face feature information, and the image block at position 3 contains almost no face feature information, it can be determined that the importance of the image block at position 1 is higher than that of the image block at position 1, so that the importance information of each image block at different positions can be obtained, and the importance information can be applied to all face images, so the importance information of each image block in the face image to be processed is determined.

[0064] As an implementation method, since the attention matrix reflects the importance of the relationship between image blocks in the calculation process of the TransformerEncoder layer of the visual conversion model, the importance information between image blocks can be determined according to the attention matrix output by each layer of the network of the visual conversion model. Its implementation method may include: inputting multiple face image samples into the visual conversion model to obtain the attention matrix corresponding to each face image sample output by each layer of the network; merging all the obtained attention matrices to obtain the weight of each image block sample; and determining the importance information of each image block in the face image to be processed according to the weight of each image block sample. Among them, since the value in the attention matrix is ​​the result of softmax (maximum normalized activation function) processing, the softmax result is the importance probability of the image block, so the weight of each image block can be determined by merging the importance probability of the image blocks in multiple image samples. The merging method can be to add the attention matrix of each image sample according to the matrix axis, or to perform weighted summation according to the difference of each layer of the network in the actual application scenario, or other merging methods can be adopted according to actual needs.

[0065] Step 203: Obtain a pruning rate of a preset visual conversion model.

[0066] In some embodiments of the present disclosure, the pruning rate of the visual conversion model refers to the proportion of the amount of computation expected to be reduced in the multi-layer network calculation. It can be obtained through input in the interactive interface, or through parameter transmission through the interface, or according to the preset value of the actual application scenario, or in other ways according to the actual application scenario, and the present disclosure does not limit this.

[0067] Step 204, input the multiple image blocks into the visual conversion model, and prune the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block to obtain the output result of the visual conversion model.

[0068] It should be noted that the output result of the visual conversion model in the embodiment of the present disclosure is a node output in the face recognition model, and the output result serves as input information for subsequent nodes of the face recognition model.

[0069] That is to say, multiple image blocks in the face image to be processed are input into the visual transformation model, and the input of each layer of the network is pruned based on the pruning rate and the importance information of each image block in the face image to be processed. This can reduce the computational complexity of each layer of the network without affecting the feature extraction of the model.

[0070] As an example, the number of image blocks that can be pruned in the input of each layer of the network can be determined based on the pruning rate, and then according to the importance information of each image block, the image blocks with lower importance can be selected layer by layer as the image blocks to be pruned. In this way, the feature information of the image blocks to be pruned in the input of each layer of the network can be pruned, and the output result of the visual transformation model can be obtained.

[0071] As another example, multiple image blocks in the face image to be processed can be sorted according to the importance information of each image block, for example, the image blocks can be sorted in descending order of importance; based on the determined number of image blocks that can be cut off in the input of each layer of the network, the features of the corresponding number of image blocks sorted at the end of the input of each layer of the network are cropped, thereby achieving pruning of non-important image blocks so as not to affect the feature extraction of the face image to be processed by the visual transformation model.

[0072] It should be noted that, in some embodiments of the present disclosure, each network layer in the visual conversion model refers to each TransformerEncoder layer in the visual conversion model.

[0073] Step 205: Determine the feature vector of the face image to be processed according to the output result of the visual conversion model.

[0074] In some embodiments of the present disclosure, when multiple image blocks of a facial image to be processed are input into a visual transformation model, the visual transformation model can supplement a virtual image block, and the result of the virtual image block after passing through the Transformer Encoder layer is used as the overall information expression of the facial image to be processed, so the corresponding feature vector in the virtual image block can be used as the feature vector of the facial image to be processed in the output result of the visual transformation model. In addition, some visual transformation models do not supplement a virtual image block to learn the overall information of the facial image to be processed, so in this case, the output result of the visual transformation model can be directly used as the feature vector of the facial image to be processed.

[0075] According to the image processing method of the embodiment of the present disclosure, multiple image blocks of the face image to be processed are input into the visual transformation model, and the input of each layer of the network in the visual transformation model is pruned according to the pruning rate of the model and the important information of each image block. In this way, the efficiency of image processing can be improved by reducing the input features of each layer of the network in the visual transformation model without affecting the extraction of face image features.

[0076] Based on the above embodiments, the present disclosure proposes another embodiment for pruning the input of each layer of the network in the visual conversion model.

[0077] Figure 3is a flowchart of the pruning process of each layer of network input in the embodiment of the present disclosure. Figure 3 As shown, the pruning process includes the following steps:

[0078] Step 301, determining the number of image blocks to be pruned in each layer of the network according to the pruning rate.

[0079] Since the visual conversion model contains multiple layers of networks, in order to reduce the impact of pruning on feature extraction, pruning can be performed layer by layer, that is, pruning is performed step by step when the model runs layer by layer to avoid affecting the feature extraction of a certain layer of the network and subsequent networks due to excessive information clipping in the input of the network.

[0080] In some embodiments of the present disclosure, the number of image blocks pruned in each network layer refers to the number of image blocks that need to be pruned in each network layer based on the pruning rate. The number of image blocks pruned in each network layer can be calculated by the pruning rate, and the number of image blocks pruned in each layer can be the same or different, and can be determined according to the actual situation. As an example, the total number of image blocks pruned in the visual conversion model can be calculated based on the number of image blocks input to the visual conversion model and the pruning rate. If the number of image blocks input is 120, and the visual conversion model has a total of 10 layers of networks, it means that when no pruning is performed, each layer of the network input has the characteristics of 120 image blocks. If the pruning rate is 10%, the total number of image blocks pruned in the model = 120*10*10% = 120. In this way, the cumulative sum of the number of image blocks actually pruned in each network layer is 120 image blocks. If the number of pruned image blocks in the first layer is 2, and the number of pruned image blocks in the second layer is also 2, then the actual number of pruned image blocks in the second layer is 4, and so on, until the actual number of pruned image blocks in each layer of the model is 120, then the pruning rate is reached. It should be noted that the number of pruned image blocks in each layer of the network is the same, and different numbers of pruned image blocks can be set according to actual needs.

[0081] Step 302: Determine the image blocks to be pruned in each layer of the network among the multiple image blocks according to the importance information of each image block and the number of image blocks to be pruned in each layer of the network.

[0082] It can be understood that which image blocks can be pruned can be determined based on the importance information of each image block, so based on the number of pruned image blocks in each layer of the network, the image blocks to be pruned in each layer of the network can be determined.

[0083] As an example, if the number of input image blocks is 9, the number of image blocks pruned in each layer of the network is 1, and the importance information of each image block is the image block at position 3 < the image block at position 9 < the image block at position 2 < the image block at position 1 < the image block at position 4 < the image block at position 5 < the image block at position 6 < the image block at position 7 < the image block at position 8, then it can be determined that the image block to be pruned in the first layer of the network input is the image block at position 3, the image block to be pruned in the second layer of the network input is the image block at position 9, the image block to be pruned in the third layer of the network input is the image block at position 2, and so on. For ease of description, the form of "image block + label" will be used below to represent image blocks at different positions, such as image block 3 represents the image block at position 3.

[0084] Step 303, for the input features of each layer of the network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the current layer of the network.

[0085] That is to say, the input features of each layer of the network are first cropped, and then the cropped features are input into the corresponding layer of the network, so as to reduce the computational complexity of the visual conversion model by reducing the input of each layer of the network.

[0086] In some embodiments of the present disclosure, the input features of each layer of the network are equivalent to the output features of the previous layer of the network. For example, for the third layer of the network, the input features of the third layer of the network are equivalent to the output features of the second layer of the network. That is to say, in the embodiments of the present disclosure, before the input features of each layer of the network are input to the network, the input features are pruned first, and then the pruned features are input to the corresponding layer of the network.

[0087] For example, for the input features of the third layer network in the above example, the features corresponding to the image block at position 2 in the input features are first cropped, and then the cropped features are input into the third layer network.

[0088] According to the image processing method provided by the embodiment of the present disclosure, the number of image blocks to be pruned in each layer of the network is determined according to the pruning rate, and the image blocks to be pruned in each layer of the network are determined based on the importance information of each image block. Therefore, the features corresponding to the image blocks to be pruned can be cropped for the input features of each layer of the network and then input into the current layer of the network. In other words, the amount of computation of each layer of the network can be reduced by reducing the information input of non-essential image blocks to each layer of the network, thereby achieving the purpose of reducing the computing power of the visual conversion model under the premise of ensuring that no feature information is lost.

[0089] Based on the above embodiments, the present disclosure provides another embodiment for the pruning process of the input of each layer of the network.

[0090] Figure 4 FIG. 1 is another flowchart of pruning processing for each layer of network input in an embodiment of the present disclosure. Figure 4 As shown, the pruning process includes the following steps:

[0091] Step 401: sort the image blocks according to the importance information of the image blocks.

[0092] That is to say, according to the importance information of each image block, each image block is sorted according to its importance.

[0093] In some embodiments of the present disclosure, after the facial image to be processed is cut into multiple image blocks, each image block is arranged based on its position in the facial image to be processed. That is to say, cutting the facial image to be processed into multiple image blocks is equivalent to dividing the facial image to be processed into different multiple rows and different multiple columns, and each image block is arranged based on its position in the facial image to be processed, and can be arranged in the order of rows and columns, from top to bottom, and from left to right.

[0094] According to the importance information of each image block, reordering each image block is equivalent to disrupting the original position arrangement order, which can be to arrange the image blocks with higher importance in front, and so on, or to arrange the image blocks with higher importance in the back, and so on. As an example, there are 120 image blocks after cutting. If the image blocks after cutting are arranged as {image block 1, image block 2, image block 3, image block 4, ..., image block 120}; the importance information of each image block is image block 3 < image block 10 < image block 11 < image block 34 < image block 1 < image block 2 < image block 115 < image block 13 ... < image block 44 < image block 45 < image block 47; then according to the importance information of each image block, the result of sorting each image block according to importance can be: {image block 47, image block 45, image block 44, ..., image block 13, image block 115, image block 2, image block 1, image block 34, image block 11, image block 10, image block 3}.

[0095] Step 402: input each image block and the sorting result of each image block into a visual conversion model.

[0096] Step 403: Determine the number of image blocks to be pruned in each layer of the network according to the pruning rate.

[0097] Step 404, for the input features of each layer of the network, according to the sorting results of each image block, the features corresponding to the image blocks of the pruning number of the image blocks are pruned from the input features, and the features obtained after pruning are input to the current layer of the network.

[0098] That is to say, before the input features of each layer of the network are input into the network layer, the features corresponding to the image blocks of the pruning number can be pruned out from the input features according to the sorting results of each image block, and then the pruned features can be input into the corresponding layer of the network.

[0099] For example, based on the above example, the result after sorting the image blocks from high to low in importance is: {image block 47, image block 45, image block 44, …, image block 13, image block 115, image block 2, image block 1, image block 34, image block 11, image block 10, image block 3}. If the number of image block pruning in the first layer network is 1, the input feature before inputting the first layer network is the initial feature of {image block 47, image block 45, image block 44, …, image block 13, image block 115, image block 2, image block 1, image block 34, image block 11, image block 10, image block 3}; according to the sorting result, the features corresponding to the image blocks sorted at the end can be cropped, and the cropped features are {image block 47, image block 45, image block 44, …, FIG. The initial features of image block 13, image block 115, image block 2, image block 1, image block 34, image block 11, image block 10} are processed, and the processed features are input to the first layer network; if the number of image blocks pruned in the second layer network is 3, then the first feature corresponding to the input feature {image block 47, image block 45, image block 44, …, image block 13, image block 115, image block 2, image block 1, image block 34, image block 11, image block 10} before being input to the second layer network refers to the feature output by the first layer network after learning and calculation; the cropped feature is the first feature corresponding to {image block 47, image block 45, image block 44, …, image block 13, image block 115, image block 2, image block 1}, and the cropped feature is input to the second layer network.

[0100] According to the image processing method of the embodiment of the present disclosure, the image blocks of the face image to be processed are first sorted according to the importance information of each image block, and the features corresponding to the corresponding number of image blocks are cropped in each layer of input features according to the sorting, and then the features obtained after cropping are input to the corresponding network layer. Therefore, during cropping, the features of the first few image blocks or the features of the last few image blocks can be directly cropped based on the sorting, thereby further reducing the computational complexity of the cropping process, improving the cropping efficiency, and further improving the efficiency of image processing.

[0101] In order to further avoid the impact of cropping of each layer of network input on the feature extraction of facial images, the present disclosure provides another embodiment for this purpose.

[0102] Figure 5, which is another flowchart of pruning processing for each layer of network input provided by the embodiment of the present disclosure. For ease of description, in the embodiment of the present disclosure, N is used to represent the number of network layers in the visual conversion model, where N is an integer greater than 1. Figure 5 As shown, the pruning process includes the following steps:

[0103] Step 501, determining the number of image blocks to be pruned in the i-th layer network according to the pruning rate; wherein i is an integer greater than 0 and less than or equal to N-1.

[0104] That is to say, in the embodiment of the present disclosure, the number of image blocks to be pruned in the first N-1 layers of the network is determined only according to the pruning rate, and the input of the Nth layer of the network is not pruned.

[0105] Step 502: Determine the image blocks to be pruned in the i-th layer of the network among the multiple image blocks according to the importance information of each image block and the number of pruned image blocks in the i-th layer of the network.

[0106] Step 503: for the input features of the i-th layer network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the i-th layer network.

[0107] The implementation method of the cropping process for the first N-1 layer network input in step 502 and step 503 is the same as Figure 3 The implementation method of the pruning process for the first N-1 layer network input in step 302 is consistent with that in step 303, and will not be repeated here.

[0108] Step 504, for the input features of the Nth layer network, the input features are spliced ​​with the features of all the cropped image blocks, and the features obtained after the splicing process are input to the Nth layer network.

[0109] That is to say, the output features of the N-1th layer network are concatenated with the features of all the cropped image blocks in the input of the first N-1th layer network, and the features are input into the Nth layer network. This can not only reduce the computing power of the first N-1th layer network, but also further reduce the impact of cropping on the feature extraction of the processed face image.

[0110] For ease of understanding, the implementation of the embodiments of the present disclosure can be as follows: Figure 6 As shown in the figure, if the visual conversion model has a total of 6 layers of networks, and the features corresponding to an image block are cut out in each layer of the input of the first 5 layers of networks, then the input of the 6th layer of the network is the features obtained by concatenating the output features of the 5th layer of the network and the features corresponding to the image blocks cut out by the first 5 layers of the network. In other words, during the operation of the visual conversion model, the features corresponding to the image blocks cut out during each pruning process need to be stored, and when the operation reaches the last layer, the features of the cut image blocks are called.

[0111] It can be understood that the input of the Nth layer network is equivalent to integrating all the features of the face image to be processed, thereby ensuring that the features of the face image are not lost while reducing the amount of calculation.

[0112] According to the image processing method of the embodiment of the present disclosure, for the visual conversion model of the N-layer network, the inputs of the first N-1 layers of the network are pruned respectively, and the output features of the N-1th layer of the network are spliced ​​with the features corresponding to the image blocks cropped in the input of the first N-1 layers of the network, and the spliced ​​features are input into the Nth layer of the network. On the one hand, the influence of the pruning process on the feature extraction of the face image can be further reduced. On the other hand, the computational amount of the model can be reduced by pruning the first N-1 layers of the network, thereby further improving the effect of the pruning process on the image processing.

[0113] The disclosed embodiment also provides a method for training a face recognition model.

[0114] Figure 7 A training method for a face recognition model provided in an embodiment of the present disclosure includes a visual conversion model. It should be noted that the training method for a face recognition model in an embodiment of the present disclosure can be applied to a training device for a face recognition model in an embodiment of the present disclosure, and the device can be configured in an electronic device. Figure 7 As shown, the training method may include the following steps:

[0115] Step 701: obtain a face image sample and cut the face image sample into multiple image blocks.

[0116] It can be understood that in order to enable the model to fully extract the features of the face image samples to be processed, each face image in the face image samples can be cut into multiple image blocks, and the sizes of the multiple image blocks after cutting are equal, and the number of image blocks after cutting is consistent with the number of image block inputs of the visual transformation model.

[0117] Step 702: Determine the importance information of each image block in the face image sample.

[0118] It is understandable that not all image blocks in every face image contain important features of the face. Some image blocks may only be the background of the face image, which is not very useful for extracting face features. Therefore, if the visual transformation model extracts features from each image block in the face image sample through learning, a certain amount of computing power will be wasted on some unimportant image blocks.

[0119] At the same time, for the face recognition model, detection, alignment and other operations will be performed before the image is input into the model. Therefore, after these operations, each face image has roughly the same pattern, that is, the importance distribution of each image block in each face image is roughly the same. Therefore, statistical analysis based on a large number of face images can be performed to determine the importance information of each image block.

[0120] In some embodiments of the present disclosure, multiple face images can be acquired in advance, and each face image can be cut into a preset number of image blocks, and the feature information contained in each image block can be determined by a trained visual conversion model. The feature information of each image block in each image is combined. If the image block at position 1 of each face image contains a large amount of face feature information, and the image block at position 3 contains almost no face feature information, it can be determined that the importance of the image block at position 1 is higher than the image block at position 1, so that the importance information of each image block at different positions can be obtained, and the importance information can be applied to all face images, so the importance information of each image block in the face image to be processed is determined.

[0121] As an implementation method, since the attention matrix reflects the importance of the relationship between image blocks during the calculation process of the TransformerEncoder layer of the visual conversion model, the importance information between image blocks can be determined according to the attention matrix output by each layer of the network of the visual conversion model. Its implementation method may include: inputting multiple face images into the visual conversion model to obtain the attention matrix corresponding to each face image output by each layer of the network; merging all the obtained attention matrices to obtain the weights of the image blocks in the face image; and determining the importance information of each image block in each face image of the face image sample according to the weights of the image blocks in the face image. Among them, since the value in the attention matrix is ​​the result after softmax, the softmax result is the importance probability of the image block, so the weight of each image block can be determined by merging the importance probabilities of the image blocks in multiple image samples. The merging method may be to add the attention matrix of each image sample according to the matrix axis, or to perform weighted summation according to the difference of each layer of the network in the actual application scenario, or to adopt other merging methods according to actual needs.

[0122] Step 703: Obtain the pruning rate of the visual conversion model.

[0123] In some embodiments of the present disclosure, the pruning rate of the visual conversion model refers to the proportion of the amount of computation expected to be reduced in the multi-layer network calculation. It can be obtained through input in the interactive interface, or through parameter transmission through the interface, or according to the preset value of the actual application scenario, or in other ways according to the actual application scenario, and the present disclosure does not limit this.

[0124] Step 704: input multiple image blocks into the visual conversion model, and prune the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block to obtain the output result of the visual conversion model.

[0125] It should be noted that the output result of the visual conversion model in the embodiment of the present disclosure is a node output in the face recognition model, and the output result is used as input information for subsequent nodes of the face recognition model. Among them, the face recognition model is a model that has been trained by a relevant training method, that is, the visual conversion model is also a model that has been trained by a relevant training method.

[0126] In order to reduce the amount of calculation when applying the face recognition model and ensure the accuracy of the model after pruning, the face recognition model training method in the embodiment of the present disclosure is equivalent to a fine-tuning process based on the input pruning processing of each network layer.

[0127] As an implementation method, the implementation method for pruning the input of each layer of the network in the visual transformation model can be: according to the pruning rate, determine the number of image blocks to be pruned in each layer of the network; according to the importance information of each image block and the number of image blocks to be pruned in each layer of the network, determine the image blocks to be pruned in each layer of the network among multiple image blocks; for the input features of each layer of the network, crop the features of the image blocks to be pruned in the input features, and input the features obtained after cropping into the current layer of the network.

[0128] As another implementation method, the implementation method for pruning the input of each layer of the network in the visual transformation model can be: sorting each image block according to the importance information of each image block; inputting each image block and the sorting result of each image block into the visual transformation model; determining the number of image blocks to be pruned in each layer of the network according to the pruning rate; for the input features of each layer of the network, according to the sorting results of each image block, pruned the features corresponding to the image blocks of the pruned number of image blocks from the input features, and inputting the pruned features into the current layer of the network.

[0129] As another implementation, for ease of description, N is used to represent the number of network layers in the visual conversion model. The implementation method of the pruning process for each layer of network input can be: according to the pruning rate, determine the number of image blocks pruned in the i-th layer of the network; wherein, i is an integer greater than 0 and less than or equal to N-1; according to the importance information of each image block and the number of image blocks pruned in the i-th layer of the network, determine the image blocks to be pruned in the i-th layer of the network among multiple image blocks; for the input features of the i-th layer of the network, crop the features of the image blocks to be pruned in the input features, and input the cropped features to the i-th layer of the network; for the input features of the N-th layer of the network, splice the input features with the features of all cropped image blocks, and input the spliced ​​features to the N-th layer of the network.

[0130] Based on the above pruning process, the output result of the last layer of the network in the visual conversion model is the output result of the visual conversion model.

[0131] Step 705, determining a feature vector of the face image sample according to the output result of the visual conversion model, and obtaining a face recognition result according to the feature vector.

[0132] In some embodiments of the present disclosure, when multiple image blocks of a face image sample are input into a visual transformation model, the visual transformation model can supplement a virtual image block, and the result of the virtual image block after passing through the Transformer Encoder layer is used as the overall information expression of the corresponding image in the face image sample, so the corresponding feature vector in the virtual image block can be used as the feature vector of the face image sample in the output result of the visual transformation model. In addition, some visual transformation models do not supplement a virtual image block to learn the overall information of the corresponding image in the face image sample, so in this case, the output result of the visual transformation model can be directly used as the feature vector of the face image sample.

[0133] Previously, since the feature vector of the face image sample obtained by the visual transformation model was equivalent to a node in the face recognition process, the feature vector would continue to be learned by subsequent nodes in the face recognition model to obtain the face recognition result corresponding to the face image sample based on the feature vector.

[0134] Step 706: training a face recognition model according to the face recognition result.

[0135] That is to say, the corresponding loss value is calculated based on the face recognition result and the actual result of the face image sample, and then the parameters of the face recognition model are fine-tuned according to the loss value so that the model parameters can be suitable for the corresponding pruning method.

[0136] It should be noted that the detailed introduction of the pruning process of each layer network of the visual conversion model in the embodiment of the present disclosure has been presented in the embodiment of the above-mentioned image processing method, and will not be repeated here.

[0137] According to the training method of the face recognition model provided by the embodiment of the present disclosure, multiple image blocks of face image samples are input into the visual conversion model, and according to the pruning rate of the model and the important information of each image block, the input of each layer of the network in the visual conversion model is pruned, and the face recognition result is determined based on the feature vector obtained from the visual conversion model after the pruning process, so that the visual conversion model can be trained according to the face recognition result, that is, the face recognition model can be trained according to the face recognition result, so that the parameters of the model can be suitable for the pruning method, and the face recognition model using the visual conversion model can save computing power consumption when in use, thereby improving the efficiency of face recognition.

[0138] In order to implement the above-mentioned embodiments, the present disclosure provides an image processing apparatus.

[0139] Figure 8 is a structural block diagram of an image processing device provided by an embodiment of the present disclosure. Figure 8 As shown, the device comprises:

[0140] The first acquisition module 801 is used to acquire a face image to be processed and cut the face image to be processed into multiple image blocks;

[0141] A first determination module 802 is used to determine the importance information of each image block in the face image to be processed;

[0142] The second acquisition module 803 is used to obtain the pruning rate of the visual conversion model;

[0143] A pruning module 804 is used to input multiple image blocks into the visual conversion model, and prune the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block to obtain the output result of the visual conversion model;

[0144] The second determination module 805 is used to determine the feature vector of the face image to be processed according to the output result of the visual conversion model.

[0145] The first determining module 802 is specifically used for:

[0146] Input multiple face image samples into the visual conversion model to obtain the attention matrix corresponding to each face image sample output by each layer of the network;

[0147] Merge all the attention matrices obtained to obtain the weight of each image block sample;

[0148] According to the weight of each image block sample, the importance information of each image block in the face image to be processed is determined.

[0149] In some embodiments of the present disclosure, the pruning module 804 is specifically used to:

[0150] According to the pruning rate, determine the number of image blocks to be pruned in each layer of the network;

[0151] Determine the image blocks to be pruned in each layer of the network among the multiple image blocks according to the importance information of each image block and the number of image blocks pruned in each layer of the network;

[0152] For the input features of each layer of the network, the features of the image blocks to be pruned in the input features are cropped, and the cropped features are input into the current layer of the network.

[0153] In some other embodiments of the present disclosure, the pruning module 804 is specifically used to:

[0154] Sort each image block according to the importance information of each image block;

[0155] Inputting each image block and the sorting result of each image block into the visual transformation model;

[0156] According to the pruning rate, determine the number of image blocks to be pruned in each layer of the network;

[0157] For the input features of each layer of the network, according to the sorting results of each image block, the features corresponding to the image blocks of the same number as the pruning number of image blocks are pruned from the input features, and the pruned features are input into the current layer of the network.

[0158] In some other embodiments of the present disclosure, the visual conversion model includes an N-layer network, where N is an integer greater than 1; the pruning module 804 is specifically used to:

[0159] According to the pruning rate, determine the number of image blocks to be pruned in the i-th layer network; where i is an integer greater than 0 and less than or equal to N-1;

[0160] Determine an image block to be pruned in the i-th layer of the network among the multiple image blocks according to the importance information of each image block and the number of image blocks pruned in the i-th layer of the network;

[0161] For the input features of the i-th layer network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the i-th layer network;

[0162] For the input features of the N-th layer network, the input features are spliced ​​with the features of all the cropped image blocks, and the features obtained after the splicing process are input into the N-th layer network.

[0163] According to the image processing device provided by the embodiment of the present disclosure, multiple image blocks of the face image to be processed are input into the visual transformation model, and the input of each layer of the network in the visual transformation model is pruned according to the pruning rate of the model and the important information of each image block. Therefore, by reducing the input features of each layer of the network in the visual transformation model, the computing power consumption of the visual transformation model can be reduced without affecting the feature extraction of the face image, thereby improving the efficiency of image processing.

[0164] In order to implement the above embodiments, the present disclosure provides a training device for a face recognition model.

[0165] Fig. 9 is a structural block diagram of a training device for a face recognition model provided by an embodiment of the present disclosure. The face recognition model in the embodiment of the present disclosure includes a visual conversion model. Fig. 9 As shown, the device comprises:

[0166] A first acquisition module 901 is used to acquire a face image sample and cut the face image sample into a plurality of image blocks;

[0167] A first determination module 902, used to determine the importance information of each image block in the face image sample;

[0168] A second acquisition module 903 is used to acquire a pruning rate of a visual conversion model;

[0169] A pruning module 904 is used to input multiple image blocks into the visual conversion model, and prune the input of each layer of the network in the visual conversion model according to the pruning rate and the importance information of each image block to obtain the output result of the visual conversion model;

[0170] A second determination module 905 is used to determine a feature vector of the face image sample according to the output result of the visual conversion model, and obtain a face recognition result according to the feature vector;

[0171] The training module 906 is used to train the face recognition model according to the face recognition result.

[0172] The first determining module 902 is specifically used for:

[0173] Input multiple face images into the visual conversion model to obtain the attention matrix corresponding to each face image output by each layer of the network;

[0174] Merge all the attention matrices obtained to obtain the weights of the image blocks in the face image;

[0175] According to the weights of the image blocks in the face images, importance information of each image block in each face image of the face image samples is determined.

[0176] In some embodiments of the present disclosure, the pruning module 904 is specifically used to:

[0177] According to the pruning rate, determine the number of image blocks to be pruned in each layer of the network;

[0178] According to the importance information of each image block and the number of image blocks pruned in each layer of the network, determine the image blocks to be pruned in each layer of the network among the multiple image blocks in each face image sample;

[0179] For the input features of each layer of the network, the features of the image blocks to be pruned in the input features are cropped, and the cropped features are input into the current layer of the network.

[0180] In some other embodiments of the present disclosure, the pruning module 904 is specifically used to:

[0181] Sort each image block according to the importance information of each image block;

[0182] Inputting each image block and the sorting result of each image block into the visual transformation model;

[0183] According to the pruning rate, determine the number of image blocks to be pruned in each layer of the network;

[0184] For the input features of each layer of the network, according to the sorting results of each image block, the features corresponding to the image blocks of the same number as the pruning number of image blocks are pruned from the input features, and the pruned features are input into the current layer of the network.

[0185] In some other embodiments of the present disclosure, the visual conversion model includes an N-layer network, where N is an integer greater than 1; the pruning module 904 is specifically used to:

[0186] According to the pruning rate, determine the number of image blocks to be pruned in the i-th layer network; where i is an integer greater than 0 and less than or equal to N-1;

[0187] Determine an image block to be pruned in the i-th layer of the network among the multiple image blocks according to the importance information of each image block and the number of image blocks pruned in the i-th layer of the network;

[0188] For the input features of the i-th layer network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the i-th layer network;

[0189] For the input features of the N-th layer network, the input features are spliced ​​with the features of all the cropped image blocks, and the features obtained after the splicing process are input into the N-th layer network.

[0190] According to the training device of the face recognition model of the embodiment of the present disclosure, multiple image blocks of face image samples are input into the visual conversion model, and the input of each layer of the network in the visual conversion model is pruned according to the pruning rate of the model and the important information of each image block, and the face recognition result is determined based on the feature vector obtained from the visual conversion model after the pruning process, so that the visual conversion model can be trained according to the face recognition result, and then the face recognition model can be trained according to the face recognition result, so that the parameters of the model can be suitable for the pruning method, and the face recognition model using the visual conversion model can save computing power consumption when in use, thereby improving the efficiency of face recognition.

[0191] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0192] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0193] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0194] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0195] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as image processing methods, and / or training methods for face recognition models. For example, in some embodiments, the image processing method, and / or the training method for the face recognition model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the image processing method described above and / or the training method for the face recognition model may be executed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute an image processing method and / or a training method for a face recognition model in any other suitable manner (for example, by means of firmware).

[0196] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0198] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0199] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0200] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0201] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0202] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0203] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, include: Acquire a face image to be processed, and cut the face image to be processed into a plurality of image blocks, wherein the face image to be processed includes a face image after an alignment operation, the plurality of image blocks are of equal size, and the number of the plurality of image blocks is consistent with the number of image block inputs of a preset visual conversion model; Determining importance information of each image block in the face image to be processed; Get the pruning rate of the preset visual transformation model; Inputting the multiple image blocks into the visual conversion model, determining the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, performing pruning processing on the input of each layer of the network in the visual conversion model, and obtaining the output result of the visual conversion model; According to the output result of the visual conversion model, a feature vector of the face image to be processed is determined.

2. The method according to claim 1, in, The determining the importance information of each image block in the face image to be processed includes: Inputting multiple face image samples into the visual conversion model to obtain an attention matrix corresponding to each face image sample output by each layer of the network; Merge all the attention matrices obtained to obtain the weight of each image block sample; According to the weight of each image block sample, the importance information of each image block in the face image to be processed is determined.

3. The method according to claim 1, in, The step of determining the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, and performing pruning processing on the input of each layer of the network in the visual conversion model includes: Determining the number of image blocks to be pruned in each layer of the network according to the pruning rate; Determine, according to the importance information of each image block and the number of image blocks pruned in each layer of the network, image blocks to be pruned in each layer of the network among the multiple image blocks; For the input features of each layer of the network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the current layer of the network.

4. The method according to claim 1, in, The step of inputting the plurality of image blocks into the visual conversion model, determining the image blocks to be pruned in each layer of the network among the plurality of image blocks according to the pruning rate and the importance information of each image block, and performing pruning processing on the input of each layer of the network in the visual conversion model includes: sorting the image blocks according to the importance information of the image blocks; Inputting the image blocks and the sorting results of the image blocks into the visual conversion model; Determining the number of image blocks to be pruned in each layer of the network according to the pruning rate; For the input features of each layer of the network, according to the sorting results of the image blocks, the features corresponding to the image blocks of the image block pruning number are pruned from the input features, and the pruned features are input to the current layer of the network.

5. The method according to claim 1, in, The visual conversion model includes an N-layer network, where N is an integer greater than 1; determining the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, and performing pruning processing on the input of each layer of the network in the visual conversion model, including: Determine the number of image blocks to be pruned in the i-th layer network according to the pruning rate; wherein i is an integer greater than 0 and less than or equal to N-1; Determine, according to the importance information of each image block and the number of image blocks pruned in the i-th layer network, an image block to be pruned in the i-th layer network among the multiple image blocks; For the input features of the i-th layer network, the features of the image block to be pruned in the input features are cropped, and the features obtained after the cropping are input into the i-th layer network; For the input features of the Nth layer network, the input features are spliced ​​with the features of all cropped image blocks, and the features obtained after the splicing process are input into the Nth layer network.

6. A method for training a face recognition model, wherein the face recognition model includes a visual conversion model. include: Acquire a face image sample, and cut the face image sample into a plurality of image blocks, wherein the face image sample includes a face image after an alignment operation, the plurality of image blocks are equal in size, and the number of the plurality of image blocks is consistent with the number of image block inputs of a preset visual conversion model; Determining importance information of each image block in the face image sample; Obtaining a pruning rate of the visual transformation model; Inputting the multiple image blocks into the visual conversion model, determining the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, performing pruning processing on the input of each layer of the network in the visual conversion model, and obtaining the output result of the visual conversion model; Determining a feature vector of the face image sample according to an output result of the visual conversion model, and obtaining a face recognition result according to the feature vector; The face recognition model is trained according to the face recognition result.

7. An image processing device, include: A first acquisition module is used to acquire a face image to be processed and cut the face image to be processed into a plurality of image blocks, wherein the face image to be processed includes a face image after an alignment operation, the plurality of image blocks are of equal size, and the number of the plurality of image blocks is consistent with the number of image block inputs of a preset visual conversion model; A first determination module, used to determine the importance information of each image block in the face image to be processed; The second acquisition module is used to obtain the pruning rate of the visual conversion model; A pruning module, used to input the multiple image blocks into the visual conversion model, and determine the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, and perform pruning processing on the input of each layer of the network in the visual conversion model to obtain the output result of the visual conversion model; The second determination module is used to determine the feature vector of the face image to be processed according to the output result of the visual conversion model.

8. The device according to claim 7, in, The first determining module is specifically used for: Inputting multiple face image samples into the visual conversion model to obtain an attention matrix corresponding to each face image sample output by each layer of the network; Merge all the attention matrices obtained to obtain the weight of each image block sample; According to the weight of each image block sample, the importance information of each image block in the face image to be processed is determined.

9. The device according to claim 7, in, The pruning module is specifically used for: Determining the number of image blocks to be pruned in each layer of the network according to the pruning rate; Determine, according to the importance information of each image block and the number of image blocks pruned in each layer of the network, image blocks to be pruned in each layer of the network among the multiple image blocks; For the input features of each layer of the network, the features of the image blocks to be pruned in the input features are cropped, and the features obtained after cropping are input to the current layer of the network.

10. The device according to claim 7, in, The pruning module is specifically used for: sorting the image blocks according to the importance information of the image blocks; Inputting the image blocks and the sorting results of the image blocks into the visual conversion model; Determining the number of image blocks to be pruned in each layer of the network according to the pruning rate; For the input features of each layer of the network, according to the sorting results of the image blocks, the features corresponding to the image blocks of the image block pruning number are pruned from the input features, and the pruned features are input to the current layer of the network.

11. The device according to claim 7, in, The visual conversion model includes an N-layer network, where N is an integer greater than 1; and the pruning module is specifically used for: Determine the number of image blocks to be pruned in the i-th layer network according to the pruning rate; wherein i is an integer greater than 0 and less than or equal to N-1; Determine, according to the importance information of each image block and the number of image blocks pruned in the i-th layer network, an image block to be pruned in the i-th layer network among the multiple image blocks; For the input features of the i-th layer network, the features of the image block to be pruned in the input features are cropped, and the features obtained after the cropping are input into the i-th layer network; For the input features of the N-th layer network, the input features are spliced ​​with the features of all cropped image blocks, and the features obtained after the splicing process are input into the N-th layer network.

12. A training device for a face recognition model, the face recognition model comprising a visual conversion model include: A first acquisition module is used to acquire a face image sample and cut the face image sample into a plurality of image blocks, wherein the face image sample includes a face image after an alignment operation, the plurality of image blocks are of equal size, and the number of the plurality of image blocks is consistent with the number of image block inputs of a preset visual conversion model; A first determination module, used to determine the importance information of each image block in the face image sample; A second acquisition module, used to acquire the pruning rate of the visual conversion model; A pruning module, used to input the multiple image blocks into the visual conversion model, and determine the image blocks to be pruned in each layer of the network among the multiple image blocks according to the pruning rate and the importance information of each image block, and perform pruning processing on the input of each layer of the network in the visual conversion model to obtain the output result of the visual conversion model; A second determination module is used to determine a feature vector of the face image sample according to an output result of the visual conversion model, and obtain a face recognition result according to the feature vector; A training module is used to train the face recognition model according to the face recognition result.

13. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1 to 5 and / or the method of claim 6.

14. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 5, and / or to execute the method according to claim 6.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5 and / or the method according to claim 6.