Model training and image processing method, device, electronic device and storage medium

Abstract

The present disclosure relates to a model training and image processing method, device, electronic device and storage medium, belonging to the field of image processing technology. The model training method includes: obtaining a first facial image set; performing cascade processing on the first facial image set to obtain a second facial image set; the cascade processing includes a first process and a second process performed successively; inputting the first facial image set into a target model to be trained for processing to obtain a first output facial image set; using the error between the facial image in the first output facial image set and the corresponding facial image in the second facial image set, adjusting the model parameters of the target model to be trained to train the target model. The model training and image processing method, device, electronic device and storage medium provided by the present disclosure can at least solve the technical problem in the related art that texture superposition may occur in the image when the neural network model is used to generate texture.

Description

Technical Field

[0001] The present disclosure relates to the field of neural network technology, and in particular to a method, device, electronic device and storage medium for model training and image processing. Background Art

[0002] With the development of neural network technology, the scheme of image processing using neural network models has been more and more widely used. In some application scenarios, the image processing implemented by the neural network model may include facial feature texture generation at the target area of ​​the facial image.

[0003] However, there may be existing facial feature textures in the target area of ​​the facial image. In this case, the facial image processed by the neural network model may have a superposition of the generated facial feature texture and the existing facial feature texture in the target area, resulting in the quality of the processed facial image failing to reach the expected level. Summary of the invention

[0004] The present disclosure provides a method, device, electronic device and storage medium for model training and image processing, so as to at least solve the technical problem in the related art that the image quality cannot reach the expected level when the facial feature texture is generated by using a neural network model.

[0005] According to one aspect of an embodiment of the present disclosure, a model training method is provided, including:

[0006] Acquire a first facial image set;

[0007] Performing cascade processing on the first facial image set to obtain a second facial image set; wherein the cascade processing includes a first process and a second process performed successively, the first process is used to remove existing facial feature textures in a target area of ​​facial images in the first facial image set, and the second process is used to generate the facial feature textures in the target area of ​​facial images in the pre-processed facial image set obtained after the first process;

[0008] Inputting the first facial image set into a target model to be trained for processing to obtain a first output facial image set;

[0009] The model parameters of the target model to be trained are adjusted by using the errors between the facial images in the first output facial image set and the corresponding facial images in the second facial image set to train the target model.

[0010] In a possible implementation manner, performing cascade processing on the first facial image set to obtain the second facial image set includes:

[0011] Using a first model to perform the first processing on the first facial image set to obtain a preprocessed facial image set;

[0012] The second processing is performed on the pre-processed facial image set using a second model cascaded with the first model to obtain the second facial image set.

[0013] In a possible implementation manner, the model training method further includes:

[0014] Acquire a third facial image set;

[0015] Inputting the third facial image set into the second model for processing to generate the facial feature texture in the target area of ​​each facial image in the third facial image set to obtain a second output facial image set;

[0016] adjusting model parameters of the second model and training the second model using errors between facial images in the second output facial image set and corresponding facial images in a pre-configured target facial image set;

[0017] The number of facial images in the third facial image set and the target facial image set is a first number, the number of facial images in the first facial image set and the second facial image set is a second number, and the second number is greater than the first number.

[0018] In one possible implementation, the training of the second model is completed before the cascade processing of the first facial image set; or,

[0019] The training of the second model is synchronized with the training of the target model; or

[0020] The training of the second model and the training of the target model are completed alternately in stages.

[0021] In a possible implementation manner, the network scale of the target model is smaller than the network scale of the second model.

[0022] In a possible implementation manner, the model training method further includes:

[0023] Inputting part or all of the facial images in the target facial image set into the first model for processing, so as to remove existing facial feature textures in the target area of ​​each facial image in the target facial image set, and obtain a third output facial image set;

[0024] The model parameters of the first model are adjusted by using errors between facial images in the third output facial image set and corresponding partial or all facial images in the third facial image set to train the first model.

[0025] In one possible implementation, the training of the first model is completed before the cascade processing of the first facial image set; or,

[0026] The training of the first model is synchronized with the training of the target model; or

[0027] The training of the first model and the training of the target model are completed alternately in stages.

[0028] In one possible implementation, at least one of the first model, the second model, and the target model is a residual network model.

[0029] In a possible implementation manner, the target area is an eyelid area.

[0030] According to another aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0031] The target model trained by any of the above-mentioned model training methods is used to process the acquired facial image to obtain a target image.

[0032] According to another aspect of an embodiment of the present disclosure, there is provided a model training device, comprising: an interface unit, a cascade processing unit, and a target model training unit;

[0033] The interface unit is used to obtain a first facial image set;

[0034] The cascade processing unit is used to perform cascade processing on the first facial image set to obtain a second facial image set; wherein the cascade processing includes a first process and a second process that are performed successively, the first process is used to remove the existing facial feature textures in the target area of ​​the facial images in the first facial image set, and the second process is used to generate the facial feature textures in the target area of ​​the facial images in the pre-processed facial image set obtained after the first process;

[0035] The target model training unit is used to input the first facial image set into the target model to be trained for processing to obtain a first output facial image set; and use the error between the facial images in the first output facial image set and the corresponding facial images in the second facial image set to adjust the model parameters of the target model to be trained to train the target model.

[0036] In one possible implementation, the cascade processing unit includes: a first model, used to perform the first processing on the first facial image set to obtain the pre-processed facial image set; and a second model, used to perform the second processing on the pre-processed facial image set to obtain the second facial image set.

[0037] In a possible implementation manner, the model training device further includes:

[0038] The second model training unit is used to obtain a third facial image set; input the third facial image set into the second model for processing to generate the facial feature texture in the target area of ​​each facial image in the third facial image set to obtain a second output facial image set; and adjust the model parameters of the second model by using the error between the facial image in the second output facial image set and the corresponding facial image in the pre-configured target facial image set to train the second model;

[0039] The number of facial images in the third facial image set and the target facial image set is a first number, the number of facial images in the first facial image set and the second facial image set is a second number, and the second number is greater than the first number.

[0040] In one possible implementation, the training of the second model by the second model training unit is completed before the cascade processing of the first facial image set; or, the training of the second model by the second model training unit is synchronized with the training of the target model; or, the training of the second model by the second model training unit and the training of the target model are completed alternately in stages.

[0041] In a possible implementation manner, the network scale of the target model is smaller than the network scale of the second model.

[0042] In a possible implementation manner, the model training device further includes:

[0043] a first model training unit, configured to input part or all of the facial images in the target facial image set into the first model for processing, so as to remove existing facial feature textures in the target area of ​​each facial image in the target facial image set, and obtain a third output facial image set; and adjust model parameters of the first model by using errors between facial images in the third output facial image set and corresponding part or all of the facial images in the third facial image set, so as to train the first model;

[0044] The model parameters of the first model are adjusted based on the errors between the facial images in the third output facial image set and the corresponding part or all of the facial images in the third facial image set to train the first model.

[0045] In one possible implementation, the training of the first model by the first model training unit is completed before the cascade processing of the first facial image set; or, the training of the first model by the first model training unit is synchronized with the training of the target model; or, the training of the first model by the first model training unit and the training of the target model are completed alternately in stages.

[0046] In one possible implementation, at least one of the first model, the second model, and the target model is a residual network model.

[0047] In a possible implementation manner, the target area is an eyelid area.

[0048] According to another aspect of an embodiment of the present disclosure, there is provided an electronic device for model training, including:

[0049] processor;

[0050] a memory for storing executable instructions for the processor;

[0051] Wherein, the processor is configured to execute the executable instructions to implement the model training method as described in the aforementioned embodiment.

[0052] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device can implement the model training method as described in the aforementioned embodiment.

[0053] According to another aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0054] A processor for loading a target model trained by the model training method described in the aforementioned embodiment;

[0055] a memory for storing executable instructions for the processor;

[0056] Wherein, the processor is configured to process the acquired facial image using the target model.

[0057] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device is enabled to load a target model trained by the model training method described in the aforementioned embodiment, and use the target model to process an acquired facial image.

[0058] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0059] The embodiments of the present disclosure can implement a cascade of a first processing and a second processing on a first facial image set. Since the first processing has the ability to remove existing facial feature textures, regardless of whether the first facial images in the first facial image set contain existing facial feature textures, the superposition of facial feature textures can be avoided in each second facial image of the second facial image set generated by implementing the facial feature textures through the second processing. Therefore, by using the first facial image set as input samples and the second facial image set as target comparison samples to train a target model, the target model can have the ability to remove existing facial feature textures while generating facial feature textures. Furthermore, when the target model is used to generate facial feature textures for the image to be processed, superposition of facial feature textures can be avoided in the processed image, so as to improve the image quality of the processed facial image.

[0060] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0062] Figure 1 is a basic flow chart diagram of a model training method in the present disclosure according to an exemplary embodiment;

[0063] Figure 2 is a specific flow chart of the model training method in the present disclosure according to an exemplary embodiment;

[0064] Figure 3 is a schematic diagram of the process of each iterative processing during the training of the target model in the present disclosure according to an exemplary embodiment;

[0065] Figure 4 It is a basic structural diagram of a model training device in the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0066] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0067] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0068] The facial feature texture of the facial image involved in the present disclosure includes the texture presented by the face in the usual sense, that is, the uneven grooves on the surface of the face, such as eyelids, eyelashes, etc.

[0069] Furthermore, for the facial feature texture generation process, the facial feature texture involved in the present disclosure can be divided into target facial feature texture and existing facial feature texture, wherein:

[0070] The target facial feature texture may refer to: a facial feature texture for generation in a target area;

[0071] The existing facial feature texture refers to the existing facial feature texture at the texture generation position in the target area or near the texture generation position.

[0072] For example, in a scenario where facial image beautification effect is produced by facial feature texture generation, the target facial feature texture may be the transverse wrinkle texture of the eyelid region, and the target region may be the region where eyelid generation is performed. Through the generation of facial feature texture, transverse wrinkle texture may be generated in the target region to form a "double eyelid" beautification effect. If the facial image to be processed already has a "double eyelid" effect, that is, there is an existing transverse wrinkle texture in the target region, then the existing transverse wrinkle texture is the existing facial feature texture, because the existing transverse wrinkle texture may overlap with the generated transverse wrinkle texture.

[0073] For another example, in a scenario where facial images are beautified by facial texture generation, the target facial feature texture may also be a contour texture after teeth are rearranged, and the target area may be an area where teeth are rearranged. By generating facial feature textures, a fictitious tooth contour texture after teeth are rearranged may be generated in the area where teeth are rearranged, so as to form a "teeth rearrangement correction" beautification effect. If there is already a tooth contour in the facial image to be processed, the existing tooth contour is the existing facial feature texture, because the existing tooth contour may overlap with the generated fictitious tooth contour.

[0074] The following is an analysis of the reasons why facial feature texture superposition occurs when using a neural network model to process facial images.

[0075] In image processing such as beauty generation technology, in order to achieve better results and reduce the complexity of processing, when using neural networks to process images, residual networks are often used. Taking the generation of double eyelids as an example, if the entire human eye area is directly regenerated, the network fitting ability needs to be strong and the calculation complexity is high; and it is easy to have the problem of inconsistency between the generated skin color and the original skin color, making the eye skin color abnormal. The use of residual networks can overcome the problem of inconsistent skin color and reduce the amount of calculation for model training and image processing using the model.

[0076] Specifically, in the processing of the residual network model, assuming that the input is the original image i, the network output is the residual d, and the complete output result of the final image processing is i+d; or assuming that the input is the original image i, the network outputs the residual d and the mask (mask) m, and the complete output result of the final image processing is o=d*m+i*(1-m). From the above-mentioned residual network processing, it can be seen that the residual network is only responsible for generating the difference between the input and the target output, and then superimposing the corresponding difference with the input to obtain the target output value, rather than the residual network directly generating all the outputs. Due to the above-mentioned characteristics of the residual network model, the residual result of its output will be superimposed with the original input image. Therefore, in some tasks, part of the information in the original image will be retained, such as facial feature texture information. If this part of the facial feature texture is no longer needed in the target image, an incorrect result will be generated. Taking the residual network model's image processing to achieve double eyelid generation as an example, the purpose is to generate another layer of eyelid in the eyelid area for a single-eyelid portrait to create double eyelids; however, if the user already has double eyelids, the final processing result will include both the double eyelids generated by the residual network model and the user's original double eyelids. In this way, retaining both eyelids at the same time will result in incorrect eyelid generation results, that is, there will be redundant eyelid textures.

[0077] In fact, in addition to the above problems of the residual network model, many neural network models will introduce the original image or part of the original image into the structure, such as the U-type network. This type of network model will have the above-mentioned problems when implementing the task of generating texture in the target area of ​​the image.

[0078] As mentioned above, the double eyelid generation task and the teeth rearrangement correction task are both typical facial feature texture generation tasks. In the following detailed description of the present disclosure, the double eyelid generation task will be mainly used as an example to introduce the specific implementation of the training method, device and electronic device of the neural network model.

[0079] In an exemplary scenario, the user inputs a portrait picture and clicks the double eyelid special effect button. The background system uses a neural network model to process the input portrait picture and outputs a portrait picture corresponding to the input, in which the eyelids of the portrait are double eyelids.

[0080] In view of this, an embodiment of the present disclosure provides a model training method for training and generating a neural network model, which can be used to generate textures in a target area without redundant textures, so that the quality of the processed image can reach the expected level.

[0081] Figure 1 FIG. 1 is a basic flow chart of the model training method in the present disclosure according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0082] Step 101: Acquire a first facial image set.

[0083] Step 102: Perform cascade processing on the first facial image set to obtain a second facial image set.

[0084] The cascade processing includes a first processing and a second processing that are performed successively, the first processing being used to remove the existing facial feature texture in the target area of ​​the facial image in the first facial image set, and the second processing being used to generate the same type of facial feature texture as in the first processing in the target area of ​​the facial image in the pre-processed facial image set obtained after the first processing. That is, the first processing is used to remove the facial feature texture, and the second processing is used to generate the same type of facial feature texture. In the exemplary task of double eyelid generation, the first processing may be a processing of transforming double eyelids into single eyelids, and the second processing may be a processing of transforming single eyelids into double eyelids. In the exemplary task of tooth realignment correction, the first processing may be a processing of removing the original tooth contour, and the second processing may be a processing of generating a fictitious tooth contour texture after the teeth are rearranged.

[0085] Each facial image set includes multiple facial images. After cascading processing, each facial image in the first facial image set obtains a corresponding facial image. After cascading processing, the facial images in the first facial image set respectively obtain corresponding facial images that constitute the second facial image set.

[0086] Step 103: input the first facial image set into the target model to be trained for processing to obtain a first output facial image set.

[0087] Steps 103 and 104 are used to train and generate a target model. Each facial image in the first facial image set is input into the target model for processing to obtain a series of output facial images, which constitute a first output facial image set.

[0088] Step 104 , using the error between the facial images in the first output facial image set and the corresponding facial images in the second facial image set, adjust the model parameters of the target model to be trained to train the target model.

[0089] Among them, the error between the facial images in the first output facial image set and the corresponding facial images in the second facial image set is used to calculate the loss function, and the model parameters of the target model are reversely adjusted according to the loss function to reduce the error size, thereby achieving the purpose of training the target model.

[0090] exist Figure 1 In the method flow shown, on the one hand, the existing facial feature texture contained in the original image is removed by the first processing before the second processing (i.e., facial feature texture generation processing) is performed, so that the existing facial feature texture is removed from the input image for the second processing, and therefore, after the second processing is performed to generate the same type of facial feature texture, the existing facial feature texture in the original image will no longer be included; on the other hand, the input image and output image of the cascade processing composed of the first processing and the second processing are used to train the target model, so as to generate the target model by knowledge distillation, so that the target model can be as close to the cascade processing as possible and the accuracy of the target model is guaranteed. Combining the above two aspects, the target model can achieve the generation of facial feature texture in the target area while avoiding the appearance of redundant facial feature textures, and for facial images processed by the target model, the image quality and display effect can be significantly improved.

[0091] In some examples, the above-mentioned cascade processing can be implemented with the help of the model architecture of the cascade model, for example, Figure 1 Step 102 in the illustrated process may specifically include:

[0092] Performing a first process on the first facial image set using the first model to obtain a preprocessed facial image set;

[0093] A second process is performed on the preprocessed facial image set using a second model cascaded with the first model to obtain a second facial image set.

[0094] The first process is used to remove existing facial feature textures from facial images in the first facial image set, and the second process is used to generate facial feature textures from facial images in the preprocessed facial image set, so that facial images in the second facial image set can have corresponding facial feature textures in the target area. Here, the facial feature textures generated in the second process are the same type of facial feature textures as those removed in the first process, for example, both are eyelid textures, or both are tooth contour textures, etc.

[0095] The specific implementation of the above disclosed model training method is described below through a specific embodiment, wherein the double eyelid generation task is still used as an example for introduction.

[0096] Figure 2 It is a specific flow chart of the model training method in the present disclosure according to an exemplary embodiment. In this embodiment, the first processing is implemented by a first neural network model (hereinafter referred to as the first model), and the second processing is implemented by a second neural network model (hereinafter referred to as the second model). The facial images used for model training are also commonly referred to as training samples (referred to as samples), where the facial images input to the model can also be called input samples, and the facial images used for comparison with the model output can also be called target comparison samples. Figure 2 As shown, the method includes:

[0097] Step 201: Acquire a first facial image set.

[0098] The facial image set obtained in this step is the input facial image set for training the target model. For consistency in the disclosure, the facial image set prepared in this step is referred to as the first facial image set. In the exemplary task of double eyelid generation, a large number of human eye images are prepared, and optionally, these human eye images are distributed as evenly as possible.

[0099] Step 202: Load the first model and the second model.

[0100] After the first model and the second model are loaded, a cascade relationship can be formed from the first model to the second model.

[0101] The second model can be a neural network model for generating facial feature textures in the target area, such as a residual network or a U-shaped network mentioned in the background technology to achieve double eyelid generation, referred to as single eyelid to double eyelid processing. The second model has the problem of the neural network model described in the background technology, that is, the processed image may have overlapping facial feature textures.

[0102] The first model may be a neural network model for removing existing facial feature textures in a target area. The removal of existing facial feature textures in a target area may be, for example, removing a layer of eyelids in the eyelid area of ​​a portrait image with double eyelids, so that the double eyelids in the portrait image are replaced with single eyelids, referred to as double eyelid to single eyelid processing. Specifically, the first model may be generated by training in an existing manner, for example, a structure opposite to that of the second model may be adopted, and the training data of the second model may also be used for training. Specifically, the input data of the second model training may be used as the target comparison data of the first model training, and the target comparison data of the second model training may be used as the input data of the first model training.

[0103] Step 203: Initialize the target model and use the initialized target model as the current target model.

[0104] Preferably, a suitable neural network type, such as a residual network, can be selected for the target model according to the requirements and characteristics of the task, the resources and computing power of the system, etc. This step initializes the target model, including the initialization of various parameters, etc. Since the target model training is a multi-iteration process, for the sake of clarity, the target model processed in each iteration is referred to as the current target model. The initialized target model is the current target model at the first iteration.

[0105] Step 204: Select a first facial image A from the first facial image set to input into the first model and the current target model.

[0106] The facial images selected from the first facial image set may be selected in an existing manner according to actual needs or computing power, or facial images may be selected randomly. In the present disclosure, the target model is trained by knowledge distillation, so the first model and the target model use the same input facial images.

[0107] Step 205: Perform a first process on the selected first facial image A using the first model to obtain a preprocessed facial image B.

[0108] The first model is used to perform the first processing on the first facial image A, thereby erasing the existing facial feature texture in the target area of ​​the first facial image A. For example, if the first model is a network model that transforms double eyelids into single eyelids, the first processing is to remove the second layer of eyelids in the eyelid area of ​​the first facial image A, leaving only one layer of eyelids, so that the first facial image A becomes a single eyelid image. In this way, the existing facial feature texture is removed in advance by introducing the first model, avoiding the subsequent overlapping of facial feature textures.

[0109] Step 206: input the pre-processed facial image B into the second model, and use the second model to perform a second process on the pre-processed facial image set B to obtain a second facial image C.

[0110] The second model is used to perform the second processing on the preprocessed facial image B, that is, to generate texture in the target area of ​​the preprocessed facial image B. For example, the second model is a neural network model for generating double eyelids, and the second processing is to regenerate a layer of eyelids in the eyelid area of ​​the preprocessed facial image B, thereby forming a double eyelid, so that the second facial image C becomes a double eyelid image.

[0111] As mentioned above, the preprocessed facial image B obtained by the processing in step 204 removes the existing facial feature texture. Therefore, after the corresponding preprocessed facial image B is input into the second model again, it is ensured that the input image of the second model has no facial feature texture in the target area, which is equivalent to the target area in the original facial image input into the second model having no existing facial feature texture. In this way, after the second model is used for the second processing, although the original image information will be introduced in the final output, since there is no facial feature texture in the target area in the original image, the final output image will not have the problem of superposition of facial feature texture, thereby effectively improving the image quality and display effect.

[0112] Step 207: Input the first facial image A into the current target model for processing to obtain a first output facial image D.

[0113] The target model in the present disclosure may be a type of neural network model selected based on comprehensive considerations such as task requirements, system computing capabilities, and computing resources, such as a residual network or a U-shaped network, etc. At the same time, the training target model in the present disclosure tries to be close to the overall model formed by sequentially cascading the first model and the second model, so as to achieve the same processing function as the overall model formed by sequentially cascading the first model and the second model, that is, to avoid overlapping facial feature textures while generating facial feature textures in the target area of ​​the facial image.

[0114] Step 208, determining the error L between the second facial image C and the first output facial image D, and optimizing and updating the parameters in the target model.

[0115] Generally, the training of neural network models requires labeled data (for example, marked as double eyelids or single eyelids), which are usually manually labeled. How much data can be used for training depends on how much data has been labeled. In the present disclosure, knowledge distillation is introduced to train the target model. The input first facial image A is processed by the overall model X composed of the first model and the second model cascaded in sequence to obtain the output second facial image B, and the first facial image A and the second facial image B are used as a pair of training data to train the target model. Among them, the second facial image B is used as the target comparison sample for the training of the target model. In this way, the overall model X can be used to automatically generate the labeled samples required for the training of the target model without the need for manually labeled samples. This is equivalent to having nearly unlimited training data, so that the training of the target model has enough training samples to train a more accurate model. At the same time, the training samples generated by the overall model X ensure that the input samples and output samples of the target model are from the same face, which can further improve the accuracy of the model.

[0116] Based on this, in this step, the second facial image C output by the second model is used as a target comparison sample, and is compared with the facial image D output by the target model to determine the error, and the parameters of the target model are optimized and updated accordingly to train the target model. The specific process of determining the error and updating the target parameters can be performed in various existing ways, such as using a gradient descent method.

[0117] In addition, to adapt to systems with relatively limited computing power or computing resources, the target model can be designed as a smaller-scale neural network model, specifically, the amount of calculation and the scale of parameters can be smaller than the overall model X to be approximated. For example, the overall model X can be used as a Teacher model, and the target model can be trained as a Student model.

[0118] Step 209, determine whether the iterative processing of the training target model is completed, if so, execute step 211; otherwise, execute step 210.

[0119] The method for determining whether the iterative processing is ended in this step can be carried out in an existing manner. For example, when the error between the first output facial image D and the second facial image C is within a set threshold range, it is determined to stop the iteration; or, when the error between the first output facial image D and the second facial image C no longer decreases or decreases insignificantly within a set number of iterations, it is determined to stop the iteration.

[0120] If it is determined that the iterative processing is completed, that is, the training of the target model is completed, step 311 can be executed to output the target model with updated parameters; if it is determined that the iterative processing is not completed, step 310 needs to be continued to prepare for the next iteration.

[0121] Step 210 , increase the number of iterations by 1, and use the target model after updating the parameters as the current target model, and return to step 204 .

[0122] Step 211, outputting the target model after updating the parameters.

[0123] At this point, the model training method process in the embodiment of the present disclosure ends. In the above processing flow, the target model training process mainly performed with one facial image is described in detail, all first facial images A input to the target model and the first model constitute a first facial image set, all pre-processed facial images B output by the first model constitute a pre-processed facial image set, all second facial images C output by the second model constitute a second facial image set, and all first output facial images D output by the target model constitute a first output facial image set.

[0124] At the same time, in the above process, each iteration of training the target model includes two branches of processing, one branch is the processing of the input facial image by the overall model X, and the other branch is the processing of the input facial image by the target model, such as Figure 3 As shown. In fact, when the target model is trained, before training, the overall model X can first complete the processing of the first facial image set, generate all the target comparison samples for training the target model, and form the second facial image set. In other words, the facial images for training have been generated before the target model is trained. Figure 2 The processing flow shown requires less storage resources and is easier to implement than the method of completely generating training samples before training.

[0125] In addition, in the above Figure 2 In the process shown, the first model and the second model are obtained in step 202, that is, Figure 2 In the process shown, before the target model is trained, the first model and the second model have been trained. Among them, after the first facial image A in the first facial image set is input into the overall model X and the current target model, the processing of the overall model X to generate the second facial image C and the target model to generate the first output facial image B can be performed simultaneously, so that the generated second facial image C can be directly compared with the first output facial image B to determine the error, and the second facial image C does not need to be saved additionally, which can greatly save the storage resources consumed by the target model training. Of course, according to actual needs, the facial images in the first facial image set can also be processed in advance using the overall model X and then the second facial image set composed of the second facial images can be saved; when training the target model, the second facial image set and the first facial image set are taken out, and the samples are paired for training the target model, but such processing requires relatively more storage resources.

[0126] Next, the training contents of the first model and the second model are introduced as follows:

[0127] The first model and the second model may be trained independently of each other. In some examples, the training of the first model and the training of the second model may use completely different facial image samples.

[0128] In other examples, considering that the first model and the second model perform symmetrical processing of removing existing facial feature textures and generating facial feature textures, the facial image samples used to train the first model and the second model can also be symmetrical. Based on this, the facial image samples used to train the first model and the second model can be partially or fully reused.

[0129] Specifically, the training of the second model can be performed using the third facial image set and the pre-configured target facial image set. The general training process of the second model can include: obtaining the third facial image set; inputting the third facial image set into the second model for processing to generate facial feature textures in the target area of ​​each facial image in the third facial image set to obtain a second output facial image set; using the error between the facial image in the second output facial image set and the corresponding facial image in the pre-configured target facial image set, adjusting the model parameters of the second model to train the second model.

[0130] The training of the first model can also be performed using the third facial image set and the target facial image set. The general training process of the first model may include: inputting part or all of the facial images in the target facial image set into the first model for processing, so as to remove the existing facial feature texture in the target area of ​​each facial image in the target facial image set, and obtain a third output facial image set; using the error between the facial images in the third output facial image set and the corresponding part or all of the facial images in the third facial image set, adjusting the model parameters of the first model to train the first model.

[0131] As can be seen from the above, when training the second model, the facial images in the third facial image set are used as input samples for training the second model, and the facial images in the target facial image set are used as target comparison samples for training the second model. When training the first model, some or all of the facial images in the third facial image set and some or all of the facial images in the target facial image set can be used to train the second model, and some or all of the facial images in the third facial image set reused for training the second model are used as target comparison samples for training the first model, and some or all of the facial images in the target facial image set reused for training the second model are used as input samples for training the first model. Since the same training samples are reused for the training of the first model and the training of the second model, the number of sample preparations for the input samples can be reduced, which is conducive to simplifying the sample collection for model training.

[0132] In some examples, the network scale of the second model can be comparable to that of the first model, and the network scale of the target model can be smaller than that of the first model and the second model, and in particular, much smaller than that of the second model. That is, relative to the target model, the second model can be a neural network model with a larger network scale. In this case, the first model and the second model have stronger learning capabilities than the target model due to their larger network scales, and in particular, the second model can have a much stronger learning capability than the target model.

[0133] In the training of the second model, the facial images in the third facial image set are used as input samples for training the second model, and the facial images in the target facial image set are used as target comparison samples for training the second model. Therefore, the number of facial images in the third facial image set and the target facial image set is the same.

[0134] In the training of the target model, the facial images in the first facial image set are used as the input of the target model, and the facial images in the second facial image set are used as the target comparison samples for the training of the target model. Therefore, the number of facial images in the first facial image set and the second facial image set is also the same.

[0135] Among them, since the network scale of the second model is larger and the learning ability is stronger, the number of facial images in the third facial image set and the second target facial image set (for example, the first number) can be smaller than the number of facial images in the first facial image set and the second facial image set (for example, the second number).

[0136] Based on the above sample configuration method, the cascade model (especially the second model) with smaller training sample requirements can be used to automatically generate more samples for the target model, simplifying the manual labeling process of the target model that requires a large number of samples, and then supporting the situation where the target model is a small-scale model, so that the training of the second model and the target model can be more efficient.

[0137] In some examples, the training timings of the first model and the second model may be consistent or inconsistent. Moreover, the relationship between the training timings of the first model and the second model and the training timing of the target model may also be arbitrarily set.

[0138] Specifically, for the training of the first model, it is assumed that the fourth facial image set and the fifth facial image set are used as input samples and target comparison samples for training the first model, respectively. Of course, if the training of the first model and the second model reuses some training samples, the facial images in the fourth facial image set can be part or all of the facial images in the target facial image set, and the facial images in the fifth facial image set can be part or all of the facial images in the third facial image set.

[0139] Based on the above setting of training samples, the training of the first model can be completed before the cascade processing is performed on the first facial image set, that is, the training of the target model is performed based on the trained first model.

[0140] Alternatively, the training of the first model can also be completed alternately with the training of the target model in stages. That is, before training the target model, the facial images in the fourth facial image set are divided into multiple parts, and the training of the first model is performed in stages; the partial facial images in the fourth facial image set are processed using the overall model X including the staged training results of the first model to generate partial facial images in the second facial image set for staged training of the target model; in this way, the staged training of the first model and the target model are performed alternately until the training of the first model and the target model is completed.

[0141] Alternatively, the training of the first model can also be performed synchronously with the training of the target model. That is, the first model is trained using the facial images in the fourth facial image set, and the parameters of the first model are updated after each iterative process, and the first model parameters constituting the overall model X are updated synchronously; at the same time, the facial images in the first facial image set are input into the overall model X to generate the facial images in the second facial image set for the training of the target model.

[0142] Compared with the method of completing the first model training before cascading the first facial image set, the processing method that is synchronized with the training of the target model or alternately performed in stages can more efficiently train the model for the target model with weaker learning ability.

[0143] Specifically, some target models with weaker learning ability may not be able to quickly learn useful information in the early stages of training if they directly learn the accurate large-scale first model. If the first model and the target model can be trained synchronously or alternately, the learning difficulty of the target model with weaker learning ability in the early stages of training may be reduced.

[0144] For the training of the second model, assuming as above, the third facial image set is used as input samples for training the second model, and the target facial image set is used as target comparison samples for training the second model.

[0145] Based on the above setting of training samples, the training of the second model can be completed before the cascade processing of the first facial image set, that is, the training of the target model is based on the trained second model.

[0146] Alternatively, the training of the second model can also be completed alternately with the training of the target model in stages. That is, before training the target model, the facial images in the third facial image set are divided into multiple parts, and the training of the second model is performed in stages; the partial facial images in the first facial image set are processed using the overall model X including the staged training results of the second model to generate partial facial images in the second facial image set, which are used for staged training of the target model; in this way, the staged training of the second model and the target model are performed alternately until the training of the second model and the target model is completed.

[0147] Alternatively, the training of the second model can also be performed synchronously with the training of the target model. That is, the second model is trained using the facial images in the third facial image set, and the parameters of the second model are updated after each iterative process, and the second model parameters constituting the overall model X are updated synchronously; at the same time, the facial images in the first facial image set are input into the overall model X to generate the facial images in the second facial image set for the training of the target model.

[0148] Similar to the first model, compared with the method of completing the training of the second model before cascading the first facial image set, the processing method that is synchronized with the training of the target model or alternately performed in stages can help reduce the learning difficulty of the target model in the initial stage of training for the target model with weaker learning ability, and can train the model more efficiently.

[0149] The above is the specific implementation of the model training method in this disclosure.

[0150] The present application also discloses an image processing method, which can use the target model S trained by the aforementioned model training method to process the acquired facial image to obtain the target image. For the target model S trained by the aforementioned model training method, the target model S can also be embedded in other image processing processes of a certain device. Specifically, in the usage scenario, the device can send facial pictures collected from mobile phones, clouds, etc. or output facial pictures of other previous image processing processes as input to the target model S. The target model S processes the input facial picture (for example, double eyelid generation or tooth realignment correction, etc.) and then outputs it, and then outputs the facial picture output by the target model S as an image processing result or outputs it to a subsequent image processing process for processing.

[0151] The present disclosure also provides a model training device, which can be used to implement the above-mentioned model training method. Figure 4 Figure 1 is a basic structural diagram of the model training device. Figure 4 As shown, the device includes: an interface unit, a cascade processing unit and a target model training unit.

[0152] Wherein, the interface unit is used to obtain a first facial image set.

[0153] The cascade processing unit is used to perform cascade processing on the first facial image set to obtain a second facial image set; wherein the cascade processing includes a first processing and a second processing performed successively, the first processing is used to remove the existing facial feature texture in the target area of ​​the facial image in the first facial image set, and the second processing is used to generate facial feature texture in the target area of ​​the facial image in the pre-processed facial image set obtained after the first processing. For example, the target area can be an eyelid area, or the target area can also be an area of ​​tooth rearrangement.

[0154] The target model training unit is used to use the first facial image set to input the target model to be trained for processing to obtain a first output facial image set; and to adjust the model parameters of the target model to be trained by using the error between the facial images in the first output facial image set and the corresponding facial images in the second facial image set to train the target model. For example, at least one of the first model, the second model and the target model can be a residual network model.

[0155] exist Figure 4 In the model training device shown, on the one hand, the existing facial feature texture contained in the original image is removed by the first process before the second process (i.e., facial feature texture generation process) is performed, so that the existing facial feature texture is removed from the input image for the second process, and therefore, after the texture generation operation is performed by the second process, there will be no overlap of facial feature textures; on the other hand, the input image and output image of the cascade process composed of the first process and the second process are used to train the target model, so as to generate the target model by knowledge distillation, so that the target model can be as close to the cascade process as possible and the accuracy of the target model is guaranteed. Combining the above two aspects, the target model can achieve the generation of facial feature textures in the target area while avoiding texture overlap, and for images processed by the target model, the image quality and display effect can be significantly improved.

[0156] In some examples, the above-mentioned cascade processing may be implemented with the help of a model architecture of a cascade model. For example, the cascade processing unit may specifically include:

[0157] A first model is used to perform a first process on the first facial image set to obtain a preprocessed facial image set;

[0158] The second model is used to perform a second process on the pre-processed facial image set using the second model to obtain a second facial image set.

[0159] The first processing is used to remove existing facial feature textures from the facial images in the first facial image, and the second processing is used to generate facial feature textures in a target area of ​​the facial images in the pre-processed facial image set.

[0160] In a possible implementation, in order to implement training of the first model and the second model, the model training device also includes a first model training unit for training the first model, and a second model training unit for training the second model.

[0161] As described above, the first model and the second model may be trained independently of each other. In some examples, the training of the first model and the training of the second model may use completely different samples.

[0162] In other examples, considering that the first model and the second model perform symmetrical processing such as erasing texture and generating texture, the facial images used to train the first model and the second model can also be symmetrical. Based on this, the facial images used to train the first model and the second model can be partially or fully reused. For example:

[0163] The method for performing the second model training in the second model training unit may include:

[0164] Acquire a third facial image set;

[0165] Inputting the third facial image set into the second model for processing to generate the facial feature texture in the target area of ​​each facial image in the third facial image set, thereby obtaining a second output facial image set;

[0166] adjusting a model parameter of the second model by using an error between a facial image in the second output facial image set and a corresponding facial image in a pre-configured target facial image set to train the second model;

[0167] The first model training method in the first model training unit also includes:

[0168] Inputting part or all of the facial images in the target facial image set into the first model for processing, so as to remove existing facial feature textures in the target area of ​​each facial image in the target facial image set, and obtain a third output facial image set;

[0169] The model parameters of the first model are adjusted by using errors between facial images in the third output facial image set and corresponding partial or all facial images in the third facial image set to train the first model.

[0170] Based on this, since the same training samples are reused for the training of the first model and the training of the second model, the number of sample preparations for the input samples can be reduced, which is conducive to simplifying the sample collection for model training.

[0171] In some examples, the network scale of the second model can be comparable to that of the first model, and the network scale of the target model can be smaller than that of the first model and the second model, and in particular, much smaller than that of the second model. That is, relative to the target model, the second model can be a neural network model with a larger network scale. In this case, the first model and the second model have stronger learning capabilities than the target model due to their larger network scales, and in particular, the second model can have a much stronger learning capability than the target model.

[0172] In this case, the number of facial images in the third facial image set and the target facial image set is a first number, the number of facial images in the first facial image set and the second facial image set is a second number, and the second number is greater than the first number.

[0173] Based on the above sample configuration method, the cascade model (especially the second model) with smaller training sample requirements can be used to automatically generate more samples for the target model, simplifying the manual labeling process of the target model that requires a large number of samples, and then supporting the situation where the target model is a small-scale model, so that the training of the second model and the target model can be more efficient.

[0174] In some examples, the training of the first model by the first model training unit may be completed before the cascade processing unit performs cascade processing on the first facial image set; or, the training of the first model by the first model training unit may be synchronized with the training of the target model by the target model training unit; or, the training of the first model by the first model training unit may be completed alternately in stages with the training of the target model by the target model training unit.

[0175] In some examples, the training of the second model by the second model training unit may be completed before the cascade processing unit performs cascade processing on the first facial image set; or, the training of the second model by the second model training unit may be synchronized with the training of the target model by the target model training unit; or, the training of the second model by the second model training unit may be completed alternately in stages with the training of the target model by the target model training unit.

[0176] Compared with the method of completing the training of the first model and the second model before cascading the first facial image set, the processing method of synchronously or alternatingly performing the training of the target model in stages can more efficiently train the model for the target model with weaker learning ability.

[0177] It should be noted that: the above embodiment only illustrates the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0178] The present disclosure also provides an electronic device for model training, which will be referred to as a model training electronic device hereinafter. Generally, considering the computing resources and computing requirements of the model training electronic device, the model training electronic device is usually a server or a terminal device with powerful processing capabilities. The model training electronic device includes at least one processor and at least one memory. The memory is used to store at least one instruction, and the at least one instruction is used to be executed by the processor to implement the model training method provided in the present disclosure.

[0179] The present disclosure further provides an electronic device for image processing, which will be referred to as an image processing electronic device hereinafter, and the image processing electronic device may be a terminal or a server. The image processing electronic device includes at least one processor and at least one memory. The processor is used to load a target model trained using the model training method provided by the present disclosure; the memory is used to store executable instructions of the processor; the processor is configured to process the acquired facial image using the target model to obtain a target image.

[0180] Specifically, when the model training electronic device or the image processing electronic device is a terminal, the terminal may be a smart phone, a tablet computer, a laptop computer or a desktop computer. The terminal may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0181] Typically, a terminal includes: a processor and a memory.

[0182] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0183] The memory may include one or more computer-readable storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices.

[0184] In some embodiments of the model training electronic device, the non-transitory computer-readable storage medium in the memory is used to store at least one instruction, which is used to be executed by the processor to implement the model training method provided in the present disclosure.

[0185] Those skilled in the art will appreciate that the above structure does not limit the terminal, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0186] When the model training electronic device or image processing electronic device provided in the present disclosure is a server, the server may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein, for the model training electronic device, the memory stores at least one program code, which is loaded and executed by the processor to implement the model training method provided in the above-mentioned embodiments; for the image processing electronic device, the memory stores at least one program code, which is loaded and executed by the processor to enable the image processing electronic device to load the target model trained by the model training method provided in the above-mentioned embodiments, and use the target model to process the acquired image to be processed. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be repeated here.

[0187] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to complete the model training method provided by the present disclosure.

[0188] In an exemplary embodiment, a computer-readable storage medium including at least one instruction is also provided, such as a memory including at least one instruction. The at least one instruction can be executed by a processor in a computer device to enable the image processing electronic device to load a target model trained by the model training method provided in the above embodiments, and use the target model to process the acquired image to be processed.

[0189] Optionally, the above-mentioned computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may include ROM (Read-Only Memory), RAM (Random-Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc.

[0190] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0191] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A model training method, characterized in that: include: Acquire a first facial image set; Performing cascade processing on the first facial image set to obtain a second facial image set; wherein the cascade processing includes a first process and a second process performed successively, the first process is used to remove existing facial feature textures in a target area of ​​facial images in the first facial image set, and the second process is used to generate the facial feature textures in the target area of ​​facial images in the pre-processed facial image set obtained after the first process; Inputting the first facial image set into a target model to be trained for processing to obtain a first output facial image set; adjusting a model parameter of the target model to be trained by using an error between a facial image in the first output facial image set and a corresponding facial image in the second facial image set, so as to train the target model; The step of performing cascade processing on the first facial image set to obtain a second facial image set includes: Using a first model to perform the first processing on the first facial image set to obtain the pre-processed facial image set; performing the second processing on the pre-processed facial image set using a second model cascaded with the first model to obtain the second facial image set; The method of training the second model comprises: acquiring a third facial image set; Inputting the third facial image set into the second model for processing to generate the facial feature texture in the target area of ​​each facial image in the third facial image set to obtain a second output facial image set; adjusting model parameters of the second model by using errors between facial images in the second output facial image set and corresponding facial images in a pre-configured target facial image set to train the second model; The method of training the first model comprises: Inputting part or all of the facial images in the target facial image set into the first model for processing, so as to remove existing facial feature textures in the target area of ​​each facial image in the target facial image set, and obtain a third output facial image set; The model parameters of the first model are adjusted by using errors between facial images in the third output facial image set and corresponding partial or all facial images in the third facial image set to train the first model.

2. The method according to claim 1, characterized in that The number of facial images in the third facial image set and the target facial image set is a first number, the number of facial images in the first facial image set and the second facial image set is a second number, and the second number is greater than the first number.

3. The method according to claim 2, characterized in that The training of the second model is completed before the cascade processing of the first facial image set; or, The training of the second model is synchronized with the training of the target model; or, The training of the second model and the training of the target model are completed alternately in stages.

4. The method according to claim 2, characterized in that: The network scale of the target model is smaller than the network scale of the second model.

5. The method according to claim 1, characterized in that The training of the first model is completed before the cascade processing of the first facial image set; or, The training of the first model is synchronized with the training of the target model; or The training of the first model and the training of the target model are completed alternately in stages.

6. The method according to claim 1, characterized in that At least one of the first model, the second model and the target model is a residual network model.

7. The model training method according to claim 1, characterized in that: The target area is the eyelid area.

8. An image processing method, characterized in that: include: The target model trained by the model training method according to any one of claims 1 to 7 is used to process the acquired facial image to obtain a target image.

9. A model training device, characterized in that: include: An interface unit, a cascade processing unit, and a target model training unit; The interface unit is used to obtain a first facial image set; The cascade processing unit is used to perform cascade processing on the first facial image set to obtain a second facial image set; wherein the cascade processing includes a first process and a second process that are performed successively, the first process is used to remove the existing facial feature textures in the target area of ​​the facial images in the first facial image set, and the second process is used to generate the facial feature textures in the target area of ​​the facial images in the pre-processed facial image set obtained after the first process; The target model training unit is used to input the first facial image set into the target model to be trained for processing to obtain a first output facial image set; and adjust the model parameters of the target model to be trained by using the errors between the facial images in the first output facial image set and the corresponding facial images in the second facial image set to train the target model; The cascade processing unit comprises: a first model, configured to perform the first processing on the first facial image set to obtain the pre-processed facial image set; a second model, configured to perform the second processing on the pre-processed facial image set to obtain the second facial image set; Wherein, the model training device also includes: The second model training unit is used to obtain a third facial image set; input the third facial image set into the second model for processing to generate the facial feature texture in the target area of ​​each facial image in the third facial image set to obtain a second output facial image set; and adjust the model parameters of the second model by using the error between the facial image in the second output facial image set and the corresponding facial image in the pre-configured target facial image set to train the second model; The first model training unit is used to input part or all of the facial images in the target facial image set into the first model for processing, so as to remove the existing facial feature texture in the target area of ​​each facial image in the target facial image set to obtain a third output facial image set; and adjust the model parameters of the first model by using the errors between the facial images in the third output facial image set and the corresponding part or all of the facial images in the third facial image set to train the first model.

10. The model training device according to claim 9, characterized in that: The number of facial images in the third facial image set and the target facial image set is a first number, the number of facial images in the first facial image set and the second facial image set is a second number, and the second number is greater than the first number.

11. The model training device according to claim 10, characterized in that: The training of the second model by the second model training unit is completed before the cascade processing is performed on the first facial image set; or, The training of the second model by the second model training unit is synchronized with the training of the target model; or, The training of the second model by the second model training unit and the training of the target model are completed alternately in stages.

12. The model training device according to claim 10, characterized in that: The network scale of the target model is smaller than the network scale of the second model.

13. The model training device according to claim 9, characterized in that: The training of the first model by the first model training unit is completed before the cascade processing is performed on the first facial image set; or, The training of the first model by the first model training unit is synchronized with the training of the target model; or, The training of the first model by the first model training unit and the training of the target model are completed alternately in stages.

14. The model training device according to claim 9, characterized in that: At least one of the first model, the second model and the target model is a residual network model.

15. The model training device according to claim 9, characterized in that: The target area is the eyelid area.

16. An electronic device, characterized in that: include: A processor for loading a target model trained by the model training method according to any one of claims 1 to 7; a memory for storing executable instructions for the processor; Wherein, the processor is configured to process the acquired facial image using the target model.

17. A computer-readable storage medium, characterized in that: When at least one instruction in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device can load a target model trained by the model training method according to any one of claims 1 to 7, and use the target model to process the acquired facial image.

Images