A training method and system for head migration model

Abstract

The present application provides a model training method and system for head migration. The method obtains a sample person target in the driving frame image by obtaining a driving video and extracting a driving frame image from the driving video. The driving frame image is cropped based on the sample person target to obtain a local feature image of the sample person target. Among them, the local feature image includes a head and shoulder feature image and a facial feature image. The head and shoulder feature image includes a head feature image and a shoulder feature image, and the shoulder feature image is used to adjust the scaling ratio of the facial feature image. Then, the head and shoulder feature image and the facial feature image are respectively input into the training network so that the face, head and shoulders have a higher degree of fusion to obtain the target image. Finally, the generation loss is calculated according to the target image. When the generation loss is less than the loss threshold, the current training parameters can be output to obtain a trained head migration model.

Description

Technical Field

[0001] The present application relates to the technical field of expression transfer, and in particular to a training method and system for a head transfer model. Background Art

[0002] Digital humans are created and used by computer means and can have the same appearance or behavior as humans. Smart terminals such as mobile terminals and display devices can all be used as carriers of digital humans. When using smart terminals as carriers, digital humans can interact with users and can be applied to multiple fields such as film and television, media, games, and retail.

[0003] When used, some digital humans can imitate the expressions and movements of the characters in the driving video and the driving video template to complete some social work in various fields, increasing the fun while liberating productivity. In order to make the digital human restore the expressions and movements of the template characters more accurately, it is necessary to improve the quality of head migration and movement migration. When performing head migration, the key points of the template character's head are usually collected, and affine transformation is performed based on the key points to perform head migration. This method can achieve better migration effects.

[0004] During the head migration process, the larger the migration area, the more difficult the affine transformation will be and the lower the computational efficiency will be. When the shoulder movements of the character in the driving video are too large, the position of the head area will change significantly, making it impossible to accurately obtain the positional relationship between the shoulder and the head, which in turn makes the adaptation of the migrated head and shoulders poor. Summary of the invention

[0005] The present application provides a model training method and system for head migration to solve the problem that the position of the head area changes greatly due to excessive shoulder movements in sample tasks in driving videos, resulting in poor adaptability between the migrated head and the shoulders of the template character.

[0006] In a first aspect, the present application provides a model training method for head migration, comprising:

[0007] Acquire a driving video, and extract a driving frame image from the driving video, wherein the driving frame image includes a sample human target;

[0008] The driving frame image is cropped according to the sample human target to obtain a local feature image containing the sample human target, wherein the local feature image includes a head-shoulder feature image and a facial feature image; the head feature image and the facial feature image of the head-shoulder feature image are target images to be migrated, and the shoulder feature image of the head-shoulder feature image is used to adjust the scaling ratio of the facial feature image;

[0009] Inputting the head and shoulder feature image and the facial feature image into a training network model respectively to obtain a first target image output by the training network model, wherein the first target image is a training result image;

[0010] Calculate the generation loss between the first target image and the standard image of the sample person using a loss function;

[0011] If the value of the generated loss is less than the loss threshold, the training parameters of the current training network model are output to obtain a trained head migration model.

[0012] In some embodiments, the method further comprises:

[0013] If the value of the generation loss is greater than the loss threshold, the training parameters of the training network model are updated according to the generation loss, and iterative training is performed to recalculate the generation loss.

[0014] In some embodiments, the method further comprises:

[0015] Obtain the head and shoulder feature image of the template person and the face area image of the target person;

[0016] The head-shoulder region image and the face region image are input into the head migration model to obtain a second target image, where the second target image is an application image.

[0017] In some embodiments, when the driving frame image is cropped according to the sample task target to obtain a local feature image containing the sample person target, the method includes:

[0018] Performing a cropping operation according to the sample character features to obtain the head and shoulder feature image;

[0019] A cropping operation is performed on the head and shoulder feature image to obtain the facial feature image.

[0020] In some embodiments, performing a cropping operation on the head and shoulder feature image to obtain the facial feature image includes:

[0021] Identifying a facial region in the head and shoulders feature image;

[0022] Taking the face area as the center, setting the cropping offset ratio;

[0023] The head and shoulders feature image is cropped according to the cropping offset ratio to obtain the facial feature image.

[0024] In some embodiments, when the head and shoulder feature image is cropped according to the cropping offset ratio to obtain the facial feature image, the method further includes:

[0025] Marking a cropped area in the head and shoulders feature image;

[0026] Adjusting the pixel value of the cropped area;

[0027] The facial image obtained by cropping the head and shoulder feature image is normalized to obtain the facial feature image.

[0028] In some embodiments, the method further comprises:

[0029] Performing clarity recognition on the first target image to obtain a training resolution;

[0030] If the training resolution is less than the resolution threshold, the clarity of the first target image is improved.

[0031] In some embodiments, after the head and shoulder feature image and the facial feature image are respectively input into a training network, the method further includes:

[0032] Encoding the head-shoulder feature image and the facial feature image respectively to obtain an encoded image of the head-shoulder feature image and an encoded image of the facial feature image;

[0033] According to the training parameters of the training network, feature fusion is performed on the encoded image of the head and shoulder feature image and the encoded image of the facial feature image to obtain a feature fused image;

[0034] Decoding is performed on the feature fusion image to obtain the first target image.

[0035] In a second aspect, the present application also provides a head migration training system, the system comprising a preprocessing module and a training module;

[0036] The preprocessing module is also used to preprocess the sample person to obtain a local feature image of the sample person, wherein the local feature image includes a head-shoulder feature image and a facial feature image; the head feature image of the head-shoulder feature image and the facial feature image are target images to be migrated, and the shoulder feature image of the head-shoulder feature image is used to adjust the scaling ratio of the facial feature image;

[0037] The training module is used to input the head and shoulder feature image and the facial feature image into a training network respectively to obtain a first target image, where the first target image is a training image;

[0038] The training module is also used to calculate the generation loss between the first target image and the standard image of the sample person using a loss function;

[0039] The training module is also used to output the training parameters of the current training network model when the value of the generated loss is less than the loss threshold, so as to obtain a trained head migration model.

[0040] In some embodiments, the training module is further configured to:

[0041] When the value of the generation loss is greater than the loss threshold, the training parameters of the training network model are updated according to the generation loss, and iterative training is performed to recalculate the generation loss.

[0042] In some embodiments, the system further comprises an application module;

[0043] The application module is used to obtain a head and shoulder area image of a template person and a face area image of a target person;

[0044] The application module is further used to input the head-shoulder region portrait and the face region image into the head migration model to obtain a second target image, where the second target image is an application image.

[0045] It can be seen from the above technical content that the present application provides a model training method and system for head migration. The method obtains a sample person target in the driving frame image by obtaining a driving video and extracting a driving frame image from the driving video. The driving frame image is cut based on the sample person target to obtain a local feature image of the sample person target. The local feature image includes a head and shoulder feature image and a facial feature image. The head and shoulder feature image includes a head feature image and a shoulder feature image, and the shoulder feature image is used to adjust the scaling ratio of the facial feature image. Then the head and shoulder feature image and the facial feature image are respectively input into the training network so that the face, head and shoulders have a higher degree of fusion to obtain the target image. Finally, the generation loss is calculated according to the target image. When the generation loss is less than the loss threshold, the current training parameters can be output to obtain a trained head migration model. The present application inputs the head and shoulder feature image and the facial feature image into the training network, and encodes them respectively to extract their respective features, so that the facial features of the head migration model can be scaled with the head and shoulder features as a reference, and then the features are fused, so that the fusion effect of the head migration model is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 A schematic diagram of a head migration model training method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of a network training model provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of clarity judgment provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of an application header migration model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0052] It should be noted that the terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] The "multiple embodiments", "some embodiments", "one embodiment" or "embodiment" mentioned throughout this specification means that the specific features, components or characteristics described in conjunction with the embodiment are included in at least one embodiment. Therefore, the phrases "in multiple embodiments", "in some embodiments", "in at least another embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment. In addition, in one or more embodiments, specific features, components or characteristics may be combined in any suitable manner. Therefore, without limitation, the specific features, components or characteristics shown or described in conjunction with one embodiment may be combined in whole or in part with the features, components or characteristics of one or more other embodiments. Such modifications and variations are also included within the scope of this application.

[0054] In the related technologies of digital human synthesis, the features of the sample character in the driving video can be partially migrated to the template character, so that the digital human template can perform the same expressions and actions as the sample character. The migration process is mostly realized with the help of the migration model that has been pre-set in the software and intelligent terminal memory, and the local features of the sample character to be migrated are transplanted to the template character.

[0055] The migration model can be obtained by repeated iterative training based on a large amount of sample character data with the help of neural network models and training parameters. For the local features to be migrated, different neural network models can be used and parameters can be adjusted to obtain a model with better adaptability. However, in the process of migrating some local features, the quality of the final model is also related to the sample image. For example, when migrating the head feature image, the large range of shoulder movement of the sample character will lead to inaccurate positioning of the head feature, which will in turn cause an imbalance in the head-shoulder ratio, and the corresponding facial features will not be able to be successfully migrated.

[0056] In view of the above problems, if Figure 1 As shown, some embodiments of the present application provide a model training method for head migration, including:

[0057] S100: Acquire a driving video, and extract a driving frame image from the driving video;

[0058] The driving video is the training sample of the head transfer model, and the driving video can come from a local database or a cloud database. The driving video includes sample human targets, so the driving frame image captured from the driving video also includes sample human targets. The sample human targets are used to extract local features and are input into the network model for training to obtain the final head transfer model.

[0059] S200: cropping the driving frame image according to the sample human target to obtain a local feature image containing the sample human target;

[0060] Because head migration is a migration of local features, local features that are less relevant to head migration can be omitted during training to reduce the amount of calculation, saving computing power while improving training efficiency. Therefore, in the driving frame image, the sample person target can be used as a reference to crop the driving frame image to obtain images related to head migration, such as head and shoulder feature images and facial feature images. For local features such as chest features and waist features below the shoulders, these invalid local features are filtered out during the cropping process.

[0061] The local feature images required for the head migration training model in the embodiment of the present application include a head and shoulder feature image and a facial feature image. The head feature image of the head and shoulder feature image and the facial feature image are the target images to be migrated, and the shoulder feature image of the head and shoulder feature image is used to adjust the scaling ratio of the facial feature image. In the driving frame image, when the shoulder image has a large degree of complete exposure of the shoulder, it is equivalent to the sample person being farther away from the lens when the image is taken, so the facial proportion will be smaller. In the process of training the model, restrictions can be set on the facial proportion according to the completeness of the shoulder exposure, and the facial proportion and the head proportion are also adapted to each other. Furthermore, when the trained model is actually applied, the migrated facial features and head features can be more integrated with the shoulder features of the template person, and the effect is more natural.

[0062] In order to obtain the local feature image, the cropping operation needs to be performed step by step, and the steps include:

[0063] Performing a cropping operation according to the sample character features to obtain the head and shoulder feature image;

[0064] A cropping operation is performed on the head and shoulder feature image to obtain the facial feature image.

[0065] The facial feature image is located in the head area of ​​the sample person. In the driving frame image, in order to facilitate the location of the head and shoulder area, it is necessary to rely on facial features to identify the head and shoulder area. After the facial features are identified, a head and shoulder feature image can be obtained by diffusing to a certain extent with the facial features as the center. Therefore, when cropping the head and shoulder feature image, it is necessary to first crop the driving frame image to obtain the head and shoulder feature image, and then obtain the facial feature image on the head area of ​​the head and shoulder feature image.

[0066] When cropping the head and shoulder features, a fixed cropping method is used. The fixed cropping method means that when cropping the driving frame image, each cropping of the head and shoulder feature image is performed in the form of equal height, equal length and equal width, so as to ensure that the proportional relationship between the shoulder features and the face will not be affected by the different size ratios of the cropped images. In addition, in the process of training images using the training network model, the proportional relationship between the shoulder features and the face features is maintained, which is conducive to the fusion of local features and helps to solve the problem of difficult recognition of head and face features due to excessive shoulder movement in the driving image.

[0067] After the head-shoulder feature image is obtained by cropping, the facial feature image can be extracted from the head region of the head-shoulder feature image. The step of performing a cropping operation on the head-shoulder feature image to obtain the facial feature image includes:

[0068] Identifying a facial region in the head and shoulders feature image;

[0069] Taking the face area as the center, setting the cropping offset ratio;

[0070] The head and shoulders feature image is cropped according to the cropping offset ratio to obtain the facial feature image.

[0071] To crop the face area, you need to locate the face area based on its common features, such as by using facial feature points, lines, and grayscale values ​​of some areas. During training, the face area should be completely captured, so a certain cropping offset ratio needs to be set. The offset ratio should not exceed the head area. With the face as the center, cropping offset ratios are set from the top, bottom, left, and right to appropriately expand the cropping range.

[0072] In the embodiment of the present application, the facial feature image and the head and shoulder feature image are respectively input into the training network model. Therefore, after the facial feature image is cropped, in order to train the positional relationship between the face and the head and shoulders, it is necessary to mark the cropped area in the head and shoulder feature image. The cropped area is usually marked by a mask. That is, the step of cropping the head and shoulder feature image according to the cropping offset ratio to obtain the facial feature image also includes:

[0073] Marking a cropped area in the head and shoulders feature image;

[0074] Adjusting the pixel value of the cropped area;

[0075] The facial image obtained by cropping the head and shoulder feature image is normalized to obtain the facial feature image.

[0076] In the embodiment of the present application, a mask is used to mark the cropped area, that is, the pixels of the cropped area are set to 0. The area set to 0 is visually blank, and in the recognition process of the training model, the cropped area can also be determined by identifying that the pixels of the cropped area are 0. In the training process, the cropped area and the facial feature area are repeatedly trained and learned.

[0077] For the facial image obtained by cropping the head and shoulder feature image, normalization processing can also be performed to obtain a facial feature image. Normalization processing will not affect the image, but will only scale the pixel values ​​by a certain proportion, for example, changing the pixel value from the range of 0-255 to the range of 0-1. It can speed up the convergence of the training network, improve the training efficiency, and prevent the situation where it is difficult to find the optimal solution during the training process due to excessive values.

[0078] S300: inputting the head and shoulder feature image and the facial feature image into a training network model respectively to obtain a first target image output by the training network model;

[0079] Among them, the first target image is the training result image; the training network model can be set with multiple input terminals for receiving different sample data. In the embodiment of the present application, the head and shoulder feature image and the facial feature image are respectively input into the training network model. Compared with only inputting the head and shoulder feature image into the training network model, the above training method can additionally input the facial feature image into the training network model, effectively extracting the facial area features and the head and shoulder area features, and can further utilize the relationship between the regional features. Input the two parts of the features of the training network model, and set restrictions on the facial features according to the shoulder features. At the same time, the proportional relationship between the facial features and the head features will also be subject to certain restrictions. Therefore, on the basis of each local feature being input separately to fully reflect their own characteristics, the degree of fusion between the local features will also be higher, and the fusion effect will be more natural.

[0080] After the head and shoulder feature image and the facial feature image are input into the training network model, the network model will fuse the feature images and obtain the first target image. The first target image refers to a phased image obtained during the training process. After the first target image is generated, the first target image can be evaluated. If the evaluation is passed, it means that the network training model and parameters corresponding to the first target image are appropriate, and the head migration model can be output according to the network training model and parameters corresponding to the first target image.

[0081] During the training process, the input sample image needs to be encoded to facilitate training of the sample image; after the training is completed, the target image is obtained by decoding, and the steps include:

[0082] Encoding the head-shoulder feature image and the facial feature image respectively to obtain an encoded image of the head-shoulder feature image and an encoded image of the facial feature image;

[0083] According to the training parameters of the training network, feature fusion is performed on the encoded image of the head and shoulder feature image and the encoded image of the facial feature image to obtain a feature fused image;

[0084] Decoding is performed on the feature fusion image to obtain the first target image.

[0085] like Figure 2 As shown in the figure, after the head and shoulder feature image and the facial feature image are input into the training network, they are encoded separately to fully extract their respective features and convert them into trainable data. The training network trains and fuses the encoded images of the head feature image and the facial feature image according to the training parameters. Finally, the encoded image is decoded to obtain the training image.

[0086] In some embodiments, the head and shoulder feature images and the facial feature images are input into the training network in the form of RGB three-channel input. Correspondingly, the network output is also in the form of RGB three-channel. After the head and shoulder feature images and the facial feature images are input, the images are encoded according to 4 layers of two-bit convolutional layers, and a 2x downsampling operation is performed during each convolution operation, which can reduce redundant operations in the training network. The use of a 9-layer residual structure can improve the depth of the training network, enhance the learning ability of the training network, and thereby improve the accuracy and efficiency of model training. The trained encoded image ultimately needs to be decoded to obtain a training image. A 4-layer bilinear interpolation method is used during decoding. The bilinear interpolation method is used to complete the upsampling process, that is, the image method. The training results corresponding to the method can reduce the probability of artifact effects in the training image and improve the image training quality.

[0087] S400: using a loss function to calculate the generation loss between the first target image and the standard image of the sample person; if the value of the generation loss is less than a loss threshold, outputting the training parameters of the current training network to obtain a trained head migration model.

[0088] The generation loss can be used to evaluate the quality of the first target image, indicating the difference between the first target image and the standard image. Therefore, a threshold can be set for the generation loss, and the threshold can be set to a proportional value or a fixed value according to the actual situation. When the generation loss is less than the loss threshold, it means that the difference between the first target image and the standard image is already very small, that is, the head transfer model can be output according to the network model and training parameters corresponding to the first target image for use.

[0089] Before calculating the generation loss, the clarity of the first target image can also be identified, that is, the clarity is also one of the constraints for training the network. The steps include:

[0090] Performing clarity recognition on the first target image to obtain a training resolution;

[0091] If the training resolution is less than the resolution threshold, the clarity of the first target image is improved.

[0092] like Figure 3As shown, the clarity of the first target image will affect the user's intuitive experience, so the clarity of the first target image can be trained by changing the parameters used to train the resolution in the training network. During the training process, it is difficult for the first target image to completely restore the standard image of the sample person. Therefore, in order to prevent the training network from training the target image clarity without limit, resulting in a waste of computing power, a resolution threshold can be set to limit the training process. It should be understood that according to the calculation examples of the training network and the improvement of software and hardware equipment, the theoretical value of the restoration degree of the first target image will continue to improve, and the corresponding resolution threshold can also be set higher to achieve a better training effect.

[0093] In some embodiments, the training resolution corresponding to the first target image is less than the resolution threshold, and the image is continuously trained according to the training resolution and the resolution threshold until the training resolution is not less than the resolution threshold, so that the clarity of the first target image reaches the expected level. After the generation loss and clarity of the first target image meet the threshold standard, the corresponding head transfer model can be output for subsequent application.

[0094] like Figure 4 As shown, after the head migration model is output, it can be put into use. The method provided in the embodiment of the present application can obtain the target image according to the template data and the target data by combining the head migration model after obtaining the head migration model:

[0095] Obtain the head and shoulder feature image of the template person and the face area image of the target person;

[0096] The head-shoulder region image and the face region image are input into the head migration model to obtain a second target image, where the second target image is an application image.

[0097] When training the head migration model in the embodiment of the present application, it is necessary to input the head and shoulder feature images and the facial feature images as sample data respectively to train the relationship between the head, shoulders and face. When the trained model is actually applied, the head and shoulder features are mostly provided by the template character. The head and shoulder features will restrict the proportion of the facial features to a certain extent, and then combined with the facial features that need to be migrated, the application image can be obtained.

[0098] In the actual migration process, it is equivalent to splicing the target person's facial area image with the template person's head and shoulder feature image. Therefore, the target person's facial area image and the template person's head and shoulder feature image are used as input data for actual application, and are input into the head migration model to obtain the second target image. The second target image is an application image with a generation loss lower than the loss threshold. The second target image is an image obtained by migrating a frame image in the driving video. If you want to obtain a video of continuous action, you can splice several second target images in chronological order to achieve the effect of the template task and the character in the driving video performing the same action.

[0099] It is difficult for the first target image to reach a better standard through one iteration. Relatively speaking, it is also difficult to obtain a better training model through one parameter setting. Therefore, the generation loss of the first target image is greater than / equal to the loss threshold. At this time, the head migration model training method provided in the embodiment of the present application also includes the following steps:

[0100] If the value of the generation loss is greater than the loss threshold, the training parameters of the training network model are updated according to the generation loss, and iterative training is performed to recalculate the generation loss.

[0101] In some embodiments, if the generation loss of the first target image is greater than the loss threshold, it means that the current training network model needs to continue training based on the sample image to obtain training parameters that better meet the expected requirements. The generation of the first target image is an iterative process, and the first target image does not simply refer to a stage image in a single iteration, but refers to a class of stage images obtained in the iteration process. Therefore, before the generation loss corresponding to the first target image is lower than the loss threshold, the training network model will iterate repeatedly and adjust the training parameters according to the generation loss until the survival loss of the first target image is lower than the loss threshold, and then the corresponding head migration model will be output.

[0102] The present application also provides a training system for a head migration model, including a preprocessing module, a training module and an application module.

[0103] The preprocessing module is used to obtain a driving video and extract a driving frame image from the driving video, wherein the driving frame image includes a sample person;

[0104] The preprocessing module is also used to preprocess the sample person to obtain a local feature image of the sample person, wherein the local feature image includes a head-shoulder feature image and a facial feature image; the head feature image of the head-shoulder feature image and the facial feature image are target images to be migrated, and the shoulder feature image of the head-shoulder feature image is used to adjust the scaling ratio of the facial feature image;

[0105] The training module is used to input the head and shoulder feature image and the facial feature image into a training network respectively to obtain a first target image, where the first target image is a training image;

[0106] The training module is also used to calculate the generation loss between the first target image and the standard image of the sample person using a loss function;

[0107] The training module is also used to output the training parameters of the current training network model when the value of the generated loss is less than the loss threshold, so as to obtain a trained head migration model.

[0108] The training module is also used to: when the value of the generation loss is greater than a loss threshold, update the training parameters of the training network model according to the generation loss, and perform iterative training to recalculate the generation loss.

[0109] In some embodiments, when the generation loss is greater than the loss threshold, the system will continue to train the model. And according to the generation loss, the training parameters of the network model are adjusted, and the iteration is repeated. Each iteration will continue to generate the target image, and continue to calculate the generation loss according to the target image, until the generation loss is less than the loss threshold, and the head transfer model can be obtained according to the current training parameters and the network model.

[0110] The application module is used to obtain a head and shoulder area image of a template person and a face area image of a target person;

[0111] The application module is further used to input the head-shoulder region portrait and the face region image into the head migration model to obtain a second target image, where the second target image is an application image.

[0112] In some embodiments, the application module is used to receive the feature data to be migrated. The feature data is input into the trained model to generate a target image composed of the facial features in the driving video and the head and shoulder features in the template character. Because the training model takes into account the scaling relationship between the facial features and the shoulder features in the head and shoulder features, the migrated facial features and the head and shoulders of the template character are better integrated.

[0113] The present application provides a model training method and system for head migration. The method obtains a sample person target in the driving frame image by obtaining a driving video and extracting a driving frame image from the driving video. The driving frame image is cropped based on the sample person target to obtain a local feature image of the sample person target. The local feature image includes a head and shoulder feature image and a facial feature image. The head and shoulder feature image includes a head feature image and a shoulder feature image, and the shoulder feature image is used to adjust the scaling ratio of the facial feature image. Then, the head and shoulder feature image and the facial feature image are respectively input into the training network so that the face, head and shoulders have a higher degree of fusion to obtain the target image. Finally, the generation loss is calculated according to the target image. When the generation loss is less than the loss threshold, the current training parameters can be output to obtain a trained head migration model.

[0114] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A training method for a head migration model, characterized in that: include: Acquire a driving video, and extract a driving frame image from the driving video, wherein the driving frame image includes a sample human target; The driving frame image is cropped according to the sample human target to obtain a local feature image containing the sample human target, wherein the local feature image includes a head-shoulder feature image and a facial feature image; the head feature image and the facial feature image of the head-shoulder feature image are target images to be migrated, and the shoulder feature image of the head-shoulder feature image is used to adjust the scaling ratio of the facial feature image; Inputting the head and shoulder feature image and the facial feature image into a training network model respectively to obtain a first target image output by the training network model, wherein the first target image is a training result image; Calculate the generation loss between the first target image and the standard image of the sample person using a loss function; If the value of the generated loss is less than the loss threshold, the training parameters of the current training network model are output to obtain a trained head migration model.

2. The method according to claim 1, characterized in that Also includes: If the value of the generation loss is greater than the loss threshold, the training parameters of the training network model are updated according to the generation loss, and iterative training is performed to recalculate the generation loss.

3. The method according to claim 1, characterized in that include: Obtain the head and shoulder feature image of the template person and the face area image of the target person; The head-shoulder feature image and the facial region image are input into the head migration model to obtain a second target image, where the second target image is an application image.

4. The method according to claim 1, characterized in that: When the driving frame image is cropped according to the sample human target to obtain a local feature image containing the sample human target, the method includes: Performing a cropping operation according to the sample character features to obtain the head and shoulder feature image; A cropping operation is performed on the head and shoulder feature image to obtain the facial feature image.

5. The method according to claim 4, characterized in that Performing a cropping operation on the head and shoulder feature image to obtain the facial feature image includes: Identifying a facial region in the head and shoulders feature image; Taking the face area as the center, setting the cropping offset ratio; The head and shoulders feature image is cropped according to the cropping offset ratio to obtain the facial feature image.

6. The method according to claim 5, characterized in that When the head and shoulder feature image is cropped according to the cropping offset ratio to obtain the facial feature image, the method further includes: Marking a cropped area in the head and shoulders feature image; Adjusting the pixel value of the cropped area; The facial image obtained by cropping the head and shoulder feature image is normalized to obtain the facial feature image.

7. The method according to claim 1, characterized in that Also includes: Performing clarity recognition on the first target image to obtain a training resolution; If the training resolution is less than the resolution threshold, the clarity of the first target image is improved.

8. The method according to claim 1, characterized in that After the head and shoulder feature image and the facial feature image are respectively input into the training network, the method further includes: Encoding the head-shoulder feature image and the facial feature image respectively to obtain an encoded image of the head-shoulder feature image and an encoded image of the facial feature image; According to the training parameters of the training network, feature fusion is performed on the encoded image of the head and shoulder feature image and the encoded image of the facial feature image to obtain a feature fused image; Decoding is performed on the feature fusion image to obtain the first target image.

9. A head migration model training system, characterized in that: include: Preprocessing module, training module; The preprocessing module is used to obtain a driving video and extract a driving frame image from the driving video, wherein the driving frame image includes a sample person; The preprocessing module is also used to preprocess the sample person to obtain a local feature image of the sample person, wherein the local feature image includes a head-shoulder feature image and a facial feature image; the head feature image of the head-shoulder feature image and the facial feature image are target images to be migrated, and the shoulder feature image of the head-shoulder feature image is used to adjust the scaling ratio of the facial feature image; The training module is used to input the head and shoulder feature image and the facial feature image into a training network respectively to obtain a first target image, where the first target image is a training image; The training module is also used to calculate the generation loss between the first target image and the standard image of the sample person using a loss function; The training module is also used to output the training parameters of the current training network model when the value of the generated loss is less than the loss threshold, so as to obtain a trained head migration model.

10. The system according to claim 9, characterized in that The training module is also used to: When the value of the generation loss is greater than the loss threshold, the training parameters of the training network model are updated according to the generation loss, and iterative training is performed to recalculate the generation loss.

11. The system according to claim 10, characterized in that Also includes: Application modules; The application module is used to obtain a head and shoulder area image of a template person and a face area image of a target person; The application module is also used to input the head-shoulder area portrait and the face area image into the head migration model to obtain a second target image, where the second target image is an application image.

Images