Method, device, storage medium and electronic device for beautifying virtual objects

Abstract

The present disclosure relates to a method, device, storage medium and electronic device for beautifying a virtual object, which can obtain preset initial parameters corresponding to a first virtual object to be beautified; the preset initial parameters are parameters used when generating the first virtual object; the beautification parameters corresponding to the first virtual object are generated according to the preset initial parameters through a pre-trained target parameter editing model; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, the target object includes a preset real object and a second virtual object; the beautified target virtual object is generated according to the beautification parameters through a pre-trained object generation model.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to a method, device, storage medium and electronic device for beautifying a virtual object. Background Art

[0002] In the related art, in order to meet the personalized needs of game players, a "face-pinching" system for game characters is provided, through which players can design the appearance of their own game characters. Specifically, hundreds of parameters can be manually adjusted in the face-pinching system to create a game character, and then the face-pinching system can render the game character's face corresponding to the parameters input by the player through the game engine.

[0003] However, the current face-shaping system cannot beautify the faces of game characters created by game players. The existing face attribute editing solutions are only applicable to real faces and cannot be directly applied to game character faces. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for beautifying a virtual object, the method comprising:

[0006] Acquire preset initial parameters corresponding to a first virtual object to be beautified, where the preset initial parameters are parameters used when generating the first virtual object;

[0007] Generate the beautification parameters corresponding to the first virtual object through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, and the target object includes a preset real object and a second virtual object;

[0008] A beautified target virtual object is generated according to the beautification parameters through a pre-trained object generation model.

[0009] In a second aspect, the present disclosure provides a device for beautifying a virtual object, the device comprising:

[0010] An acquisition module, used to acquire preset initial parameters corresponding to a first virtual object to be beautified, wherein the preset initial parameters are parameters used when generating the first virtual object;

[0011] A parameter generation module, configured to generate, according to the preset initial parameters, a beautification parameter corresponding to the first virtual object through a pre-trained target parameter editing model; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, the target object including a preset real object and a second virtual object;

[0012] A beautification module is used to generate a beautified target virtual object through a pre-trained object generation model according to the beautification parameters.

[0013] In a third aspect, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in the first aspect of the present disclosure are implemented.

[0014] In a fourth aspect, an electronic device is provided, including:

[0015] a storage device having a computer program stored thereon;

[0016] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in the first aspect of the present disclosure.

[0017] Through the above technical solution, the preset initial parameters corresponding to the first virtual object to be beautified are obtained, and the preset initial parameters are the parameters used when generating the first virtual object; the beautification parameters corresponding to the first virtual object are generated by the pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape structure characteristics of the target object, and the target object includes a preset real object and a second virtual object; the beautified target virtual object is generated by the pre-trained object generation model according to the beautification parameters, so that virtual objects such as game character faces can be beautified, and the user experience can be improved. In addition, the shape structure features with domain invariance are used to describe real objects and virtual objects, so that real objects with attribute labels can be used in the training of beautification learning of virtual objects, without the need to manually annotate good-looking virtual objects (such as game character faces), saving manpower and time costs.

[0018] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings:

[0020] Figure 1is a flow chart showing a method for beautifying a virtual object according to an exemplary embodiment;

[0021] Figure 2 is a schematic diagram showing a beautification result of a game face according to an exemplary embodiment;

[0022] Figure 3 is a flowchart of a method for training a target parameter editing model according to an exemplary embodiment;

[0023] Figure 4 is a flowchart of a method for training an object to generate a model according to an exemplary embodiment;

[0024] Figure 5 is a block diagram of a device for beautifying a virtual object according to an exemplary embodiment;

[0025] Figure 6 is based on Figure 5 The illustrated embodiment shows a block diagram of a device for beautifying a virtual object;

[0026] Figure 7 is based on Figure 6 The illustrated embodiment shows a block diagram of a device for beautifying a virtual object;

[0027] Figure 8 It is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0035] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0036] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0037] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0038] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0039] The present disclosure is mainly applied to the scene of beautifying virtual objects such as the face of game characters and the face of any virtual object. Taking the face of game characters as an example, game players can manually adjust hundreds of parameters through the face pinching system to create game characters, such as adjusting the bone point parameters of the eyes, nose, lips, face shape, etc. that control the face shape, as well as the appearance parameters such as hairstyle, eye makeup, lip makeup, etc. After that, the face pinching system can render through the game engine to obtain the game character face corresponding to the parameters input by the player, but the current face pinching system cannot beautify the game character face created by the game player.

[0040] In order to solve the above-mentioned problems, the present disclosure can obtain preset initial parameters corresponding to a first virtual object to be beautified; generate beautification parameters corresponding to the first virtual object through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape structure characteristics of the target object, and the target object includes a preset real object and a second virtual object. Afterwards, the beautified target virtual object can be generated through a pre-trained object generation model according to the beautification parameters, so that virtual objects such as game character faces can be beautified, thereby improving the user experience.

[0041] In addition, existing face attribute editing solutions are all aimed at modifying specific attributes of real faces, and train face attribute editors based on a real face database with manually marked attributes by using the generative adversarial network method. However, game character faces are virtual objects and do not have the attribute labels of real faces, so attribute modification and editing cannot be achieved. Manually adding attribute labels to the game character faces generated by the game engine will require a lot of manpower and time.

[0042] In the model training process of the existing face attribute editor, the generative adversarial network method is used for model training. The generative adversarial network mainly consists of two parts: a generator and a discriminator. During the training of the existing face attribute editor, the picture of a real face is input into the discriminator for judgment. The difference between the pictures of real faces is small. Therefore, the generator trained based on the generative adversarial network method can generate pictures that are difficult for the discriminator to distinguish between true and false, so that the pictures generated by the generator are similar in distribution to the target input pictures. However, the feature distribution of real faces and game character faces is obviously different. For example, the color of the game character face is relatively simple, the background is fixed, and the shape is different from that of the real face. Therefore, during the model training process, if the real face and the game character face output by the game engine are respectively input into the discriminator, the discriminator can easily identify whether the input is a real face or a game character face, so that the target generator cannot be trained. The target generator refers to a generator that can generate pictures that are difficult for the discriminator to distinguish between true and false.

[0043] In order to solve this technical problem, open source good-looking real objects (such as real faces) can be collected, and the shape structure features of virtual objects and real objects (such as face shape structure features) can be extracted using the existing object segmentation network. The shape structure features have domain invariance. Compared with color and texture features, the shape structure features of virtual objects and real objects have little feature difference. Therefore, in the process of pre-training the target parameter editing model, the present application can use the shape structure features as the input of the discriminator instead of inputting virtual objects and real objects with obvious feature differences into the discriminator. The beautified target virtual object generated based on the beautification parameters output by the target parameter editing model is made consistent with the distribution of the shape structure features of the good-looking real object by generating adversarial learning, thereby training the target parameter editing model that can generate beautification parameters, and then realizing the beautification of virtual objects such as game character faces according to the beautification parameters. That is, the present application uses shape structure features with domain invariance to describe real objects and virtual objects, so that real objects with attribute labels can be used in the training of beautification learning of virtual objects, without manually annotating good-looking virtual objects (such as game character faces), saving manpower and time costs.

[0044] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0045] Figure 1 is a flowchart of a method for beautifying a virtual object according to an exemplary embodiment. Figure 1 As shown, the method comprises the following steps:

[0046] In step S101, the preset initial parameters corresponding to the first virtual object to be beautified are obtained.

[0047] Among them, the first virtual object and the second virtual object mentioned in the present disclosure can include virtual objects such as the face of a game character, the face of any virtual object, etc., and the preset initial parameters are the parameters used to generate the first virtual object.

[0048] Taking the face of a game character as an example, in one possible implementation method, a game player can manually set the appearance parameters and bone point parameters of the game character to be generated (for example, the user can adjust the face pinching panel of the game interface to complete the setting of the preset initial parameters). The appearance parameters may include, for example, parameters such as hairstyle, eye makeup, and lip makeup. The bone point parameters may include, for example, parameters for controlling the shape of the face such as eyes, nose, lips, and face shape. The preset initial parameters include the appearance parameters and bone point parameters. In other words, the preset initial parameters are the generation parameters of the virtual object preset by the user.

[0049] In step S102, the beautification parameters corresponding to the first virtual object are generated according to the preset initial parameters through a pre-trained target parameter editing model.

[0050] Among them, the target parameter editing model is a model trained according to the shape structure characteristics of the target object, and the target object includes a preset real object and a second virtual object. It is assumed that the virtual object (including the first virtual object and the second virtual object) is a game character face, and the preset real object can be a real human face. The shape structure characteristics may include facial shape structure characteristics corresponding to the game character face and the real human face respectively, and the facial shape structure characteristics include facial contour, facial features and other features.

[0051] In one possible implementation, the real face and the game character face generated by the generator can be segmented by a preset face segmentation network to extract the face shape and structure features.

[0052] The target parameter editing model may include a fully connected network with a preset number of layers. The target parameter editing model is used to convert the preset initial parameters set by the user into beautification parameters corresponding to the first virtual object, so that the first virtual object can be beautified according to the beautification parameters. Taking the virtual object to be beautified as a game face as an example, the target parameter editing model can edit the game face parameters input by the user and output the beautified game face parameters. The dimensions of the input parameters and the output parameters can be the same as the dimensions of the game face parameters. The specific training process of the target parameter editing model will be introduced in other embodiments.

[0053] In this step, the preset initial parameters may be input into the fully connected network to obtain the beautification parameters.

[0054] In step S103, a beautified target virtual object is generated according to the beautification parameters through a pre-trained object generation model.

[0055] Among them, the object generation model includes a generator in a generative adversarial network, that is, in the present disclosure, a neural network-based generator can be used to imitate a game engine, the input of the generator is beautification parameters, and the output is a target virtual object rendered based on the beautification parameters.

[0056] In this step, the beautification parameters may be input into the generator and rendered to obtain the target virtual object.

[0057] For example, Figure 2 is a schematic diagram showing a beautification result of a game face according to an exemplary embodiment. Figure 2 As shown, the left picture is an unbeautified face of the gamer, and the right picture is a beautified face of the gamer based on the solution disclosed in the present invention. Obviously, the visual experience of the beautified face of the gamer is better.

[0058] By adopting the above method, virtual objects such as game character faces can be beautified to improve the user experience. In addition, the shape structure features with domain invariance are used to describe real objects and virtual objects, so that real objects with attribute labels can be used in the training of virtual object beautification learning, without the need to manually label good-looking virtual objects (such as game character faces), saving manpower and time costs.

[0059] The following is an explanation of the training process of the model involved in the above beautification method.

[0060] The target parameter editing model mentioned above is a model obtained by training the preset beautification model according to the first training sample. Figure 3 is a flowchart of a method for training a target parameter editing model according to an exemplary embodiment. Figure 3 As shown, the method comprises the following steps:

[0061] In step S301, a first training sample and a preset beautification model to be trained are obtained.

[0062] Among them, the preset beautification model includes a preset parameter editing model to be trained, the object generation model connected to the preset parameter editing model, a target segmentation network connected to the object generation model, and a preset discriminator to be trained connected to the target segmentation network; the first training sample includes a real image corresponding to a preset real object and initial sample parameters corresponding to the second virtual object, the initial sample parameters can be parameters pre-set by the user for generating the second virtual object, and the initial sample parameters can also include appearance parameters and bone point parameters of the second virtual object, etc.; the target segmentation network is used to extract shape structure features of a target object, and the target object includes the preset real object and the second virtual object.

[0063] Here, the preset parameter editing model can be a fully connected network with a preset number of layers, for example, it can be one fully connected layer. The object generation model is the generator. The model structure can refer to the model structure of stylegan. In the present disclosure, the generator consists of two networks: a mapping network and a synthesis network.

[0064] Taking the game face as an example, the process of generating the game face through the generator is explained below. In a possible implementation, the mapping network can be composed of eight fully connected layers, the input vector is the game face parameter z, the output vector is w, and the output vector w is used as the input of the synthesis network. The synthesis network can include multiple sub-modules connected in sequence, each sub-module is used to output a synthesized image of different resolutions, and output a target image of the resolution required by the user through the last sub-module. The target image is the image corresponding to the game face corresponding to the game face parameter z. For example, assuming that the resolution of the target image to be generated is 1024*1024, the synthesis network can be composed of 9 sub-modules, each sub-module corresponds to the resolution of a feature vector, and the resolution increases from 4*4, 8*8 to 1024*1024 in sequence.

[0065] The model structure of the preset discriminator can also refer to stylegan. For example, the preset discriminator can be composed of 8 downsampling layers, 17 convolutional layers and 2 fully connected layers.

[0066] In this way, taking the virtual object as a game face as an example, the first training sample may include a real face image and initial sample parameters required to generate a game face, the initial sample parameters include appearance parameters and bone point parameters of the game character, the target segmentation network may include a preset face segmentation network, which is used to perform face segmentation on the real face and the game face generated by the generator, and then extract the face shape structure features, the face shape structure features include the face contour, facial features and other features, the model training and model structure of the face segmentation network can adopt the open source model BiSegNet, for example.

[0067] The present invention uses an adversarial learning method in the process of model training, and performs adversarial learning on a preset parameter editing model and a preset discriminator respectively, so as to train and obtain the target parameter editing model that can generate virtual object beautification parameters.

[0068] It should be noted that, considering that the role of the generator (i.e., the object generation model) is to imitate the rendering process of the game engine to generate pictures corresponding to virtual objects, it can be trained separately. The target segmentation network is used to extract the shape and structural features of the target object and can also be trained separately. In other words, the (i.e., the generator) and target segmentation network in the preset beautification model are usually pre-trained separately and do not participate in the training of the target parameter editing model and the discriminator. The trained object generation model and segmentation network keep the network parameters fixed and are used to train the target parameter editing model and the discriminator.

[0069] In step S302, a first loss function corresponding to the preset parameter editing model is determined according to the initial sample parameters and the preset beautification model.

[0070] In this step, the initial sample parameters can be input into the preset parameter editing model to obtain beautified sample parameters; the beautified sample parameters can be input into the object generation model to obtain a second virtual object corresponding to the beautified sample parameters; the second virtual object can be input into the target segmentation network to obtain a first shape structure feature corresponding to the second virtual object; the first shape structure feature can be input into the preset discriminator to obtain a first prediction value, which is a probability for characterizing the recognition of the second virtual object as a real object; and the first loss function can be determined based on the first prediction value.

[0071] Here, a target segmentation network is used to extract the shape structure features of the second virtual object, so that a target generator can be trained based on the shape structure features with domain invariance. The target generator refers to a generator that can generate pictures that are difficult for the discriminator to distinguish between true and false.

[0072] The process of training the preset parameter editing model is to conduct mutual adversarial training with the preset discriminator. The generator inputs the beautification parameters output by the preset parameter editing model. The training goal is to enable the generator to generate segmentation results that are difficult for the discriminator to judge as virtual objects, so that the generated picture has good-looking facial features. Therefore, the first loss function corresponding to the preset parameter editing model can be defined as the loss of the discriminator judging that the segmentation result of the picture generated by the generator is the segmentation result of the virtual object. Therefore, the first prediction value can be determined as the first loss function. For example, the first loss function can be expressed as L_g=D(S_f), where D(S_f) represents the first prediction value.

[0073] In step S303, a second loss function corresponding to the preset discriminator is determined according to the real image, the initial sample parameters and the preset beautification model.

[0074] In this step, the real image can be input into the target segmentation network to obtain the second shape structure feature corresponding to the preset real object; the second shape structure feature can be input into the preset discriminator to obtain a second prediction value, which is a probability of characterizing the recognition of the preset real object as a real object; and the second loss function can be determined based on the first prediction value and the second prediction value.

[0075] For example, assuming that the first prediction value is represented by D(S_f) and the second prediction value is represented by D(S_r), the second loss function is L_d=(1+D(S_r)+(1–D(S_f).

[0076] In step S304, the preset parameter editing model and the preset discriminator are cyclically and alternately trained according to the first loss function and the second loss function until a training termination condition is met.

[0077] Among them, the training termination condition includes that the sum of the loss value of the first loss function and the loss value of the second loss function is less than or equal to a preset loss threshold.

[0078] In step S305, the target parameter editing model is determined according to the model parameters of the preset parameter editing model at the end of training.

[0079] During the model training process, the model parameters of the preset parameter editing model are continuously adjusted. Therefore, the model parameters of the preset parameter editing model at the end of the training can be used as the model parameters of the target parameter editing model.

[0080] By adopting the above method, in the process of pre-training the target parameter editing model, open source good-looking real objects (such as real human faces) can be collected, and the shape structure features of virtual objects and real objects (such as face shape structure features) can be extracted using the existing object segmentation network. The shape structure features with domain invariance are used to generate adversarial learning methods to make the beautified target virtual objects generated based on the beautification parameters output by the target parameter editing model tend to be consistent with the distribution of shape structure features of good-looking real objects, thereby training the target parameter editing model that can generate beautification parameters, and then realizing the beautification of virtual objects such as game character faces according to the beautification parameters. That is to say, the present application uses shape structure features with domain invariance to describe real objects and virtual objects, so that real objects with attribute labels can be used in the beautification learning training of virtual objects, without the need to manually label good-looking virtual objects (such as game character faces), saving manpower and time costs.

[0081] Figure 4 is a flowchart of a method for training an object to generate a model according to an exemplary embodiment. Figure 4 As shown, the method comprises the following steps:

[0082] In step S401, a second training sample and a preset generation model to be trained are obtained.

[0083] The second training sample includes a plurality of training sample pairs, each training sample pair includes a preset object parameter and a virtual sample object corresponding to the preset object parameter, and the virtual sample object is a virtual object generated by a preset engine according to the preset object parameter.

[0084] For example, continuing to take the virtual object as a game face, each training sample pair may include randomly set preset game face parameters and a game face corresponding to the preset game face parameters rendered by the game engine after the preset game face parameters are input into the game engine.

[0085] In step S402, for each of the training sample pairs, the preset object parameters in the training sample pair are input into the preset generation model to obtain a model output virtual object corresponding to the preset object parameters.

[0086] Among them, the preset generation model refers to the generator, and the model structure can also refer to the model structure of stylegan.

[0087] In step S403, a third loss function of the preset generation model is determined according to the virtual sample object and the model output virtual object.

[0088] For example, the virtual sample object can be recorded as I_z as a training label (ie, the true value), and the model output virtual object (ie, the virtual object predicted by the model) is recorded as G(z). In this way, the third loss function can be expressed as L=|G(z)–I_z|.

[0089] In step S404, the preset generation model is trained by a back propagation algorithm according to the third loss function to obtain the object generation model.

[0090] In this step, the object generation model can be trained by minimizing the third loss function and combining the stochastic gradient descent method to perform back propagation on the preset generation model and optimize the network parameters.

[0091] Figure 5 is a block diagram of a device for beautifying a virtual object according to an exemplary embodiment. Figure 5 As shown, the device comprises:

[0092] An acquisition module 501 is used to acquire preset initial parameters corresponding to a first virtual object to be beautified, where the preset initial parameters are parameters used when generating the first virtual object;

[0093] A parameter generation module 502 is used to generate the beautification parameters corresponding to the first virtual object according to the preset initial parameters through a pre-trained target parameter editing model; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, and the target object includes a preset real object and a second virtual object;

[0094] The beautification module 503 is used to generate a beautified target virtual object through a pre-trained object generation model according to the beautification parameters.

[0095] Optionally, the target parameter editing model includes a fully connected network with a preset number of layers, and the parameter generation module 502 is used to input the preset initial parameters into the fully connected network to obtain the beautification parameters.

[0096] Optionally, the object generation model includes a generator in a generative adversarial network, and the beautification module 503 is used to input the beautification parameters into the generator to render the target virtual object.

[0097] Optionally, the target parameter editing model is a model obtained by training a preset beautification model according to a first training sample, and the preset beautification model includes the preset parameter editing model to be trained, the object generation model connected to the preset parameter editing model, the target segmentation network connected to the object generation model, and the preset discriminator to be trained connected to the target segmentation network; the first training sample includes a real image corresponding to the preset real object and an initial sample parameter corresponding to the second virtual object.

[0098] Optionally, Figure 6 is based on Figure 5 The embodiment shown is a block diagram of a device for beautifying a virtual object, such as Figure 6 As shown, the device also includes: a first model training module 504,

[0099] The first model training module 504 is used to pre-train the target parameter editing model in the following manner:

[0100] Obtaining a first training sample and a preset beautification model to be trained;

[0101] Determine a first loss function corresponding to the preset parameter editing model according to the initial sample parameters and the preset beautification model;

[0102] Determine a second loss function corresponding to the preset discriminator according to the real image, the initial sample parameters and the preset beautification model;

[0103] The preset parameter editing model and the preset discriminator are cyclically and alternately trained according to the first loss function and the second loss function until a training termination condition is met, wherein the training termination condition includes that the sum of the loss value of the first loss function and the loss value of the second loss function is less than or equal to a preset loss threshold;

[0104] The target parameter editing model is determined according to the model parameters of the preset parameter editing model at the end of training.

[0105] Optionally, the shape structure feature includes a first shape structure feature, and the first model training module 504 is used to input the initial sample parameters into the preset parameter editing model to obtain beautified sample parameters;

[0106] Inputting the beautified sample parameters into the object generation model to obtain a second virtual object corresponding to the beautified sample parameters;

[0107] Inputting the second virtual object into the target segmentation network to obtain a first shape structure feature corresponding to the second virtual object;

[0108] Inputting the first shape structure feature into the preset discriminator to obtain a first prediction value, wherein the first prediction value represents a probability of identifying the second virtual object as a real object;

[0109] The first loss function is determined according to the first prediction value.

[0110] Optionally, the shape structure feature includes a second shape structure feature, and the first model training module 504 is used to input the real image into the target segmentation network to obtain the second shape structure feature corresponding to the preset real object;

[0111] Inputting the second shape structure feature into the preset discriminator to obtain a second prediction value, wherein the second prediction value represents the probability of identifying the preset real object as a real object;

[0112] The second loss function is determined according to the first predicted value and the second predicted value.

[0113] Optionally, Figure 7 is based on Figure 6 The embodiment shown is a block diagram of a device for beautifying a virtual object, such as Figure 7 As shown, the device also includes: a second model training module 505,

[0114] The second model training module 505 is used to pre-train the object generation model in the following manner:

[0115] Acquire a second training sample and a preset generation model to be trained; the second training sample includes a plurality of training sample pairs, each training sample pair includes a preset object parameter and a virtual sample object corresponding to the preset object parameter, the virtual sample object is a virtual object generated by a preset engine according to the preset object parameter;

[0116] For each of the training sample pairs, inputting the preset object parameters in the training sample pair into the preset generation model to obtain a model output virtual object corresponding to the preset object parameters;

[0117] Determine a third loss function of the preset generation model according to the virtual sample object and the model output virtual object;

[0118] The preset generation model is trained by a back propagation algorithm according to the third loss function to obtain the object generation model.

[0119] By using the above-mentioned device, preset initial parameters corresponding to the first virtual object to be beautified can be obtained; the beautification parameters corresponding to the first virtual object can be generated through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape structure characteristics of the target object, and the target object includes a preset real object and a second virtual object; the beautified target virtual object is generated through a pre-trained object generation model according to the beautification parameters, so that virtual objects such as game character faces can be beautified, thereby improving the user experience.

[0120] In addition, in the process of pre-training the target parameter editing model, open source good-looking real objects (such as real human faces) can be collected, and the shape structure features of virtual objects and real objects (such as face shape structure features) can be extracted using the existing object segmentation network. The shape structure features with domain invariance are used to generate adversarial learning methods to make the beautified target virtual objects generated based on the beautification parameters output by the target parameter editing model tend to be consistent with the distribution of shape structure features of good-looking real objects, thereby training the target parameter editing model that can generate beautification parameters, and then realizing the beautification of virtual objects such as game character faces according to the beautification parameters. In other words, the present application uses shape structure features with domain invariance to describe real objects and virtual objects, so that real objects with attribute labels can be used in the beautification learning training of virtual objects, without the need to manually label good-looking virtual objects (such as game character faces), saving manpower and time costs.

[0121] Reference below Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 8As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0123] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0124] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0125] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0126] In some embodiments, the client can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0127] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0128] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains preset initial parameters corresponding to the first virtual object to be beautified; the preset initial parameters are the parameters used when generating the first virtual object; generates beautification parameters corresponding to the first virtual object through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, and the target object includes a preset real object and a second virtual object; generates the beautified target virtual object through a pre-trained object generation model according to the beautification parameters.

[0129] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a module does not limit the module itself in some cases. For example, an acquisition module may also be described as a "module for acquiring preset initial parameters".

[0132] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] According to one or more embodiments of the present disclosure, Example 1 provides a method for beautifying a virtual object, including: obtaining preset initial parameters corresponding to a first virtual object to be beautified; the preset initial parameters are parameters used when generating the first virtual object; generating beautification parameters corresponding to the first virtual object through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, the target object including a preset real object and a second virtual object; generating a beautified target virtual object through a pre-trained object generation model according to the beautification parameters.

[0135] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the target parameter editing model includes a fully connected network with a preset number of layers, and generating the beautification parameters corresponding to the first virtual object through the pre-trained target parameter editing model according to the preset initial parameters includes:

[0136] The preset initial parameters are input into the fully connected network to obtain the beautification parameters.

[0137] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 1, wherein the object generation model includes a generator in a generative adversarial network, and generating a beautified target virtual object through a pre-trained object generation model according to the beautification parameter includes:

[0138] The beautification parameters are input into the generator, and the target virtual object is obtained by rendering.

[0139] According to one or more embodiments of the present disclosure, Example 4 provides the method described in any one of Examples 1-3, wherein the target parameter editing model is a model obtained after training a preset beautification model according to a first training sample, and the preset beautification model includes a preset parameter editing model to be trained, the object generation model connected to the preset parameter editing model, a target segmentation network connected to the object generation model, and a preset discriminator to be trained connected to the target segmentation network; the first training sample includes a real image corresponding to the preset real object and an initial sample parameter corresponding to the second virtual object.

[0140] According to one or more embodiments of the present disclosure, Example 5 provides the method described in Example 4, wherein the target parameter editing model is pre-trained in the following manner:

[0141] Acquire the first training sample and the preset beautification model to be trained;

[0142] Determine a first loss function corresponding to the preset parameter editing model according to the initial sample parameters and the preset beautification model;

[0143] Determine a second loss function corresponding to the preset discriminator according to the real image, the initial sample parameters and the preset beautification model;

[0144] The preset parameter editing model and the preset discriminator are cyclically and alternately trained according to the first loss function and the second loss function until a training termination condition is met, wherein the training termination condition includes that the sum of the loss value of the first loss function and the loss value of the second loss function is less than or equal to a preset loss threshold;

[0145] The target parameter editing model is determined according to the model parameters of the preset parameter editing model at the end of training.

[0146] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 5, wherein the shape structure feature includes a first shape structure feature, and determining the first loss function corresponding to the preset parameter editing model according to the initial sample parameter and the preset beautification model includes:

[0147] Inputting the initial sample parameters into the preset parameter editing model to obtain beautified sample parameters;

[0148] Inputting the beautified sample parameters into the object generation model to obtain a second virtual object corresponding to the beautified sample parameters;

[0149] Inputting the second virtual object into the target segmentation network to obtain the first shape structure feature corresponding to the second virtual object;

[0150] Inputting the first shape structure feature into the preset discriminator to obtain a first prediction value, wherein the first prediction value represents a probability of identifying the second virtual object as a real object;

[0151] The first loss function is determined according to the first prediction value.

[0152] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 6, wherein the shape structure feature includes a second shape structure feature, and determining the second loss function corresponding to the preset discriminator according to the real image, the initial sample parameter, and the preset beautification model includes:

[0153] Inputting the real image into the target segmentation network to obtain the second shape structure feature corresponding to the preset real object;

[0154] Inputting the second shape structure feature into the preset discriminator to obtain a second prediction value, wherein the second prediction value represents the probability of identifying the preset real object as a real object;

[0155] The second loss function is determined according to the first predicted value and the second predicted value.

[0156] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 4, wherein the object generation model is trained in the following manner:

[0157] Acquire a second training sample and a preset generation model to be trained; the second training sample includes a plurality of training sample pairs, each training sample pair includes a preset object parameter and a virtual sample object corresponding to the preset object parameter, the virtual sample object is a virtual object generated by a preset engine according to the preset object parameter;

[0158] For each of the training sample pairs, inputting the preset object parameters in the training sample pair into the preset generation model to obtain a model output virtual object corresponding to the preset object parameters;

[0159] Determine a third loss function of the preset generation model according to the virtual sample object and the model output virtual object;

[0160] The preset generation model is trained by a back propagation algorithm according to the third loss function to obtain the object generation model.

[0161] According to one or more embodiments of the present disclosure, Example 9 provides a device for beautifying a virtual object, including:

[0162] An acquisition module, used to acquire preset initial parameters corresponding to a first virtual object to be beautified, wherein the preset initial parameters are parameters used when generating the first virtual object;

[0163] A parameter generation module, configured to generate, according to the preset initial parameters, a beautification parameter corresponding to the first virtual object through a pre-trained target parameter editing model; the target parameter editing model is a model trained according to the shape and structure characteristics of the target object, the target object including a preset real object and a second virtual object;

[0164] A beautification module is used to generate a beautified target virtual object through a pre-trained object generation model according to the beautification parameters.

[0165] According to one or more embodiments of the present disclosure, Example 10 provides a computer-readable medium having a computer program stored thereon, which implements the steps of any of the methods described in Examples 1-8 when executed by a processing device.

[0166] According to one or more embodiments of the present disclosure, Example 11 provides an electronic device, including:

[0167] a storage device having a computer program stored thereon;

[0168] A processing device is used to execute the computer program in the storage device to implement the steps of any one of the methods described in Examples 1-8.

[0169] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0170] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0171] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A method for beautifying a virtual object, characterized in that: The method comprises: Acquire preset initial parameters corresponding to a first virtual object to be beautified; the preset initial parameters are parameters preset by a user for use in generating the first virtual object; the first virtual object includes a face of a game character to be beautified; Generate the beautification parameters corresponding to the first virtual object through the pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape structure features of the target object, the target object includes a preset real object and a second virtual object; wherein the second virtual object includes the face of any virtual object; the shape structure features include the first shape structure features corresponding to the second virtual object and the second shape structure features corresponding to the preset real object; the second virtual object includes the face of any game character, the preset real object includes a real human face, and the shape structure features include face shape structure features; the face shape structure features are obtained after performing face segmentation on the target object through a preset face segmentation network; Generate a beautified target virtual object through a pre-trained object generation model according to the beautification parameters; Among them, the target parameter editing model is a model obtained after training according to a first training sample, the first training sample includes a real image corresponding to the preset real object and initial sample parameters corresponding to the second virtual object, and the initial sample parameters include parameters pre-set by the user for generating the second virtual object.

2. The method according to claim 1, characterized in that The target parameter editing model includes a fully connected network with a preset number of layers, and generating the beautification parameters corresponding to the first virtual object through the pre-trained target parameter editing model according to the preset initial parameters includes: The preset initial parameters are input into the fully connected network to obtain the beautification parameters.

3. The method according to claim 1, characterized in that The object generation model includes a generator in a generative adversarial network, and generating a beautified target virtual object through a pre-trained object generation model according to the beautification parameters includes: The beautification parameters are input into the generator, and the target virtual object is obtained by rendering.

4. The method according to any one of claims 1 to 3, characterized in that: The target parameter editing model is a model obtained after training a preset beautification model according to the first training sample. The preset beautification model includes a preset parameter editing model to be trained, the object generation model connected to the preset parameter editing model, a target segmentation network connected to the object generation model, and a preset discriminator to be trained connected to the target segmentation network.

5. The method according to claim 4, characterized in that The target parameter editing model is pre-trained in the following way: Acquire the first training sample and the preset beautification model to be trained; Determine a first loss function corresponding to the preset parameter editing model according to the initial sample parameters and the preset beautification model; Determine a second loss function corresponding to the preset discriminator according to the real image, the initial sample parameters and the preset beautification model; The preset parameter editing model and the preset discriminator are cyclically and alternately trained according to the first loss function and the second loss function until a training termination condition is met, wherein the training termination condition includes that the sum of the loss value of the first loss function and the loss value of the second loss function is less than or equal to a preset loss threshold; The target parameter editing model is determined according to the model parameters of the preset parameter editing model at the end of training.

6. The method according to claim 5, characterized in that Determining the first loss function corresponding to the preset parameter editing model according to the initial sample parameters and the preset beautification model includes: Inputting the initial sample parameters into the preset parameter editing model to obtain beautified sample parameters; Inputting the beautified sample parameters into the object generation model to obtain a second virtual object corresponding to the beautified sample parameters; Inputting the second virtual object into the target segmentation network to obtain the first shape structure feature corresponding to the second virtual object; Inputting the first shape structure feature into the preset discriminator to obtain a first prediction value, wherein the first prediction value represents a probability of identifying the second virtual object as a real object; The first loss function is determined according to the first prediction value.

7. The method according to claim 6, characterized in that The determining of the second loss function corresponding to the preset discriminator according to the real image, the initial sample parameters and the preset beautification model comprises: Inputting the real image into the target segmentation network to obtain the second shape structure feature corresponding to the preset real object; Inputting the second shape structure feature into the preset discriminator to obtain a second prediction value, wherein the second prediction value represents the probability of identifying the preset real object as a real object; The second loss function is determined according to the first predicted value and the second predicted value.

8. The method according to claim 4, characterized in that The object generation model is trained in the following way: Acquire a second training sample and a preset generation model to be trained; the second training sample includes a plurality of training sample pairs, each training sample pair includes a preset object parameter and a virtual sample object corresponding to the preset object parameter, the virtual sample object is a virtual object generated by a preset engine according to the preset object parameter; For each of the training sample pairs, inputting the preset object parameters in the training sample pair into the preset generation model to obtain a model output virtual object corresponding to the preset object parameters; Determine a third loss function of the preset generation model according to the virtual sample object and the model output virtual object; The preset generation model is trained by a back propagation algorithm according to the third loss function to obtain the object generation model.

9. A device for beautifying a virtual object, characterized in that: The device comprises: An acquisition module, used to acquire preset initial parameters corresponding to a first virtual object to be beautified, wherein the preset initial parameters are parameters preset by a user for generating the first virtual object; the first virtual object includes a face of a game character to be beautified; A parameter generation module is used to generate the beautification parameters corresponding to the first virtual object through a pre-trained target parameter editing model according to the preset initial parameters; the target parameter editing model is a model trained according to the shape structure characteristics of the target object, the target object includes a preset real object and a second virtual object; wherein the second virtual object includes the face of any virtual object; the shape structure characteristics include a first shape structure characteristic corresponding to the second virtual object and a second shape structure characteristic corresponding to the preset real object; the second virtual object includes any game character face, the preset real object includes a real human face, and the shape structure characteristics include a human face shape structure characteristic; after performing face segmentation on the target object through a preset human face segmentation network, the human face shape structure characteristic is obtained; A beautification module, used for generating a beautified target virtual object through a pre-trained object generation model according to the beautification parameters; Among them, the target parameter editing model is a model obtained after training according to a first training sample, the first training sample includes a real image corresponding to the preset real object and initial sample parameters corresponding to the second virtual object, and the initial sample parameters include parameters pre-set by the user for generating the second virtual object.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processing device, the steps of the method described in any one of claims 1 to 8 are implemented.

11. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 8.

Images