Identity feature representation model training method, image generation method and device

Abstract

The application provides an identity feature representation model training method, an image generation method and device, and relates to the technical field of artificial intelligence. The model training method comprises: obtaining a training sample set, comprising at least one image sample of a first object and a label corresponding to the image sample; inputting a first image sample in the at least one image sample into a first feature extraction network to obtain an intermediate feature of the first image sample; inputting the intermediate feature into a reconstruction network to obtain a reconstructed image; inputting the intermediate feature into a mapping layer to obtain an identity feature representation of the first object, and inputting the identity feature representation into a classifier to obtain an identity identification category probability corresponding to the first image sample; and updating model parameters according to the reconstructed image, the identity identification category probability and the label corresponding to the first image sample to obtain a trained identity feature representation model. The embodiment of the application can extract features more suitable for an image generation task, which is beneficial to improving the effect of the image generation task.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an identity feature representation model training method, an image generation method, and an apparatus. Background Technology

[0002] With the development of generative techniques from Generative Adversarial Networks (GANs) to Stable Diffusion models, more and more face generation methods have emerged, such as video face swapping, input signal (such as audio, facial expressions, text, or video) driven methods, and digital human speech generation. The core goal of face swapping is to make the output face-swapped result as similar as possible to the input source image.

[0003] Current solutions often rely on a pre-trained face recognition model to extract facial identity features and ensure that the features of the swapped face match those of the source face. However, the face recognition model used here is optimized for face recognition tasks and cannot be fully adapted to existing face generation tasks. Therefore, how to further improve the performance of face generation tasks is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method for training an identity feature representation model, an image generation method, and an apparatus, which can extract features that are more suitable for image generation tasks, thereby improving the performance of image generation tasks.

[0005] In a first aspect, embodiments of this application provide a method for training an identity feature representation model, including:

[0006] Obtain a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample;

[0007] The first image sample from the at least one image sample is input into the first feature extraction network to obtain the intermediate features of the first image sample;

[0008] The intermediate features are input into the reconstruction network to obtain the reconstructed image of the first object;

[0009] The intermediate features are input into the mapping layer to obtain the identity feature representation of the first object, and the identity feature representation is input into the classifier to obtain the identity category probability corresponding to the first image sample;

[0010] Based on the reconstructed image, the identity category probability, and the label corresponding to the first image sample, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier are updated to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0011] Secondly, embodiments of this application provide an image generation method, including:

[0012] Obtain the source image of the target object;

[0013] The source image is input into the identity feature representation model, and the intermediate features of the source image are obtained by using the first feature extraction network; and the intermediate features are input into the mapping layer to obtain the identity feature representation of the source image.

[0014] Based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image;

[0015] The identity feature representation model includes the first feature extraction network and the mapping layer, and the identity feature representation model is trained according to the method described in the first aspect.

[0016] Thirdly, a training device for an identity feature representation model is provided, including:

[0017] An acquisition unit is used to acquire a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample;

[0018] A first feature extraction network is used to take a first image sample from the at least one image sample as input and obtain intermediate features of the first image sample.

[0019] A reconstruction network is used to input the intermediate feature inputs to obtain a reconstructed image of the first object;

[0020] A mapping layer is used to input the intermediate features and obtain the identity feature representation of the first object;

[0021] A classifier is used to take the identity feature representation as input and obtain the probability of the identity category corresponding to the first image sample;

[0022] The parameter update unit is used to update the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier according to the reconstructed image, the identity category probability, and the label corresponding to the first image sample, so as to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0023] Fourthly, an image generation apparatus is provided, comprising:

[0024] The acquisition unit is used to acquire the source image of the target object;

[0025] An identity feature representation model is used to input the source image, use a first feature extraction network to obtain intermediate features of the source image, and input the intermediate features into a mapping layer to obtain the identity feature representation of the source image.

[0026] The generation unit is used to replace the object in the template image with the target object in the source image according to the identity feature representation, and generate a target image;

[0027] The identity feature representation model includes the first feature extraction network and the mapping layer, and the identity feature representation model is trained according to the method described in the first aspect.

[0028] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in the first aspect, or the method as described in the second aspect.

[0029] In a sixth aspect, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the method as described in the first aspect or the method as described in the second aspect.

[0030] In a seventh aspect, embodiments of this application provide a computer program product including computer program instructions that cause a computer to perform the method as described in the first aspect or the method as described in the second aspect.

[0031] Eighthly, embodiments of this application provide a computer program that causes a computer to perform the method as described in the first aspect or the method as described in the second aspect.

[0032] The above technical solution extracts intermediate features from a first image sample using a first feature extraction network, inputs these intermediate features into a mapping layer and a classifier to obtain the probability of the identity category corresponding to the first image sample, and obtains a reconstructed image of the first image sample of the first object based on the intermediate features of the first image sample. Then, the model parameters are updated based on the reconstructed image, the identity category probability, and the label of the first image sample, so that the model can extract features for representing identity from the image. These features have the ability to reconstruct the image, so the features extracted by the model can be more adapted to the image generation task, which is conducive to improving the effect of the image generation task. Attached Figure Description

[0033] Figure 1A This is a schematic diagram illustrating an application scenario of an embodiment of this application;

[0034] Figure 1B This is another schematic diagram illustrating an application scenario of an embodiment of this application;

[0035] Figure 2 A schematic diagram of a system architecture provided for an embodiment of this application;

[0036] Figure 3 A schematic flowchart illustrating a training method for an identity feature representation model provided in an embodiment of this application;

[0037] Figure 4 A schematic flowchart illustrating another identity feature representation model training method provided in this application embodiment;

[0038] Figure 5 A schematic flowchart illustrating another identity feature representation model training method provided in this application embodiment;

[0039] Figure 6 A schematic flowchart illustrating an image generation method provided in an embodiment of this application;

[0040] Figure 7 A schematic diagram illustrating the image generation task provided in an embodiment of this application;

[0041] Figure 8 A schematic flowchart of an image detection model training device provided in an embodiment of this application;

[0042] Figure 9 A schematic block diagram of an image detection device provided in an embodiment of this application;

[0043] Figure 10 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. In the description of this application, unless otherwise stated, "at least one" means one or more, and "multiple" means two or more. In addition, "and / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following associated objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0046] It should also be understood that the descriptions such as "first," "second," etc., appearing in the embodiments of this application are only for illustration and to distinguish the objects being described, and have no order, nor do they indicate a particular limitation on the number of devices in the embodiments of this application, and cannot constitute any limitation on the embodiments of this application. It should also be understood that specific features, structures, or characteristics related to embodiments in the specification are included in at least one embodiment of this application. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In addition, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0047] First, the relevant terms involved in the embodiments of this application will be described.

[0048] 1. Facial Identity Features: These are feature vectors that represent the identity of a face, also known as facial recognition features or IDs. They are typically obtained by extracting features from a facial image using facial recognition algorithms. Facial identity features can be used to distinguish between different individuals.

[0049] This application does not limit the method of facial identity feature extraction. For example, it can be obtained through deep learning methods, such as using convolutional neural networks (CNN), deep residual networks (ResNet), or the ArcFace algorithm.

[0050] 2. Face generation tasks: These refer to technologies that generate or modify facial images or videos using algorithms and models, and are widely used in entertainment, film and television production, virtual reality, and other fields. Generally, face-swapping technology and related algorithms can be used to generate videos or images of people.

[0051] Face-swapping technology replaces a human face in an input source image with a template face in a template image, while maintaining the expression, angle, background, and other information of the template face in the output fake face.

[0052] 3. Three-dimensional (3D) face parametric model: This is a mathematical model used to model and reconstruct three-dimensional faces. It uses a set of parameters to represent information such as the shape, texture, expression, pose, and lighting of the face. These parameters can be used to render realistic three-dimensional face images.

[0053] Current solutions often rely on a pre-trained face recognition model to extract facial identity features and ensure that the features of the swapped face match those of the source image. However, the face recognition model used here is optimized for face recognition tasks and cannot be fully adapted to existing face generation tasks. Therefore, how to further improve the performance of face generation tasks is an urgent problem to be solved.

[0054] In view of this, embodiments of this application provide an identity feature representation model training method, an image generation method, and an apparatus, which can extract features that are more suitable for image generation tasks, thereby improving the performance of image generation tasks.

[0055] Specifically, a training sample set can be obtained, which includes at least one image sample of a first object and a label corresponding to the image sample. The label is used to indicate the identity identifier corresponding to the image sample. The first image sample from the at least one image sample is input into a first feature extraction network to obtain intermediate features of the first image sample. The intermediate features are input into a reconstruction network to obtain a reconstructed image of the first object. The intermediate features are input into a mapping layer to obtain an identity feature representation of the first object. The identity feature representation is input into a classifier to obtain the identity identifier category probability corresponding to the first image sample. Based on the reconstructed image, the identity identifier recognition probability, and the label corresponding to the first image sample, at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier is updated to obtain a trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0056] This application embodiment extracts intermediate features of a first image sample using a first feature extraction network, inputs these intermediate features into a mapping layer and a classifier to obtain the identity category probability corresponding to the first image sample, and obtains a reconstructed image of the first image sample of the first object based on the intermediate features of the first image sample. Then, the model parameters are updated based on the reconstructed image, the identity category probability, and the label of the first image sample, so that the model can extract features for representing identity from the image. These features have the ability to reconstruct the image, so the features extracted by the model can be more adapted to the image generation task, which is beneficial to improving the effect of the image generation task.

[0057] In the image generation method, a source image of the target object can be obtained, and this source image can be input into an identity feature representation model. A first feature extraction network is used to obtain intermediate features of the source image, and these intermediate features are then input into a mapping layer to obtain the identity feature representation of the source image. Based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image. In this embodiment, the identity feature representation extracted by the model has image reconstruction capabilities; therefore, the target image generated in this embodiment can achieve better results.

[0058] This application's embodiments can be used in face generation tasks, such as image / video face swapping, audio, expression, text, or video-driven face swapping scenarios. Face swapping has numerous applications, including but not limited to film and television portrait production, game character design, virtual avatars, and privacy protection. The core objective of face swapping is to make the output face-swapped result as similar as possible to the input source image. In this application's embodiments, facial identity features can be extracted from the source image based on a trained identity feature representation model, and then the face identity features after face swapping are constrained to be consistent with the facial identity features of the source image. These facial identity features possess face reconstruction capabilities, thus better adapting to face generation tasks. Therefore, by replacing the facial recognition features in related solutions with the new facial identity features provided in this application's embodiments, the performance of the face generation task can be further improved.

[0059] Figure 1A A schematic diagram illustrating an application scenario of an embodiment of this application is shown.

[0060] like Figure 1A As shown, this application scenario involves server 1 and terminal device 2. Terminal device 2 can communicate with server 1 via a communication network. Server 1 can be the backend server for terminal device 2.

[0061] For example, terminal device 2 can refer to a type of device with rich human-computer interaction methods, internet access capabilities, typically running various operating systems, and possessing strong processing capabilities. Terminal device 2 can be a smartphone, tablet computer, laptop computer, desktop computer, wearable device, in-vehicle device, etc., but is not limited to these. Optionally, in this embodiment, terminal device 2 is equipped with an image generation program (Application, APP), or an APP with image generation functionality. For example, the image generation includes, but is not limited to, face generation, object image generation, etc.

[0062] Server 1 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. A server can also become a node in a blockchain.

[0063] Optionally, in this embodiment, server 1 can train the identity feature representation model, and the trained identity feature representation model can be stored on server 1. Alternatively, the server can send the trained identity feature representation model to terminal device 2; this embodiment does not limit this. Optionally, terminal device 2 can train and store the identity feature representation model.

[0064] There may be one or more servers. When there are multiple servers, at least two servers are used to provide different services, and / or at least two servers are used to provide the same service, such as providing the same service in a load-balanced manner. This application does not limit this.

[0065] Terminal devices and servers can be connected directly or indirectly via wired or wireless communication, and this application does not limit this. This application does not limit the number of servers or terminal devices. The solution provided in this application can be implemented independently by the terminal device, independently by the server, or jointly by the terminal device and the server; this application does not limit this.

[0066] It should be understood that Figure 1A This is merely an illustrative example and does not specifically limit the application scenarios of the embodiments in this application. For example, Figure 1A An exemplary terminal device and a server are shown, but in practice, other numbers of terminal devices and servers may be included, and this application does not limit this.

[0067] Figure 1BThis is a schematic diagram of an application scenario involved in an embodiment of this application.

[0068] like Figure 1B As shown, the system architecture may include user equipment 101, data acquisition equipment 102, training equipment 103, execution equipment 104, database 105, and content library 106.

[0069] The data acquisition device 102 is used to read training data from the content library 106 and store the read training data in the database 105. As an example, the training data may include at least one image sample of a first object and a label corresponding to the image sample, the label indicating the identity of the image sample. Optionally, the training data may also include at least one image sample of at least one object and a label corresponding to the image sample.

[0070] Training device 103 trains a machine learning model based on training data maintained in database 105. For example, the machine learning model may include an identity feature representation model. In this embodiment, the identity feature representation model includes a first feature extraction network and a mapping layer. For example, a first image sample from the at least one image sample can be input into the first feature extraction network to obtain intermediate features of the first image sample; the intermediate features can be input into a reconstruction network to obtain a reconstructed image of the first object; the intermediate features can be input into the mapping layer to obtain an identity feature representation of the first object; and the identity feature representation can be input into a classifier to obtain the identity category probability corresponding to the first image sample; based on the reconstructed image, the identity recognition probability, and the label corresponding to the first image sample, at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier is updated with parameters to obtain the trained identity feature representation model.

[0071] Optional, see reference Figure 1B The execution device 104 is equipped with an I / O interface 107 for data interaction with external devices, such as user device 101. The computing module 109 in the execution device 104 uses a trained identity feature representation model to replace the object in the template image with the target object input from the user device 101, and outputs the generated target image. Specifically, a first feature extraction network can be used to extract intermediate features from the source image, and these intermediate features can be input into a mapping layer to obtain the identity feature representation of the source image. Then, based on this identity feature representation, the object in the template image can be replaced with the target object in the source image to generate the target image.

[0072] Optionally, the execution device 104 can send the generated target image to the user device 101 via the I / O interface.

[0073] Optionally, the trained identity feature representation model can also be deployed on user device 101. The user device can input the source image of the template object into the identity feature representation model, and the model outputs the corresponding identity feature representation. Then, the image generation task is constrained according to the identity feature representation to obtain the target image.

[0074] The user equipment 101 may include a mobile phone, computer, intelligent voice interaction device, smart home appliance, vehicle terminal, aircraft, or other terminal device; this embodiment does not limit this. Optionally, the user equipment 101 may be... Figure 1A Terminal device 2 in the middle.

[0075] The execution device 104 can be a server. For example, the server can be a rack server, blade server, tower server, or cabinet server, etc. The server can be a standalone server or a server cluster composed of multiple servers; this embodiment does not limit this. Optionally, the execution device 104 can be... Figure 1A Server 1 in the middle.

[0076] In this embodiment, the execution device 104 is connected to the user equipment 101 via a network. The network can be an intranet, the Internet, the Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, voice communication network, or other wireless or wired networks.

[0077] It should be noted that, Figure 1B This is merely a schematic diagram of a system architecture provided in this application embodiment, and the positional relationships between the devices, components, modules, etc. shown in the figure do not constitute any limitation. In some embodiments, the data acquisition device 102, user device 101, training device 103, and execution device 104 may be the same device. The database 105 may be distributed across one server or multiple servers, and the content library 106 may be distributed across one server or multiple servers.

[0078] Figure 2 This is a schematic diagram of a system architecture for model training according to an embodiment of this application. Optionally, this system architecture can be configured in... Figure 1A In server 1 or terminal device 2, or configured in Figure 1B The present application does not limit the execution device 104 shown.

[0079] like Figure 2 As shown, the system architecture includes a feature extraction network 210, a reconstruction network 220, a mapping layer 230, and a classification layer 240. The feature extraction network 210 is used to extract intermediate features from an image sample of the object, such as a first image sample. The reconstruction network 220 is used to generate a reconstructed image from the intermediate features. The mapping layer is used to map the intermediate features to obtain an identity representation of the object. The classifier is used to obtain the identity category probability corresponding to the first image sample from this identity representation. In this embodiment, based on the reconstructed image, the identity category probability, and the label of the image sample, the parameters of the model can be updated. For example, the parameters of at least one of the modules such as the feature extraction network 210, the reconstruction network 220, and the mapping layer 230 can be updated to obtain a trained identity feature representation model.

[0080] Optionally, the system architecture may further include an image generation module 250, used to input at least one image sample to obtain an average image of the object. Optionally, in this embodiment, the parameters of the model can be updated based on the average image, the reconstructed image, the probability of the identity category, and the label of the image sample. For example, the parameters of at least one of the modules such as the feature extraction network 210, the reconstruction network 220, and the mapping layer 230 can be updated to obtain a trained identity feature representation model.

[0081] One possible implementation is, such as Figure 2 As shown, the image generation module 250 may include a 3D network (Net), an average 3D coefficient calculation module, and a renderer. The 3DNet can extract 3D coefficients from multiple image samples of the same object, the average 3D coefficient calculation module can calculate the average 3D coefficients of the multiple image samples, and the renderer can render an average image based on these average 3D coefficients.

[0082] For example, in the model inference stage, for the source image of the target object, the intermediate features of the source image can be obtained by inputting the feature extraction network 210, and then the intermediate features can be input into the mapping layer 230 to obtain the identity feature representation of the target object. Then, based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image.

[0083] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0084] Figure 3This is a schematic flowchart illustrating an identity feature representation model training method 300 according to an embodiment of this application. The method 300 can be executed by any electronic device with data processing capabilities; for example, the electronic device can be implemented as... Figure 1A The server or terminal device in, or Figure 1B Training device 103, or Figure 2 The system architecture in [the context]. For example... Figure 3 As shown, the identity feature representation model training method 300 includes steps S310 to S350.

[0085] S310, Obtain a training sample set, which includes at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample.

[0086] For example, the training sample set may include image samples of at least one object, wherein each object may correspond to at least one image sample. The at least one object includes the first object described above. Optionally, the at least one object may also include other objects besides the first object; this embodiment of the application does not limit this. Optionally, each image sample may correspond to a label, which is used to indicate the identity of the object corresponding to the image sample.

[0087] This application does not limit the type of object. For example, the object may include, but is not limited to, a person, human biometrics, person identity, digital human, intelligent agent, etc. The identity identifier can be used to uniquely identify the object and can be implemented as various characters, strings, etc., without limitation.

[0088] For example, in a face generation task scenario, the image sample may include a face image sample, the object may be the person identity corresponding to the face, and the identity identifier may be the identity ID corresponding to each person identity.

[0089] One possible implementation is that when the training sample set is a collection of face images, the image samples corresponding to the same face all have the same identity ID, where the identity ID is the label, i.e., the ground truth (th), which can be represented by a number. For example, the face image set can include face images corresponding to 100,000 people. The identity ID of the face image corresponding to the first person is 0, the identity ID of the face image corresponding to the second person is 1, and so on.

[0090] In some embodiments, at least two image samples from different angles may be generated based on the source image of the first object, template images from at least two different object angles, and prompt text.

[0091] For example, based on a source image, a template image, and a piece of text, a result of the source image in the pose corresponding to the template image can be generated. One implementation involves inputting the source image, the template image, and a piece of text into an InstantID to generate image samples. Based on this, for each source image, at least two image samples at different angles can be generated based on at least two template images at different angles and multiple pieces of text.

[0092] As a concrete example, 30 template images from different angles and hundreds of text segments can be selected. For each source image, 30 images from different angles can be generated. Among them, the hundreds of text segments can generate image samples with different styles, backgrounds, labels, or scenes, thus enriching the diversity of the generated results.

[0093] Therefore, by generating image samples from at least two different angles and then training the model based on these image samples, the identity feature representation extracted by the model can still maintain a good object feature preservation capability under multiple angles, thereby helping to further improve the effect of image generation tasks.

[0094] In some embodiments, image samples may be preprocessed. Exemplarily, this preprocessing includes, but is not limited to, at least one of the following operations: object detection, object image registration, object image alignment, and cropping. The following description uses a face image as an example to illustrate the image preprocessing workflow.

[0095] First, since faces often occupy a small area in the input image sample, face detection is required to obtain the target face region. Then, face registration is performed within the target face region to obtain facial key points. For example, facial key points include, but are not limited to, the eyes, corners of the mouth, and nose. Second, the face can be aligned and cropped based on the facial key points. Optionally, when the face in the image sample is skewed, rotational alignment can be performed.

[0096] S320, input the first image sample from at least one image sample into the first feature extraction network to obtain the intermediate features of the first image sample.

[0097] Specifically, a first image sample from at least one image sample can be input into a first feature extraction network. The first feature extraction network extracts features from the input first image sample to obtain intermediate features (middle_features) of the first image sample. For example, the intermediate features of the first image sample may include features of the identity identifier corresponding to the first image sample. Taking a face image sample as an example, the intermediate features may include features of the identity ID corresponding to the face.

[0098] For example, the first feature extraction network can be Figure 2 The feature extraction network 210 in the middle.

[0099] Optionally, the first image sample can be any one of at least one image sample of the first object. For example, an image sample can be randomly selected from at least one image sample of the first object and input into the first feature extraction network for feature extraction.

[0100] It is understood that the embodiments of this application describe the model training process based on the first image sample of the first object as an example. The model training process can also be carried out based on other image samples of the first object, or based on image samples of other objects, and the model training process is the same as or similar to the model training process based on the first image sample of the first object. The embodiments of this application will not be described again here.

[0101] For example, the first feature extraction network includes, but is not limited to, a resent structure, which may include, for example, alternating convolutional layers and pooling layers.

[0102] In some embodiments, the first feature extraction network may also be referred to as IDnet, and this application does not limit it.

[0103] For example, the intermediate feature can be a three-dimensional tensor with a shape such as 7×7×512.

[0104] In some embodiments, step S320 can be specifically implemented as follows:

[0105] The first image sample is input into the first feature extraction network after either masking the first region or setting the second region of the first image sample to the first color, to obtain the intermediate features of the first image sample.

[0106] Specifically, by masking the first region of the first image sample, a first image sample with the masked portion can be obtained. Then, the first image sample with the masked portion is input into the first feature extraction network to extract features and obtain the intermediate features of the first image sample.

[0107] Alternatively, by setting the second region of the first image sample to a first color (such as various colors), a first image sample with the occluded portion being the first color can be obtained. This first image sample with the occluded portion being the first color can then be input into the aforementioned first feature extraction network for feature extraction, yielding intermediate features of the first image sample. By setting the second region to the first color, it is possible to occlude the second region while simultaneously interfering with the category prediction result through the first color, further improving the occlusion effect.

[0108] In this embodiment, by performing feature extraction based on the first image sample after occlusion and subsequent model training, the identity feature representation extracted by the model can still maintain a good object feature preservation ability even with occlusion, thereby helping to further improve the effect of image generation tasks.

[0109] One possible approach is to randomly mask a first region of a first image sample to obtain a first image sample with the masked portion. For example, 5% of the first image sample can be randomly masked.

[0110] One possible approach is to randomly select a second region from the first image sample and randomly set it to a first color to obtain the first image sample. For example, 5% of the region randomly selected from the first image sample can be set to a random color.

[0111] Optionally, the first area to be covered can be further set to the first color to further enhance the covering effect.

[0112] S330: Input the intermediate features into the reconstruction network to obtain the reconstructed image of the first object.

[0113] Specifically, the intermediate features of the first image sample can be input into the reconstruction network. The reconstruction network then reconstructs the real image based on the input intermediate features, i.e., the reconstructed image. For example, this reconstruction network can be... Figure 2 The reconstruction network 220 in the middle.

[0114] For example, taking the first image sample as a face image sample, the reconstruction network can reconstruct the real face image based on the given intermediate features (such as the features of the identity ID corresponding to the face), that is, reconstruct the face image, which can be denoted as reconstructed_face.

[0115] Optionally, the reconstruction network may include, but is not limited to, a decoder structure, such as alternating convolutional layers and upsampling layers.

[0116] S340, input the intermediate features into the mapping layer to obtain the identity feature representation of the first object, and input the identity feature representation into the classifier to obtain the identity category probability corresponding to the first image sample.

[0117] Specifically, the intermediate features of the first image sample can be input into the mapping layer. The mapping layer maps the input intermediate features to a feature representation space to obtain the identity feature representation of the first object. For example, this mapping layer can be... Figure 2The mapping layer 230 in the model. This identity feature representation is the final output identity feature representation of the identity feature representation model in this embodiment. Then, this identity feature representation can be input into a classifier to obtain the identity category probability corresponding to the first image sample. For example, the classifier can be... Figure 2 The classification layer is 240.

[0118] For example, taking the first image sample as a face image sample, the mapping layer can map the features of the identity ID corresponding to the face to obtain the feature representation of the identity ID corresponding to the face.

[0119] Optionally, the mapping layer may include, but is not limited to, the first fully connected layer.

[0120] In some embodiments, see Figure 4 The step of inputting the intermediate features into the mapping layer to obtain the identity feature representation of the first object includes the following steps S341 to S342:

[0121] S341 converts intermediate features from a three-dimensional tensor into a one-dimensional vector.

[0122] S342, the one-dimensional vector is input into the first fully connected layer and mapped to the low-dimensional feature space to obtain the low-dimensional feature vector of the first image sample; wherein the identity feature representation of the first object includes the low-dimensional feature vector.

[0123] For example, intermediate features can be flattened to obtain a one-dimensional vector, that is, all elements are arranged in order into a long vector. For example, for intermediate features of 7×7×512, flattening can yield a vector of 7×7×512 = 25088 dimensions, i.e., a one-dimensional vector. Then, this one-dimensional vector can be input into the first fully connected layer, mapping it to a low-dimensional feature space to obtain the low-dimensional feature vector of the first image. The identity feature representation of the first object includes this low-dimensional feature vector. For example, a one-dimensional vector of 25088 dimensions can be input into the first fully connected layer, outputting a vector of 512 dimensions, i.e., the low-dimensional feature vector.

[0124] Here, low-dimensional feature vectors refer to feature vectors after dimensionality reduction processing. While having lower dimensionality, they still effectively represent the core information of the first image sample. Feature vectors before dimensionality reduction are high-dimensional feature vectors, such as the aforementioned 25088-dimensional one-dimensional vector. In this embodiment, a fully connected layer maps the high-dimensional one-dimensional vector to a low-dimensional feature space to obtain a low-dimensional vector. This removes redundant information from the high-dimensional features, which is not necessary for identity features, thus retaining features that are representative and suitable for analysis. Simultaneously, low-dimensional features prevent model overfitting and enhance the model's generalization ability. Furthermore, the fully connected layer can globally fuse local features, thereby learning the complex relationships between features and obtaining a more discriminative representation of identity features.

[0125] Optionally, the classifier may include, but is not limited to, a second fully connected layer and an output layer.

[0126] Specifically, the identity features of the first object output by the mapping layer can be used as input to the classifier. The dimension of the vector output by the second fully connected layer in the classifier is the number of objects in the dataset (such as the training sample set in step S310). Then, the output of the second fully connected layer can be mapped to the identity category probability through the output layer.

[0127] In some embodiments, see continue to see Figure 4 The identity feature represents the input to the classifier to obtain the probability of the identity category corresponding to the first image sample, including the following steps S343 to S345:

[0128] S343, after normalizing the identity feature representation and the weights in the second fully connected layer, input them into the second fully connected layer to obtain the dot product of the normalized identity feature representation and the normalized weights.

[0129] Let x be the identity features of the input i-th sample. i For example, identity features can be represented as x i The weights W in the second fully connected layer j Normalization is performed separately. The weight W is... j It is the weight vector in the fully connected layer, used to represent the identity feature x. i The score mapped to category j. For example, this can be achieved by representing identity features x. i Divide by its norm ||x i ‖, thus obtaining the identity feature representation x i Normalized identity feature representation Weight W j Divide by its norm ||W k ||, thus obtaining the weight W j Normalized weights Then, the normalized identity features can be represented. and weight Inputting the second fully connected layer yields the normalized identity feature representation. and weight The dot product, for example, can be expressed as

[0130] Understandably, the normalized identity feature representation and weight The dot product, i.e. It can be equivalent to calculating and The included angle θ j That is, the process satisfies the following formula (1):

[0131]

[0132] Wherein, the included angle θ j Used to represent identity features and weight Similarity. Angle θ j The smaller the value, the stronger the identity characteristic. and weight The closer they are, the more distinctive their identity characteristics. The higher the similarity to category j, the lower the similarity. Conversely, the lower the similarity, the lower the angle θ. j The larger the angle θ, the lower the similarity. In classification tasks, the included angle θ... j Used to calculate identity feature representation The probability of belonging to category j is determined by the smaller the angle between the two categories, the higher the probability value will be, thus affecting the final classification decision.

[0133] S344, determine the angle between the normalized identity feature representation and the normalized weights based on the dot product.

[0134] For example, the normalized identity feature representation can be obtained by performing an inverse cosine calculation on the above formula (1). and weight The angle between them. For example, θ can be calculated using Arccos. j This process can be expressed as the following formula (2):

[0135] θ j =arccos (cosθ) j (2)

[0136] S345 takes the cosine of the sum of the angle and marginal parameters and the product of the scale parameter as input to the output layer to obtain the identity category probability.

[0137] Specifically, for each sample i and target category y i We can first find the normalized eigenvectors. with normalized weights Angle between The sum of the marginal parameter m, that is, at the angle Based on this, add a marginal parameter m, and then calculate the cosine of the sum of the two, i.e. The specific formula is shown in formula (3) below:

[0138]

[0139] Here, the marginal parameter *m* is a hyperparameter. Specifically, the marginal parameter *m* is an additive angular margin used to add a fixed interval to the angle between the feature vector and the weight vector. It enhances intra-class compactness and inter-class separability by increasing the angle of the target class. In the embodiments of this application, the marginal parameter *m* can enhance the feature vector... The discriminative power of the model increases the angle between the feature vector and the weight vector of the target class, thereby reducing the probability of the target class and improving the model's classification ability.

[0140] Then, the product of the cosine of the sum of the angle and the marginal parameter and the scale parameter can be calculated, that is, the result of the above formula (3) can be multiplied by the scale parameter to obtain the result. Here, s is the scale parameter, a hyperparameter. Specifically, the scale parameter s is a scaling factor used to amplify or reduce the dot product of the feature vector and the weight vector, thereby enhancing the model's ability to distinguish feature vectors by increasing the magnitude of the feature vector. In this embodiment, the scaling factor s amplifies the difference between the feature vector and the weight vector, making the model more sensitive to feature differences between different categories.

[0141] Then, As the input to the output layer, such as the softmax function, we obtain the final prediction result, i.e., the identity category probability. For example, the identity category probability can be represented by the following formula (4):

[0142]

[0143] Among them, y i Let be the label of the i-th sample, and n be the total number of categories. Normalized eigenvectors with normalized weights The angle between them, θ j Normalized eigenvectors with normalized weights The angle between them.

[0144] Therefore, in this embodiment of the application, the identity feature representation of the first object is obtained by inputting intermediate features into the mapping layer, and then the identity feature representation is input into the classifier to obtain the probability of the identity category corresponding to the first image sample.

[0145] S350, based on the reconstructed image, the probability of the identity category, and the label corresponding to the first image sample, update the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier to obtain a trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0146] Specifically, based on the reconstructed image, the identity category probability, and the label corresponding to the first image sample, at least one of the following components—the first feature extraction network, the reconstruction network, the mapping layer, and the classifier—can be updated with parameters. The outputs of the first feature extraction network and the mapping layer, when the model training stopping condition is met, are then used to obtain the trained identity feature representation model. This identity feature representation model includes the first feature extraction network and the mapping layer. In this embodiment, by constraining the identity feature representation model with the identity category probability and the first image sample, the model can extract features that characterize the identity. Furthermore, by constraining the model with the reconstructed image, the extracted identity features also possess image reconstruction capabilities.

[0147] For example, in the face generation task scenario, the identity feature representation model extracts the identity ID features of the face image sample through the first feature extraction network, and inputs the identity ID features into the mapping layer to obtain the identity ID feature representation. The identity ID feature representation not only represents the identity ID corresponding to the face image sample, but also has the ability to reconstruct the face image.

[0148] Therefore, in this embodiment, intermediate features of the first image sample are extracted by the first feature extraction network, and the intermediate features are input into the mapping layer and classifier to obtain the identity category probability corresponding to the first image sample. The reconstructed image of the first image sample of the first object is obtained based on the intermediate features of the first image sample. Then, the model parameters are updated based on the reconstructed image, the identity category probability and the label of the first image sample, so that the model can extract features for representing identity from the image. These features have the ability to reconstruct the image. Therefore, the features extracted by the model can be more adapted to the image generation task, which is conducive to further improving the effect of the image generation task.

[0149] For example, in a face generation task, facial identity features can be extracted from the source image based on the identity feature representation model trained according to the embodiments of this application. These facial identity features can characterize the identity corresponding to the face, thereby constraining the facial identity features after face swapping to be consistent with the source image. Furthermore, these facial identity features have the ability to reconstruct faces, thus better adapting to face generation tasks and helping to further improve the effectiveness of human-generated tasks.

[0150] In some embodiments, in step S350, a first loss function can be obtained based on the reconstructed image.

[0151] Specifically, a first loss function can be calculated based on the reconstructed image. This first loss function can constrain the reconstructed image to be similar to the image of the first object corresponding to the first image sample, thereby making the reconstructed image closer to the image of the first object and improving the image reconstruction capability of the identity feature.

[0152] In this application embodiment, the image of the first object corresponding to the first image sample is not limited, and may include, but is not limited to, the first image sample, or an image obtained based on the first image sample, or an image obtained based on related image samples of the first object in the training sample set.

[0153] In some embodiments, an average image corresponding to the first object can also be obtained based on at least one image sample of the first object.

[0154] Specifically, an average image corresponding to the first image can be obtained from at least one (e.g., all) image samples of the first object in at least one object in the training sample set. Since this average image is obtained from at least one image sample of the first object in the training sample set, it can represent the typical features of the first object. See also, for an example... Figure 2 At least one image sample of the first object can be input into the image generation module 250 to obtain the average image of the first object.

[0155] One possible approach, see [link to relevant documentation] Figure 5 The average image of the first object can be obtained by following steps S351 to S353.

[0156] S351, input at least one image sample of the first object into a three-dimensional 3D network to obtain the 3D coefficients of each image sample.

[0157] For example, at least one image sample corresponding to the first object in the training dataset can be input into a 3DNet to extract 3D coefficients from each image sample. These 3D coefficients can be the 3D coefficients of the first object in the image sample, and an image of the first object can be rendered based on these 3D coefficients. See also: Figure 2 The 3DNet in the image generation module 250 can extract the corresponding 3D coefficients for each image sample of the first object.

[0158] Taking face image samples as an example, all face image samples of the same person in the training dataset can be input into the 3D network to extract the 3D coefficients of each face image.

[0159] For example, a 3D network can be a pre-trained 3D parametric model, such as a 3D face parametric model. This model uses a set of parameters to represent information such as the shape, texture, labels, pose, and lighting of a face. These parameters can be used to render realistic 3D face images. Based on the input object image sample, the model can output the object's 3D coefficients. Taking a face image sample as an example, the 3D face parametric model can output 3D face information such as ID coefficients (id_coeff), expression coefficients (expression_coeff), texture coefficients (texture_coeff), angles, and lighting (gamma_coeff). Based on these 3D coefficients, the corresponding face image can be rendered.

[0160] S352, based on the 3D coefficients of each image sample, obtain the average 3D coefficients of the first object.

[0161] For example, see [link to example]. Figure 2 The average 3D coefficient calculation module in the image generation module 250 can calculate the average 3D coefficient of the first object based on the 3D coefficients of multiple image samples.

[0162] For example, the average 3D coefficients of all image samples of the first object can be calculated to obtain the average 3D coefficients of the first object. For instance, the average 3D coefficients of the identity ID coefficient (id_coeff), expression coefficient (expression_coeff), texture coefficient (texture_coeff), angles, and lighting (gamma_coeff) can be calculated separately to obtain the average 3D coefficients of the face.

[0163] S353, the average 3D coefficients are input into the renderer for rendering to obtain the average image.

[0164] For example, see [link to example]. Figure 2 The renderer in the image generation module 250 can render an average image based on the input average 3D coefficients.

[0165] For example, the average 3D face coefficients are input into the renderer for rendering, such as shape reconstruction, texture mapping, pose adjustment, lighting simulation and final rendering, to obtain the average face image of the first object, which can be denoted as render_face.

[0166] Therefore, in this embodiment of the application, by inputting at least one image sample of the first object into a 3D network, the 3D coefficients of each image sample are obtained, and then the average 3D coefficients of the first object are obtained based on the 3D coefficients of the at least one image sample. By inputting the average 3D coefficients into a renderer for rendering, an average image representing the typical features of the first object can be obtained.

[0167] In some embodiments, a first loss function can be obtained based on the average image and the reconstructed image.

[0168] Specifically, a first loss function can be calculated based on the average image and the reconstructed image. This first loss function can constrain the reconstructed image to be similar to the average image of the first object, thereby making the reconstructed image closer to the average image of the first object, thus improving the image reconstruction capability of the identity features and enabling the identity features to better maintain consistency with the identity.

[0169] For example, in face generation tasks, by calculating a first loss function based on the average face image and the reconstructed face image, the reconstructed face image can be constrained to be similar to the average face image, which enables the identity ID features to better maintain consistency with the face identity.

[0170] One possible approach, see below. Figure 5 The following step S354 can also be performed to obtain the reconstruction loss function of the reconstructed image and the average image.

[0171] S354, Based on the difference between the reconstructed image and the average image, a reconstruction loss function is obtained; wherein, the first loss function includes the reconstruction loss function.

[0172] For example, a reconstruction loss function can be obtained based on the difference between the reconstructed image and the average image, such as pixel-level difference. This reconstruction loss function measures the difference between the image generated by the model and the average image. The first loss function includes this reconstruction loss function. For example, the reconstruction loss function can be L1 loss, L2 loss, etc., without limitation.

[0173] Taking the reconstructed face image and the average face image as examples, the reconstruction loss function can be expressed as follows (5):

[0174] Reconstruction_loss = |reconstructed_face– render_face| (5)

[0175] Here, Reconstruction_loss represents the reconstruction loss function, reconstructed_face represents the reconstructed face image, and render_face represents the average face image.

[0176] One possible approach, see below. Figure 5 The following steps S3515 to S3517 can also be performed to obtain the image similarity loss function of the reconstructed image and the average image.

[0177] S355, the reconstructed image is input into the pre-trained second feature extraction network to obtain first network features with at least two different feature layers.

[0178] For example, the second feature extraction network can be a deep convolutional neural network, such as AlexNet or VGG, without limitation. Taking AlexNet as an example, it can include multiple convolutional layers and pooling layers to extract features of the image at different convolutional layers. For example, features from lower convolutional layers can represent low-level features such as lines and colors, while features from higher convolutional layers can represent high-level features such as parts, styles, and semantics.

[0179] For example, the AlexNet network can extract features from four different feature layers of the reconstructed face image (reconstructed_face), denoted as recons_fea1, recons_fea2, recons_fea3, and recons_fea4 = AlexNet_feature(reconstructed_face).

[0180] S356, the average image is input into the pre-trained second feature extraction network to obtain second network features from at least two different feature layers.

[0181] For example, the second feature extraction network can refer to the description in step S3515. Taking the second feature extraction network as the AlexNet network, the AlexNet network can extract features from four different feature layers of the average face image (render_face), denoted as render_fea1, render_fea2, render_fea3, and render_fea4 = AlexNet_feature(render_face).

[0182] S357, Based on the first network features and the second network features, obtain the image similarity loss function for the reconstructed image and the average image; wherein, the first loss function includes the image similarity loss function.

[0183] Specifically, based on the network features of different layers of the reconstructed image and the average image extracted in steps S3515 and S3516 respectively, for example, the features of the same layer of the reconstructed image and the average image can be compared separately to calculate the image similarity loss function between the reconstructed image and the average image. The image similarity loss function measures the perceptual differences between the reconstructed image and the average image, such as the similarity of the overall image structure and content, rather than just focusing on the differences in pixel values ​​between the reconstructed image and the average image, thereby achieving better results in image generation tasks.

[0184] In one possible implementation, the image similarity loss function can be the Learned Perceptual Image Patch Similarity (LPIPS) loss function or perceptual loss.

[0185] For example, continuing the example above, the image similarity loss function can be obtained according to the following formula (6):

[0186] LPIPS_loss=|recons_fea1-render_fea1|+|recons_fea2-render_fea2|+

[0187] |recons_fea3-render_fea3|+|recons_fea4-render_fea4|(6)

[0188] Wherein, LPIPS_loss is the LPIPS loss.

[0189] Optionally, the first loss function may include at least one of a reconstruction loss function and an image similarity loss function.

[0190] Therefore, according to the above processes S3511 to S3517, the embodiments of this application can realize the acquisition of the average image of the first object, and the calculation of the reconstruction loss function and the image similarity loss function based on the reconstructed image and the average image, thereby constraining the pixel-level differences between the reconstructed image and the average image, and constraining the similarity of the overall structure and content of the image.

[0191] In some embodiments, see continue to see Figure 5 The following steps, S358, can also be performed to obtain the second loss function.

[0192] S358, based on the identity category probability and the label of the first image sample, the second loss function is obtained.

[0193] Specifically, a second loss function can be calculated based on the identity category probability output by the classifier and the label corresponding to the first image sample. This second loss function can constrain the identity category probability predicted by the classifier to be the same as or similar to the real identity corresponding to the image sample, thereby making the predicted identity category more accurate.

[0194] One possible implementation is that the second loss function can include ArcFace loss. ArcFace loss is an angle-based loss function that introduces additional marginal constraints in the angle space, forcing image samples of the same type to be closer together and image samples of different types to be further apart.

[0195] For example, when the identity category probability is obtained according to formula (4) above, the cross-entropy loss can be calculated to obtain the ArcFace loss. As an example, the ArcFace loss can be expressed as the following formula (7):

[0196]

[0197] Where N is the number of samples, and other parameters in the formula can be found in the relevant descriptions above.

[0198] In some embodiments, see continue to see Figure 5 You can also execute step S359.

[0199] S359, based on at least one of the first loss function and the second loss function, update the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer and the classifier.

[0200] Specifically, based on at least one of the first loss function and the second loss function calculated above, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier can be updated. The outputs of the first feature extraction network and the mapping layer when the model training stopping condition is met are obtained to obtain the trained identity feature representation model.

[0201] In some implementation methods, the total loss function can be obtained based on the first loss function and the second loss function, and then the model can be jointly trained based on this total loss function. For example, the final loss function during model training can be expressed as the following formula (8):

[0202] loss=ArcFace_loss+Reconstruction_loss+LPIPS_loss (8)

[0203] Optionally, each loss function in formula (8) can be multiplied by its corresponding constant to balance the different loss functions.

[0204] Therefore, in this embodiment, intermediate features of the first image sample are extracted by the first feature extraction network, and the intermediate features are input into the mapping layer and classifier to obtain the identity category probability corresponding to the first image sample. The reconstructed image of the first image sample of the first object is obtained based on the intermediate features of the first image sample. Then, the model parameters are updated based on the reconstructed image, the identity category probability and the label of the first image sample, so that the model can extract features for representing identity from the image. These features have the ability to reconstruct the image. Therefore, the features extracted by the model can be more adapted to the image generation task, which is conducive to improving the effect of the image generation task.

[0205] Furthermore, in this embodiment, an average image of the first object is obtained based on at least one image sample of the first object. Then, the model parameters are updated based on the average image, the reconstructed image, the identity category probability, and the label of the first image sample. This allows the model to learn the typical features of the first object from the average image, enabling the identity features extracted by the model to better maintain consistency with the identity, which is beneficial to further improving the effect of the image generation task.

[0206] In addition, by generating multiple image samples of the first object from different angles and inputting the image samples into the feature extraction network for model training after partial occlusion, the model can still maintain a good ability to preserve object features even at larger image angles or when the object image is occluded.

[0207] Figure 6 This is a schematic flowchart of an image generation method 600 according to an embodiment of this application. The method 600 can be executed by any electronic device with data processing capabilities; for example, the electronic device may be implemented as... Figure 1A The server or terminal device in, or Figure 1B The execution device 104 in the middle. For example... Figure 6 As shown, the image generation method 600 includes steps S610 to S630.

[0208] S610, Obtain the source image of the target object.

[0209] For example, the source image of the target object is the source image in the image generation task, which includes the target object that needs to be included in the generated image. That is, the object in the generated target image needs to be the same object as the target object in the source image.

[0210] For example, in face generation tasks, such as face swapping, a source image containing the target face can be obtained. After face swapping, the identity of the face in the target image needs to be the same as the identity of the face in the source image.

[0211] S620, the source image is input into the identity feature representation model, and the intermediate features of the source image are obtained by using the first feature extraction network; and the intermediate features are input into the mapping layer to obtain the identity feature representation of the source image.

[0212] The identity feature representation model includes a first feature extraction network and a mapping layer, which is based on, for example... Figure 3 The method described above is used for training. Specifically, intermediate features of the first image sample are extracted using a first feature extraction network, and these intermediate features are input into a mapping layer and a classifier to obtain the identity category probability corresponding to the first image sample. A reconstructed image of the first object is obtained based on the intermediate features of the first image sample. The model parameters are then updated based on the reconstructed image, the identity category probability, and the label of the first image sample. This allows the model to extract features representing the identity from the image, which possess image reconstruction capabilities. Therefore, the features extracted by the model are more suitable for the image generation task, thus improving the performance of the image generation task. Furthermore, this embodiment obtains an average image of the first object based on at least one image sample of the first object, and then updates the model parameters based on this average image. This allows the model to learn typical features of the first object from the average image, ensuring that the identity features maintain better consistency with the identity, further improving the performance of the image generation task. Additionally, this embodiment generates multiple image samples of the first object from different angles and inputs these image samples into the feature extraction network for model training after partial occlusion. This allows the model to maintain good object feature preservation capabilities even at larger image angles or when the object image is occluded.

[0213] For example, in face generation tasks, an identity feature representation model can extract facial identity features from a source image, which possess face reconstruction capabilities. Simultaneously, these facial identity features can better maintain consistency with the face's identity. Furthermore, in situations involving larger facial angles or face occlusion, these facial identity features also possess identity feature preservation capabilities.

[0214] Specifically, the processing procedures of the first feature extraction network and the mapping layer can be found in [reference needed]. Figure 3 The relevant descriptions in the text will not be repeated here.

[0215] S630, based on the identity feature representation, replace the object in the template image with the target object in the source image to generate the target image.

[0216] Specifically, based on the identity feature representation extracted in step S620, the objects in the template image can be constrained to replace the target objects in the source image, ensuring that the object features in the generated target image are consistent with the object features in the source image. Since the identity feature representation in this embodiment has image reconstruction capabilities, it can better maintain consistency with the identity identifier and has good object feature preservation capabilities, thus better adapting to image generation tasks.

[0217] For example, in face generation tasks, an identity feature representation model can be used to extract facial identity features from the source image. The model constrains the identity features of the face after face swapping to be consistent with those of the face in the source image; that is, the identity of the face in the target image after face swapping needs to be the same as the identity of the face in the source image, making the face swap result as similar to the input source image as possible. Simultaneously, the face in the target image also needs to retain information such as the expression, angle, and background of the template face. For example, see [link to example]. Figure 7 The template image can provide information such as expression, angle, and background, while the source image can provide identity information. The generated target image is constrained to maintain the identity information of the source image, while also having information such as the expression, angle, and background of the person in the template image.

[0218] Therefore, this embodiment of the application obtains a source image of the target object, inputs the source image into an identity feature representation model, uses a first feature extraction network to obtain intermediate features of the source image, and inputs these intermediate features into a mapping layer to obtain the identity feature representation of the source image. Then, based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image. In this embodiment, the identity feature representation extracted by the model has image reconstruction capabilities, thus the target image generated by this embodiment can have better results. In addition, the object features extracted by the model can also have better identity preservation capabilities at larger angles or under object occlusion, which is beneficial to further improve the effect of the generated image.

[0219] In some embodiments, when performing a face generation task, the following steps can be performed sequentially: video acquisition, image input, face detection, face region cropping, face identity feature extraction, face generation task, and result display. The face identity feature extraction process can be implemented using the method of this application embodiment. It should be understood that the method of this application embodiment needs to interact with other modules in actual use. First, image input needs to be received from the video acquisition module. Then, face detection is performed on the input image, and the face region is cropped. Next, the method provided in this application embodiment is implemented to extract face identity features. Then, the face identity features extracted by the method provided in this application embodiment are sent to a specific face generation task, such as face swapping. Finally, the generated image can be displayed.

[0220] The specific embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this application, various simple modifications can be made to the technical solutions of this application, and these simple modifications all fall within the protection scope of this application. For example, the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this application will not describe the various possible combinations separately. Furthermore, various different embodiments of this application can also be arbitrarily combined, as long as they do not violate the spirit of this application, they should also be considered as the content disclosed in this application.

[0221] It should also be understood that, in the various method embodiments of this application, the sequence numbers of the above processes do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. It should be understood that these sequence numbers can be interchanged where appropriate so that the embodiments of this application described can be implemented in a sequence other than those illustrated or described.

[0222] The method embodiments of this application have been described in detail above. The following description, in conjunction with... Figures 8 to 10 The following describes in detail the device embodiments of this application.

[0223] Figure 8 This is a schematic block diagram of the identity feature representation model training device 10 according to an embodiment of this application. Figure 8 As shown, the device 10 may include an acquisition unit 11, a first feature extraction network 12, a reconstruction network 13, a mapping layer 14, a classifier 15, and a parameter update unit 16.

[0224] Acquisition unit 11 is used to acquire a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample;

[0225] The first feature extraction network 12 is used to take a first image sample from the at least one image sample as input and obtain the intermediate features of the first image sample.

[0226] Reconstruction network 13 is used to input the intermediate feature input to obtain a reconstructed image of the first object;

[0227] Mapping layer 14 is used to input the intermediate features to obtain the identity feature representation of the first object;

[0228] Classifier 15 is used to take the identity feature representation as input and obtain the identity category probability corresponding to the first image sample;

[0229] The parameter update unit 16 is used to update the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer and the classifier according to the reconstructed image, the identity category probability and the label corresponding to the first image sample, so as to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0230] In some embodiments, the parameter update unit 16 is specifically used for:

[0231] Based on the reconstructed image, a first loss function is obtained;

[0232] A second loss function is obtained based on the identity category probability and the label of the first image sample;

[0233] Based on at least one of the first loss function and the second loss function, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier are updated.

[0234] In some embodiments, the apparatus 10 further includes an image generation unit, configured to obtain an average image of the first object based on at least one image sample of the first object;

[0235] In some embodiments, the parameter update unit 16 is specifically used for:

[0236] The first loss function is obtained based on the average image and the reconstructed image.

[0237] In some embodiments, the parameter update unit 16 is specifically used for:

[0238] A reconstruction loss function is obtained based on the difference between the reconstructed image and the average image; wherein the first loss function includes the reconstruction loss function.

[0239] In some embodiments, the parameter update unit 16 is specifically used for:

[0240] The reconstructed image is input into a pre-trained second feature extraction network to obtain first network features with at least two different feature layers.

[0241] The average image is input into the pre-trained second feature extraction network to obtain second network features from at least two different feature layers;

[0242] Based on the first network features and the second network features, an image similarity loss function is obtained for the reconstructed image and the average image; wherein, the first loss function includes the image similarity loss function.

[0243] In some embodiments, the mapping layer includes a first fully connected layer, and the mapping layer 14 is specifically used for:

[0244] The intermediate features are converted from a three-dimensional tensor to a one-dimensional vector.

[0245] The one-dimensional vector is input into the first fully connected layer and mapped to a low-dimensional feature space to obtain the low-dimensional feature vector of the first image sample; wherein the identity feature representation of the first object includes the low-dimensional feature vector.

[0246] In some embodiments, the classifier 15 includes a second fully connected layer and an output layer;

[0247] Specifically, classifier 15 is used for:

[0248] After normalizing the identity feature representation and the weights in the second fully connected layer, they are input into the second fully connected layer to obtain the dot product of the normalized identity feature representation and the normalized weights.

[0249] The angle between the normalized identity feature representation and the normalized weight is determined based on the dot product;

[0250] The product of the cosine of the sum of the angle and marginal parameters and the scale parameter is input into the output layer to obtain the probability of the identity category.

[0251] In some embodiments, the image generation unit is specifically used for:

[0252] At least one image sample of the first object is input into a three-dimensional 3D network to obtain 3D coefficients for each image sample;

[0253] The average 3D coefficient of the first object is obtained based on the 3D coefficient of each image sample;

[0254] The average 3D coefficients are input into the renderer for rendering to obtain the average image.

[0255] In some embodiments, the first feature extraction network 12 is specifically used for:

[0256] After the first image sample covers the first region, it is input into the first feature extraction network to obtain the intermediate features of the first image sample.

[0257] In some embodiments, the acquisition unit 11 is specifically used for:

[0258] Based on the source image of the first object, template images from at least two different object angles, and prompt text, generate at least two image samples from different angles.

[0259] This application embodiment extracts intermediate features of a first image sample using a first feature extraction network, inputs these intermediate features into a mapping layer and a classifier to obtain the identity category probability corresponding to the first image sample, and obtains a reconstructed image of the first image sample of the first object based on the intermediate features of the first image sample. Then, the model parameters are updated based on the reconstructed image, the identity category probability, and the label of the first image sample, so that the model can extract features for representing identity from the image. These features have the ability to reconstruct the image, so the features extracted by the model can be more adapted to the image generation task, which is beneficial to improving the effect of the image generation task.

[0260] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 8 The identity feature representation model training device 10 shown can execute the above method embodiments, and the aforementioned and other operations and / or functions of each module in the device 10 are respectively to implement the corresponding process in the above method 300. For the sake of brevity, they will not be described in detail here.

[0261] Figure 9 This is a schematic block diagram of the image generation apparatus 20 according to an embodiment of this application. Figure 9 As shown, the device 20 may include an acquisition unit 21, an identity feature representation model 22, and a generation unit 23.

[0262] Acquisition unit 21 is used to acquire the source image of the target object;

[0263] Identity feature representation model 22 is used to input the source image, use a first feature extraction network to obtain intermediate features of the source image, and input the intermediate features into a mapping layer to obtain the identity feature representation of the source image;

[0264] The generation unit 23 is used to replace the object in the template image with the target object in the source image according to the identity feature representation, and generate a target image;

[0265] The identity feature representation model 22 includes the first feature extraction network and the mapping layer, and the identity feature representation model 22 is trained according to the identity feature representation model training method provided in the embodiments of this application.

[0266] This application embodiment obtains a source image of the target object, inputs the source image into an identity feature representation model, uses a first feature extraction network to obtain intermediate features of the source image, and inputs these intermediate features into a mapping layer to obtain the identity feature representation of the source image. Then, based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image. In this application embodiment, the identity feature representation extracted by the model has image reconstruction capabilities, therefore the target image generated by this application embodiment can have better results.

[0267] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 10 The image generation apparatus 20 shown can execute the above-described method embodiments, and the aforementioned and other operations and / or functions of each module in the apparatus 20 are respectively for implementing the corresponding processes in the above-described method 600. For the sake of brevity, they will not be described in detail here.

[0268] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0269] Figure 10 This is a schematic block diagram of the electronic device provided in the embodiments of this application.

[0270] like Figure 10 As shown, the electronic device 30 may include:

[0271] The system includes a memory 33 for storing a computer program 34 and a processor 32 for transferring the program code 34 to the processor 32. In other words, the processor 32 can retrieve and run the computer program 34 from the memory 33 to implement the method provided in the embodiments of this application.

[0272] In some embodiments, the processor 32 may call and run the computer program 34 from the memory 33 to implement the identity feature representation model training method provided in this application embodiment, including:

[0273] Obtain a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample;

[0274] The first image sample from the at least one image sample is input into the first feature extraction network to obtain the intermediate features of the first image sample;

[0275] The intermediate features are input into the reconstruction network to obtain the reconstructed image of the first object;

[0276] The intermediate features are input into the mapping layer to obtain the identity feature representation of the first object, and the identity feature representation is input into the classifier to obtain the identity category probability corresponding to the first image sample;

[0277] Based on the reconstructed image, the identity category probability, and the label corresponding to the first image sample, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier are updated to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

[0278] For example, the processor 32 can be used to execute the steps in the method 300 described above according to the instructions in the computer program 34.

[0279] In some embodiments, the processor 32 may call and run the computer program 34 from the memory 33 to implement the image generation method of this application embodiment, including:

[0280] Obtain the source image of the target object;

[0281] The source image is input into the identity feature representation model, and the intermediate features of the source image are obtained by using the first feature extraction network; and the intermediate features are input into the mapping layer to obtain the identity feature representation of the source image.

[0282] Based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image;

[0283] The identity feature representation model includes the first feature extraction network and the mapping layer, and the identity feature representation model is trained according to the identity feature representation model training method provided in the embodiments of this application.

[0284] For example, the processor 32 can be used to execute the steps in the method 600 described above according to the instructions in the computer program 34.

[0285] In some embodiments of this application, the processor 32 may include, but is not limited to:

[0286] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0287] In some embodiments of this application, the memory 33 includes, but is not limited to:

[0288] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0289] In some embodiments of this application, the computer program 34 may be divided into one or more units, which are stored in the memory 33 and executed by the processor 32 to perform the method provided in this application. The one or more units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 34 in the electronic device 30.

[0290] Optional, such as Figure 10 As shown, the electronic device 30 may further include:

[0291] Transceiver 33, which can be connected to processor 32 or memory 33.

[0292] The processor 32 can control the transceiver 33 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include antennas, and the number of antennas may be one or more. It should be understood that the various components in this electronic device are connected through a bus system, which includes, in addition to a data bus, a power bus, a control bus, and a status signal bus.

[0293] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0294] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0295] It is understood that in the specific implementation of this application, when the above embodiments of this application are applied to specific products or technologies and involve user information and other related data, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0296] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0297] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0298] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0299] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an identity feature representation model, characterized in that, include: Obtain a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample; The first image sample from the at least one image sample is input into the first feature extraction network to obtain the intermediate features of the first image sample; The intermediate features are input into the reconstruction network to obtain the reconstructed image of the first object; The intermediate features are input into the mapping layer to obtain the identity feature representation of the first object, and the identity feature representation is input into the classifier to obtain the identity category probability corresponding to the first image sample; Based on the reconstructed image, the identity category probability, and the label corresponding to the first image sample, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier are updated to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

2. The method according to claim 1, characterized in that, The step of updating the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier based on the reconstructed image, the identity recognition probability, and the label corresponding to the first image sample includes: Based on the reconstructed image, a first loss function is obtained; A second loss function is obtained based on the identity category probability and the label of the first image sample; Based on at least one of the first loss function and the second loss function, the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier are updated.

3. The method according to claim 2, characterized in that, Also includes: Based on at least one image sample of the first object, obtain the average image of the first object; The step of obtaining the first loss function based on the reconstructed image includes: The first loss function is obtained based on the average image and the reconstructed image.

4. The method according to claim 3, characterized in that, The step of obtaining the first loss function based on the reconstructed image and the average image includes: A reconstruction loss function is obtained based on the difference between the reconstructed image and the average image; wherein the first loss function includes the reconstruction loss function.

5. The method according to claim 3, characterized in that, The step of obtaining the first loss function based on the reconstructed image and the average image includes: The reconstructed image is input into a pre-trained second feature extraction network to obtain first network features with at least two different feature layers. The average image is input into the pre-trained second feature extraction network to obtain second network features from at least two different feature layers; Based on the first network features and the second network features, an image similarity loss function is obtained; wherein, the first loss function includes the image similarity loss function.

6. The method according to any one of claims 1-5, characterized in that, The mapping layer includes a first fully connected layer, and the step of inputting the intermediate features into the mapping layer to obtain the identity feature representation of the first object includes: The intermediate features are converted from a three-dimensional tensor to a one-dimensional vector. The one-dimensional vector is input into the first fully connected layer and mapped to a low-dimensional feature space to obtain the low-dimensional feature vector of the first image sample; wherein the identity feature representation of the first object includes the low-dimensional feature vector.

7. The method according to any one of claims 1-5, characterized in that, The classifier includes a second fully connected layer and an output layer; Wherein, the identity feature represents the probability of the identity category corresponding to the first image sample obtained by inputting it into the classifier, including: After normalizing the identity feature representation and the weights in the second fully connected layer, they are input into the second fully connected layer to obtain the dot product of the normalized identity feature representation and the normalized weights. The angle between the normalized identity feature representation and the normalized weight is determined based on the dot product; The product of the cosine of the sum of the angle and marginal parameters and the scale parameter is input into the output layer to obtain the probability of the identity category.

8. The method according to claim 3, characterized in that, The step of obtaining the average image of the first object based on at least one image sample of the first object includes: At least one image sample of the first object is input into a three-dimensional 3D network to obtain 3D coefficients for each image sample; The average 3D coefficient of the first object is obtained based on the 3D coefficient of each image sample; The average 3D coefficients are input into the renderer for rendering to obtain the average image.

9. The method according to any one of claims 1-5, characterized in that, The step of inputting the first image sample into the first feature extraction network to obtain the intermediate features of the first image sample includes: The first image sample is input into the first feature extraction network after either masking the first region or setting the second region of the first image sample to the first color, to obtain the intermediate features of the first image sample.

10. The method according to any one of claims 1-5, characterized in that, The acquisition of the training sample set includes: Based on the source image of the first object, template images from at least two different object angles, and prompt text, generate at least two image samples from different angles.

11. An image generation method, characterized in that, include: Obtain the source image of the target object; The source image is input into the identity feature representation model, and the intermediate features of the source image are obtained by using the first feature extraction network. And inputting the intermediate features into the mapping layer to obtain the identity feature representation of the source image; Based on the identity feature representation, the object in the template image is replaced with the target object in the source image to generate the target image; The identity feature representation model includes the first feature extraction network and the mapping layer, and the identity feature representation model is trained according to the method described in any one of claims 1-9.

12. A training device for an identity feature representation model, characterized in that, include: An acquisition unit is used to acquire a training sample set, the training sample set including at least one image sample of the first object and a label corresponding to the image sample, the label being used to indicate the identity identifier corresponding to the image sample; A first feature extraction network is used to take a first image sample from the at least one image sample as input and obtain intermediate features of the first image sample. A reconstruction network is used to input the intermediate feature inputs to obtain a reconstructed image of the first object; A mapping layer is used to input the intermediate features and obtain the identity feature representation of the first object; A classifier is used to take the identity feature representation as input and obtain the probability of the identity category corresponding to the first image sample; The parameter update unit is used to update the parameters of at least one of the first feature extraction network, the reconstruction network, the mapping layer, and the classifier according to the reconstructed image, the identity category probability, and the label corresponding to the first image sample, so as to obtain the trained identity feature representation model, wherein the identity feature representation model includes the first feature extraction network and the mapping layer.

13. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores instructions, and when the processor executes the instructions, it causes the processor to perform the method according to any one of claims 1-10, or the method according to claim 11.

14. A computer storage medium, characterized in that, Used to store a computer program, the computer program including a method for performing any one of claims 1-10, or the method described in claim 11.

15. A computer program product, characterized in that, Includes computer program code, which, when executed by an electronic device, causes the electronic device to perform the method of any one of claims 1-10, or the method of claim 11.

Images