Image processing method, device and computer readable storage medium

Abstract

The application discloses an image processing method, device and computer readable storage medium. The method comprises the following steps: acquiring a to-be-processed face image, and obtaining a vector corresponding to the to-be-processed face image based on a first image generation model; then, the first image generation model and a second image generation model are subjected to parameter replacement to obtain a target image generation model, and the vector corresponding to the to-be-processed face image is input into the target image generation model to obtain a target face image. By fusing the real face generation model and the cartoon face generation model, the target cartoon face image obtained based on the fused model can combine the identity features of the real face image, and the similarity with the to-be-processed face image is higher.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, and computer-readable storage medium. Background Technology

[0002] Image processing is the act of using computers to process image information to meet human visual, psychological, or application needs. It is widely used in professional fields such as surveying, atmospheric science, astronomy, and graphic design.

[0003] However, when cartoonizing facial images, the similarity between the cartoonized facial images obtained based on unsupervised training and the actual facial images is not high. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and computer-readable storage medium, the technical solutions of which are as follows:

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] Obtain the face image to be processed, and obtain the vector corresponding to the face image to be processed based on the first image generation model;

[0007] The parameters of the first image generation model and the second image generation model are replaced to obtain the target image generation model; the second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as that of the first image generation model.

[0008] The vector corresponding to the face image to be processed is input into the target image generation model to obtain the target face image.

[0009] In one alternative approach of the first aspect, a vector corresponding to the face image to be processed is obtained based on the first image generation model, including:

[0010] The first random vector is input into the first image generation model to obtain the first random face image;

[0011] A loss function is constructed based on the first random vector, the face image to be processed, and the random face image;

[0012] The loss function is optimized to obtain a vector corresponding to the face image to be processed.

[0013] In another alternative to the first aspect, parameter replacement is performed on the first image generation model and the second image generation model to obtain the target image generation model, including:

[0014] According to the first preset rule, the first parameter of the first image generation model is replaced with the second parameter in the second image generation model that corresponds to the first parameter, so as to obtain the third image generation model;

[0015] The target image generation model is determined based on the third image generation model.

[0016] In another alternative to the first aspect, the target image generation model is determined based on the third image generation model, including:

[0017] The second random vector is input into the third image generation model to obtain the second random face image;

[0018] Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated.

[0019] The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

[0020] In another alternative to the first aspect, parameter replacement is performed on the first image generation model and the second image generation model to obtain the target image generation model, including:

[0021] According to the second preset rule, the third parameter of the second image generation model is replaced with the fourth parameter corresponding to the third parameter in the first image generation model to obtain the fourth image generation model;

[0022] The target image generation model is determined based on the fourth image generation model.

[0023] In another alternative to the first aspect, the second image generation model includes a first generation network and a second generation network;

[0024] The first generative network is used to generate random face images with pose angles that are the same as the target pose angles of real face images based on the input random vectors;

[0025] The second generator network is trained on sample images with known prediction values. The sample images include random face images with the same pose angle as the target pose angle of the real face image, real face images with the target pose angle, and real face images with pose angles other than the target pose angle. The random face images with the same pose angle as the target pose angle of the real face image are input into the second generator network to obtain the first prediction value. The real face images with the target pose angle are input into the second generator network to obtain the second prediction value. The real face images with the target pose angle are input into the second generator network to obtain the third prediction value.

[0026] The second image generation model is used to train the first generation network based on the predicted values ​​of random face images with the same pose angle as the target pose angle of the real face image obtained by the second generation network. This allows the trained first generation network to input random face images with the same pose angle as the target pose angle of the real face image obtained by the random vector into the second generation network to obtain target prediction values ​​belonging to a preset threshold range.

[0027] In another alternative to the first aspect, obtaining the face image to be processed includes:

[0028] Obtain the initial face image and extract the pose angles of the initial face image;

[0029] The initial face image and its pose angle are used as the face image to be processed.

[0030] Secondly, embodiments of this application also provide an image processing apparatus, the apparatus comprising:

[0031] The acquisition module is used to acquire the face image to be processed and obtain the vector corresponding to the face image to be processed based on the first image generation model;

[0032] The processing module is used to perform parameter replacement on the first image generation model and the second image generation model to obtain the target image generation model; the second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as that of the first image generation model.

[0033] The generation module is used to input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image.

[0034] In one alternative embodiment of the second aspect, the acquisition module includes:

[0035] The first acquisition unit is used to input the first random vector into the first image generation model to obtain the first random face image;

[0036] The first processing unit is used to construct a loss function based on the first random vector, the face image to be processed, and the random face image;

[0037] The second processing unit is used to optimize the loss function to obtain a vector corresponding to the face image to be processed.

[0038] In another alternative solution to the second aspect, the processing module includes:

[0039] The first generation unit is used to replace the first parameter of the first image generation model with the second parameter in the second image generation model corresponding to the first parameter according to the first preset rule, so as to obtain the third image generation model.

[0040] The first determining unit is used to determine the target image generation model based on the third image generation model.

[0041] In another alternative scheme of the second aspect, the first determining unit is specifically used for:

[0042] The second random vector is input into the third image generation model to obtain the second random face image;

[0043] Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated.

[0044] The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

[0045] In another alternative solution to the second aspect, the processing module includes:

[0046] The second generation unit is used to replace the third parameter of the second image generation model with the fourth parameter corresponding to the third parameter in the first image generation model according to the second preset rule, so as to obtain the fourth image generation model.

[0047] The second determining unit is used to determine the target image generation model based on the fourth image generation model.

[0048] In another alternative embodiment of the second aspect, the second image generation model includes a first generation network and a second generation network;

[0049] The first generative network is used to generate random face images with pose angles that are the same as the target pose angles of real face images based on the input random vectors;

[0050] The second generator network is trained on sample images with known prediction values. The sample images include random face images with the same pose angle as the target pose angle of the real face image, real face images with the target pose angle, and real face images with pose angles other than the target pose angle. The random face images with the same pose angle as the target pose angle of the real face image are input into the second generator network to obtain the first prediction value. The real face images with the target pose angle are input into the second generator network to obtain the second prediction value. The real face images with the target pose angle are input into the second generator network to obtain the third prediction value.

[0051] The second image generation model is used to train the first generation network based on the predicted values ​​of random face images with the same pose angle as the target pose angle of the real face image obtained by the second generation network. This allows the trained first generation network to input random face images with the same pose angle as the target pose angle of the real face image obtained by the random vector into the second generation network to obtain target prediction values ​​belonging to a preset threshold range.

[0052] In another alternative solution to the second aspect, the acquisition module further includes:

[0053] The second acquisition unit is used to acquire the initial face image and extract the pose angle of the initial face image;

[0054] The third processing unit is used to take the initial face image and the pose angle of the initial face image as the face image to be processed.

[0055] Thirdly, embodiments of this application also provide an image processing apparatus, including a processor and a memory;

[0056] The processor is connected to the memory;

[0057] Memory, used to store executable program code;

[0058] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the image processing method provided by the first aspect of the embodiments of this application or any implementation thereof.

[0059] Fourthly, embodiments of this application provide a computer storage medium storing a computer program, which includes program instructions. When executed by a processor, the program instructions can implement the image processing method provided by the first aspect or any implementation thereof of the embodiments of this application.

[0060] In this embodiment, a face image to be processed is first acquired, and a vector corresponding to the face image is obtained based on a first image generation model. Then, parameters are replaced in both the first and second image generation models to obtain a target image generation model. The vector corresponding to the face image to be processed is then input into this target image generation model to obtain the target face image. By fusing a real face generation model with a cartoon face generation model, the target cartoon face image obtained based on the fused model incorporates the identity features of a real face image, thus achieving a higher similarity to the face image to be processed. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram of the architecture of an image processing system provided in an embodiment of this application;

[0063] Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0064] Figure 3 An image processing effect diagram provided in an embodiment of this application;

[0065] Figure 4 A schematic diagram illustrating the processing flow of a second image generation model provided in an embodiment of this application;

[0066] Figure 5 A schematic flowchart illustrating another image processing method provided in an embodiment of this application;

[0067] Figure 6 A schematic flowchart illustrating another image processing method provided in an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0069] Figure 8 This is a schematic diagram of the structure of another image processing device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0071] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0072] When cartoonizing images of human faces, common processing techniques include two main approaches. One is based on a CycleGAN-like framework, using cycle loss to constrain the correlation between the input face image and the output cartoonized face image. The other is based on a StyleGAN-like framework, using inter-layer interpolation to constrain the correlation between the input face image and the output cartoonized face image. Taking the CycleGAN-based framework as an example, a deep neural network model combining attention modules and normalization functions can be used. Based on the acquired attention map, the translation is guided to focus on more important regions by distinguishing between the source and target domains. The acquired attention map mapping is embedded in the generator and discriminator networks, focusing on semantically important regions to facilitate shape transformation. While the generator network's attention map mapping focuses on distinguishing between the source and target domains, the discriminator network's attention map mapping adjusts parameters by aggregating the differences between real and fake images in the target domain.

[0073] However, when using various deep neural network models to cartoonize images with human faces, the aforementioned methods all employ unsupervised training, which cannot guarantee the similarity between the input face image and the output cartoonized face image.

[0074] Based on this, this application proposes one or more embodiments to improve the similarity between the output cartoon-style face image and the real face image, thereby providing users with a more satisfying image processing experience.

[0075] Please see Figure 1 , Figure 1 A schematic diagram of the architecture of an image processing system provided in an embodiment of this application is shown.

[0076] like Figure 1 As shown, the image processing system may include at least a mobile terminal 101 and a server 102, wherein:

[0077] Mobile terminal 101 can acquire a face image to be processed and send it to server 102 corresponding to mobile terminal 101, so that server 102 can obtain a vector corresponding to the face image to be processed based on a first image generation model. The face in the face image to be processed may have certain pose angles, which can be understood in conjunction with a spatial rectangular coordinate system established based on the face image to be processed. Specifically, a spatial rectangular coordinate system can be established with the center of the face in the face image to be processed as the origin. The X-axis and Y-axis can be set on the plane of the side view image, and the Z-axis can be perpendicular to the plane of the side view image. The face pose angles in the face image to be processed may include pose angles rotating around the X-axis, around the Y-axis, and around the Z-axis, for example, represented as (pitch, yaw, roll). It should be noted that the range of face pose angles in the face image to be processed can be between -90 degrees and 90 degrees. As a preferred embodiment, the mobile terminal 101 can acquire an image with a face pose angle of 0 as the face image to be processed, that is, the pose angle of the face rotating around the X-axis, the pose angle of the face rotating around the Y-axis, and the pose angle of the face rotating around the Z-axis are all 0.

[0078] The mobile terminal 101 can acquire the face image to be processed by either using its camera or by searching for it in a third-party application. For example, when acquiring the image using the mobile terminal 101's camera, the user can control the mobile terminal 101 to open an application capable of taking photos, such as, but not limited to, a camera app or a photo editing app, and capture an image containing a face. Similarly, when acquiring the image through a third-party application, the user can open a search-type third-party application installed on the mobile terminal 101, such as, but not limited to, a browser, and search for images containing a face in the corresponding search interface. It is understood that if the area of ​​the face region in the acquired face image is smaller than a preset threshold, the mobile terminal 101 can also enlarge the face image proportionally and select images where the face region area reaches the preset threshold as the face image to be processed.

[0079] The mobile terminal 101 involved in this embodiment may be a smartphone, tablet computer, desktop computer, laptop computer, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, PC device, personal digital assistant (PDA), router device, virtual reality device, etc.

[0080] After receiving a face image to be processed, server 102 can obtain a vector corresponding to the face image based on a first image generation model. It is understood that the first image generation model can be a trained deep learning neural network model that generates random face images based on input random vectors. Its network structure may include a generator network and a discriminator network. Specifically, the generator network generates random face images based on input random vectors, and the discriminator network is trained based on sample images with known prediction values. These sample images may include random face images and real face images. The random face image is input to the discriminator network to obtain a first prediction value, and the real face image is input to the discriminator network to obtain a second prediction value. This first image generation model is used to train the generator network based on the prediction values ​​corresponding to the random face images obtained by the discriminator network, so that the trained generator network, based on the random face images obtained from the random vectors, is input to the discriminator network to obtain target prediction values ​​belonging to a preset threshold range.

[0081] Furthermore, server 102 can also perform parameter replacement on the first image generation model and the second image generation model to obtain the target image generation model. The first image generation model, as described above, generates random face images based on input random vectors, and its network structure may include a generator network and a discriminator network. It is understood that the second image generation model can be trained based on the first image generation model; that is, the network structure of the second image generation model is the same as that of the first image generation model, and the initial parameters of the second image generation model are the same as those of the trained first image generation model. The trained second image generation model can generate random cartoonish face images based on input random vectors, but the similarity between these random cartoonish face images and the face system to be processed cannot be guaranteed.

[0082] Since the network structures of the first and second image generation models after training are the same, but their model parameters are different, the target image generation model can be obtained by replacing some parameters of the first image generation model with corresponding parameters in the second image generation model, or by replacing some parameters of the second image generation model with corresponding parameters in the first image generation model. Here, the target image generation model can generate random cartoon-style face images based on the input random vector. These random cartoon-style face images contain more facial feature information than the cartoon-style face images obtained by the second image generation model.

[0083] Furthermore, server 102 can also input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image. It is understood that the target image generation model can obtain a cartoonish face image with a higher similarity to the face image to be processed based on the input vector corresponding to the face image to be processed.

[0084] The server 102 involved in this embodiment is specifically, but not limited to, a hardware server, a virtual server, a cloud server, etc., and may also be a terminal, specifically, but not limited to, desktop, laptop, notebook computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), routing device, gateway device, etc.

[0085] It should also be noted that the image processing system mentioned above is not limited to being executed jointly by the mobile terminal 101 and the server 102. For example, it can also be executed by the mobile terminal 101 or the server 102 alone. This embodiment is not limited to this.

[0086] Next, this application will explain and describe several embodiments of the image processing method.

[0087] Please see Figure 2 , Figure 2 A schematic flowchart of an image processing method provided in an embodiment of this application is shown.

[0088] like Figure 2 As shown, the image processing method may include at least the following steps:

[0089] Step 202: Obtain the face image to be processed, and obtain the vector corresponding to the face image to be processed based on the first image generation model.

[0090] Specifically, when cartoonizing an image, a face image to be processed can be obtained first. The face in this image may have a certain pose angle. This pose angle can be understood in conjunction with a spatial Cartesian coordinate system established based on the face image. For example, a spatial Cartesian coordinate system can be established with the center of the face as the origin, the X-axis and Y-axis can be set on the plane of the profile image, and the Z-axis can be perpendicular to the plane of the profile image. The face pose angle in the face image to be processed can include the pose angles of rotation around the X-axis, the Y-axis, and the Z-axis, which can be represented as (pitch, yaw, roll). It should be noted that the range of these face pose angles can be between -90 degrees and 90 degrees. Preferably, an image with a face pose angle of 0 can be obtained as the face image to be processed, in order to obtain the cartoonized face image with the best cartoonized effect.

[0091] Furthermore, after acquiring the face image to be processed, a vector corresponding to the face image can be obtained based on the first image generation model. Here, the first image generation model can be a trained deep learning neural network, which generates random face images based on the input random vectors. Based on this, multiple random vectors can be input into the first image generation model to obtain their respective corresponding random face images, and each random face image is matched with the acquired face image to be processed. The random vector corresponding to the random face image that meets the preset requirements based on the matching result can be used as the vector corresponding to the face image to be processed.

[0092] The method of matching random face images with face images to be processed mentioned here can be, but is not limited to, comparing the similarity of each random face image with the face image to be processed separately. The reference target for the similarity comparison can be the feature points extracted from the image, such as eyes, nose, mouth, and contours. The random face image with the highest similarity result is regarded as the face image that is closest to the face image to be processed, and its corresponding random vector is the vector corresponding to the face image to be processed.

[0093] It should be noted that the fake face image obtained by the first image generation model based on the input vector has a very high similarity to the real face image corresponding to the vector. The discriminative network in the first image generation model cannot accurately distinguish between the fake face image and the real face image corresponding to the vector. It is understood that the first image generation model in this embodiment can also perform frontal face processing on face images with profile views to obtain fake frontal face images corresponding to face images with profile views, but this application is not limited to this.

[0094] Step 204: Replace the parameters of the first image generation model and the second image generation model to obtain the target image generation model.

[0095] Specifically, after obtaining the vector corresponding to the face image to be processed based on the first image generation model, a second image generation model can be trained through transfer learning based on the first image generation model. The network structure of the second image generation model can be the same as that of the first image generation model. Taking the first image generation model as including a generator network and a discriminator network as an example, the second image generation model can include a generator network with the same structure as the generator network of the first image generation model, and a discriminator network with the same structure as the discriminator network of the first image generation model. Furthermore, the initial parameters of the second image generation model are the parameters of the trained first image generation model. It is understood that the parameters of the trained second image generation model are different from those of the first image generation model. The trained second image generation model can generate random cartoon-style face images based on input random vectors, but when the vector corresponding to a real face image is input to the trained second image generation model, the resulting cartoon-style face image has low similarity to the real face image.

[0096] Furthermore, after obtaining the trained second image generation model, the parameters of both the first and second image generation models can be replaced to obtain the target image generation model. This target image generation model may have the same network structure as either the first or second image generation model. For example, if the first image generation model includes a generator network and a discriminator network, the target image generation model may include a generator network with the same structure as the generator network of the first image generation model, and a discriminator network with the same structure as the discriminator network of the first image generation model. Similarly, if the second image generation model includes both a generator network and a discriminator network, the target image generation model may include a generator network with the same structure as the generator network of the second image generation model, and a discriminator network with the same structure as the discriminator network of the second image generation model.

[0097] It is understood that the parameters of the target image generation model may include the parameters of the first image generation model and the parameters of the second image generation model. Since the network structure of the target image generation model is the same as that of both the first and second image generation models, each parameter of the target image generation model corresponds to either a parameter of the first or second image generation model. Taking a target image generation model with five parameters A, B, C, D, and E as an example, the first image generation model may include parameter A1 corresponding to parameter A, parameter B1 corresponding to parameter B, parameter C1 corresponding to parameter C, parameter D1 corresponding to parameter D, and parameter E1 corresponding to parameter E. The second image generation model may include parameter A2 corresponding to parameter A, parameter B2 corresponding to parameter B, parameter C2 corresponding to parameter C, parameter D2 corresponding to parameter D, and parameter E2 corresponding to parameter E. At this point, parameter A of the target image generation model can be either parameter A1 of the first image generation model or parameter A2 of the second image generation model; parameter B can be either parameter B1 of the first image generation model or parameter B2 of the second image generation model; parameter C can be either parameter C1 of the first image generation model or parameter C2 of the second image generation model; parameter D can be either parameter D1 of the first image generation model or parameter D2 of the second image generation model; and parameter E can be either parameter E1 of the first image generation model or parameter E2 of the second image generation model. For example, the parameters of the target image generation model can be, but are not limited to, A1, B1, C1, D2, and E2.

[0098] At this point, the target image generation model can obtain a cartoonish face image based on the input vector, and this cartoonish face image has a higher similarity to the real face image corresponding to the input vector.

[0099] Step 206: Input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image.

[0100] Specifically, the vector corresponding to the face image to be processed can be input into the target image generation model to obtain the target face image after cartoonization of the face image to be processed.

[0101] See also: Figure 3 The illustration shows an image processing effect provided by an embodiment of this application. Figure 3 As shown, Figure 3 Figure 3a shows a fake face image obtained by the first image generation model based on the vector corresponding to the face image to be processed, which is most similar to the face image to be processed. Figure 3Figure 3b shows a cartoonish face image obtained by the second image generation model based on the vector corresponding to the image to be processed. The similarity between the cartoonish face image and the image to be processed or the fake face image shown in 3a is not high. Figure 3 Figure 3c shows a cartoonish face image obtained by the target image generation model based on the vector corresponding to the face image to be processed. It has a significantly higher similarity to the face image to be processed or the fake face image shown in 3a.

[0102] In this embodiment of the application, by fusing a real face generation model with a cartoon face generation model, the target cartoon face image obtained based on the fusion model can be combined with the identity features of a real face image, thereby achieving a higher similarity with the face image to be processed.

[0103] As an optional step in this embodiment, acquiring the face image to be processed includes:

[0104] Obtain the initial face image and extract the pose angles of the initial face image;

[0105] The initial face image and pose angle are used as the face image to be processed.

[0106] Specifically, in order to ensure that the face angles of the generated cartoon-style face image are consistent with the face angles in the face image to be processed during the training of the second image generation model, the pose angles of the face in the initial face image can be extracted after obtaining the initial face image with the face.

[0107] The method for extracting the pose angles of the face in the initial face image can be as follows: first, identify the face region in the initial face image, perform 3D modeling on this face region, and determine the key feature points of the face in the 3D face model. Understandably, algorithms can be used to detect multiple feature points in the 3D face model, and the most representative key feature points can be determined from these multiple feature points. For example, the feature points corresponding to the left and right corners of the eyes, the tip of the nose, the left and right corners of the mouth, and the jaw can be used as key feature points. After determining the key feature points, an affine transformation matrix can be determined based on these key feature points from the 3D face model to the face in the initial face image. This affine transformation matrix can include rotation vectors and translation vectors, and this process can be, but is not limited to, using OpenCV's `solvePnP` function to solve it. Further, after obtaining the rotation vector, the rotation vector can be converted into a rotation matrix, and then the rotation matrix can be converted into corresponding pose angles. Here, pose angles can include pose angles around the X-axis, pose angles around the Y-axis, and pose angles around the Z-axis.

[0108] After extracting the pose angles of the initial face image, the initial face image and its pose angles can be used as the face image to be processed. To ensure that the generated cartoon face image has the same face angles as the face image to be processed, the pose angles of the initial face image can be used as input to train the second image generation model, or the pose angles of the initial face image can be used as input to train the first image generation model. This ensures that the cartoon face image obtained by the target image generation model based on the input vector has the same face angles as the real face image, thus satisfying the user's image processing experience.

[0109] Of course, during the training of the first image generation model or the second image generation model, a specified pose angle can be input for training, so that the final target image generation model can output a cartoonish face image with a specified pose angle based on the input vector of the real face image, in order to cater to the user's diverse image processing experience.

[0110] As another option in this embodiment, the second image generation model includes a first generation network and a second generation network;

[0111] The first generative network is used to generate random face images with pose angles that are the same as the target pose angles of real face images based on the input random vectors;

[0112] The second generator network is trained on sample images with known prediction values. The sample images include random face images with the same pose angle as the target pose angle of the real face image, real face images with the target pose angle, and real face images with pose angles other than the target pose angle. The random face images with the same pose angle as the target pose angle of the real face image are input into the second generator network to obtain the first prediction value. The real face images with the target pose angle are input into the second generator network to obtain the second prediction value. The real face images with the target pose angle are input into the second generator network to obtain the third prediction value.

[0113] The second image generation model is used to train the first generation network based on the predicted values ​​of random face images with the same pose angle as the target pose angle of the real face image obtained by the second generation network. This allows the trained first generation network to input random face images with the same pose angle as the target pose angle of the real face image obtained by the random vector into the second generation network to obtain target prediction values ​​belonging to a preset threshold range.

[0114] See here. Figure 4 The diagram illustrates a processing flow of a second image generation model provided in an embodiment of this application. For example... Figure 4 As shown, the second image generation model includes a first generation network and a second generation network, wherein:

[0115] The first generator network can be considered an image generation network used to generate random cartoon-style face images with the same pose angle as the target pose angle based on an input random vector. This random vector can be, but is not limited to, one or more random vectors that satisfy a normal distribution. Each random vector input into the first generator network produces a different random cartoon-style face image, but all share the same target pose angle. It should be noted that, to enable the first generator network to generate cartoon-style face images with the target pose angle more accurately during training, the pose angle vector corresponding to the target pose angle and the random vector can be input together into the first generator network for training.

[0116] The second generator network can be considered an image discrimination network, used to obtain a predicted value representing whether an input face image belongs to a real image. Possibly, when the input sample image is a random cartoonish face image with the same pose angle as the target pose angle of a real face image, the second generator network outputs a first predicted value representing that it does not belong to a real cartoonish face image, for example, but not limited to, a first predicted value of 0. Possibly, when the input sample image is a real cartoonish face image with a pose angle equal to the target pose angle, the second generator network outputs a second predicted value representing that it belongs to a real cartoonish face image, for example, but not limited to, a second predicted value of 1. Possibly, when the input sample image is a real cartoonish face image with a pose angle unrelated to the target pose angle, the second generator network outputs a third predicted value representing that it belongs to a real cartoonish face image. Here, the third predicted value can be understood as a real cartoonish face image with a pose angle that does not meet the target pose angle, for example, but not limited to, the third predicted value being between 0.5 and 1. Understandably, when the predicted value output by the second generator network approaches or equals 0.5, it indicates that the second generator network cannot determine the realism of the input cartoonish face image. It should also be noted that the aforementioned real cartoonish face image whose pose angle is independent of the target pose angle can be understood as a real cartoonish face image with an excessively large pose angle, such as a real cartoonish face image with a pose angle of 180 degrees rotated around the X-axis.

[0117] During the training of the second image generation model, since it includes a second generator network (i.e., a discriminator network), its training optimization process can be understood as finding a Nash balance between the first and second generator networks. In other words, the purpose of training the first generator network in the second image generation model is to enable the first generator network to obtain a cartoonish face image with a target pose angle that approximates the true pose angle based on the input vector (or a vector that includes the target pose angle). The purpose of training the second generator network in the second image generation model is to enable the second generator network to obtain a cartoonish face image with a target pose angle that approximates the true pose angle based on the cartoonish face image obtained by the first generator network. The overall training process can be understood as the alternating iterative training of the first and second generator networks.

[0118] Specifically, when training the second image generation model, multiple cartoonish face images with real pose angles as target pose angles can be set first, along with the expected number of iteration steps (or training steps). Furthermore, pose angle extraction can be performed on the cartoonish face images with real pose angles as target pose angles to obtain pose angle vectors corresponding to the target pose angles. Then, a Gaussian distribution z ~ N(0,1) is applied. z A random vector (which can also be a noise vector) of the same preset number is randomly sampled. This random vector, along with the pose angle vector corresponding to the target pose angle, is input into a first generator network with initialized parameters to obtain a randomized cartoon face image with the target pose angle. It can be understood that the initialization parameters in the first generator network are the same as the network structure parameters corresponding to the first generator network in the trained first image generation model. Next, this randomized cartoon face image with the target pose angle is converted into vector form and input into a second generator network to obtain a first predicted value representing a face image that does not belong to a real cartoon face image. Based on this first predicted value, a first prediction function is constructed, which can be expressed as S. f Next, the real cartoon-style face image with the target pose angle is converted into vector form and input into the second generator network to obtain a second predicted value representing a real cartoon-style face image. Based on this second predicted value, a second prediction function is constructed, which can be expressed as S. r Next, the real cartoon-style face image whose pose angle is unrelated to the target pose angle is converted into vector form and input into the second generator network to obtain a third predicted value representing a real cartoon-style face image but whose pose angle does not meet the requirements. Based on this third predicted value, a third prediction function is constructed, which can be expressed as S. w Next, based on the first prediction function, the second prediction function, and the third prediction function obtained above, a second generator network objective function can be constructed, which can be expressed as shown in formula (1):

[0119] L D ←log(sr )+(log(1-s w )+log(1-s f )) / twenty one)

[0120] Here, the objective function of the second generative network can be iteratively trained using gradient descent to make L D The parameters of the second generator network are optimized during the iteration process, approaching their maximum value.

[0121] Furthermore, after optimizing the parameters of the second generator network, the first generator network can be based on the first prediction function S corresponding to the first predicted value. f The objective function of the first generator network is constructed, and its form can be expressed as shown in formula (2):

[0122] L G ←log(s f (2)

[0123] Here, the objective function of the first generator network can be iteratively trained using gradient descent to make L G The algorithm approaches its maximum value and optimizes the parameters of the first generator network during the iteration process.

[0124] It should be noted that when a random cartoonish face image with the target pose angle obtained from the first generation network is input into the second generation network and the predicted value is 0.5 or close to 0.5, it indicates that the second generation network cannot distinguish the authenticity of the input random cartoonish face image with the target pose angle, and it can be determined that the training of the second image generation model is complete.

[0125] Please see Figure 5 , Figure 5 A flowchart illustrating another image processing method provided in an embodiment of this application is shown.

[0126] like Figure 5 As shown, the image processing method may include at least the following steps:

[0127] Step 502: Obtain the face image to be processed.

[0128] Specifically, step 502 can be referred to step 202, and will not be elaborated on here.

[0129] Step 504: Input the first random vector into the first image generation model to obtain the first random face image.

[0130] The first random vector here can be, but is not limited to, a random vector that satisfies a normal distribution. The first image generation model may include a first generation network and a second generation network. The first generation network is used to obtain random face images based on the input random vector, and the second generation network is used to train on sample images with known prediction values. The sample images include random face images and real face images. The random face images are input into the second generation network to obtain a first prediction value, and the real face images are input into the second generation network to obtain a second prediction value. This first image generation model is used to train the first generation network based on the prediction values ​​corresponding to the random face images obtained by the second generation network, so that the trained first generation network, when input into the second generation network based on the random face images obtained from the random vector, obtains target prediction values ​​belonging to a preset threshold range.

[0131] Specifically, a random vector can be collected from vectors that satisfy a normal distribution and input into the first generation network in the first image generation model to obtain a random face image, and this random face image can be used as the first random face image.

[0132] Step 506: Construct a loss function based on the first random vector, the face image to be processed, and the first random face image.

[0133] Specifically, the first random vector can be denoted as Z, the third face image can be converted into a vector representation denoted as G(Z), and the first face image can be converted into a vector representation denoted as X. A loss function is constructed based on the first random vector, the face image to be processed, and the first random face image. Z Specifically, it can be expressed by the following formula (3):

[0134]

[0135] Step 508: Optimize the loss function to obtain the vector corresponding to the face image to be processed.

[0136] Specifically, the first random vector can be iteratively optimized by minimizing ||G(Z)-X|| in the above formula (3) to ensure that the random face image obtained by inputting the optimized first random vector into the first generator network is as close as possible to the face image to be processed. Here, the expected number of iteration steps (or training times) can be set, the loss function can be iteratively trained based on the gradient descent method, and the first random vector can be optimized according to the training results to obtain the vector corresponding to the face image to be processed.

[0137] Step 510: Replace the parameters of the first image generation model and the second image generation model to obtain the target image generation model.

[0138] Specifically, step 510 is the same as step 204, so I will not go into details here.

[0139] Step 512: Input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image.

[0140] Specifically, step 512 is the same as step 206, so I will not go into details here.

[0141] In this embodiment, the vector corresponding to the face image to be processed can be obtained through iterative optimization, so as to improve the processing speed of obtaining the vector and ensure the accuracy of the obtained vector, thereby making the similarity between the target face image obtained based on the target image generation model and the face image to be processed higher.

[0142] Please see Figure 6 , Figure 6 A flowchart illustrating another image processing method provided in an embodiment of this application is shown.

[0143] like Figure 6 As shown, the image processing method may include at least the following steps:

[0144] Step 602: Obtain the face image to be processed, and obtain the vector corresponding to the face image to be processed based on the first image generation model.

[0145] Specifically, step 602 is the same as step 202, and will not be elaborated further here.

[0146] Step 604: Replace the first parameter of the first image generation model with the second parameter corresponding to the first parameter in the second image generation model according to the first preset rule to obtain the third image generation model.

[0147] Specifically, after training the first image generation model to obtain the second image generation model, the first parameters in the first image generation model can be replaced with the second parameters in the second image generation model according to a first preset rule to obtain multiple third image generation models. Since the second image generation model is trained based on the first image generation model, the network structure of the first image generation model is the same as that of the second image generation model. Therefore, each parameter of the first image generation model corresponds to a parameter of the second image generation model. For example, if the parameters of the first image generation model are a1, b1, and c1, then the parameters of the second image generation model can be represented as a2, b2, and c2. It is understood that the first preset rule can be, but is not limited to, replacing any n first parameters in all parameters of the first image generation model with the corresponding n second parameters in the second image generation model. The total number of parameters in the first image generation model can be m, and m is greater than n. Taking the parameters of the first image generation model mentioned above as a1, b1, c1, and the parameters of the second image generation model as a2, b2, c2 as an example, the first preset rule can be to replace any two of the three parameters of the first image generation model with the two corresponding second parameters of the second image generation model. That is to say, three third image generation models with different parameters can be formed, which can be represented as a third image generation model including parameters a1, b2, c2, a third image generation model including parameters a2, b1, c2, and a third image generation model including parameters a2, b2, c1.

[0148] It is also understood that, optionally, the parameters used to represent style and texture features in the first image generation model can be replaced with the parameters used to represent style and texture features in the second image generation model, so that the resulting third image generation model can include the parameters used to represent identity features in the first image generation model and the parameters used to represent style and texture features in the second image generation model, thereby ensuring that the cartoonish face image obtained by the third image generation model has both the identity features of a real face image and the texture features of a cartoonish face.

[0149] Step 606: Determine the target image generation model based on the third image generation model.

[0150] Specifically, after obtaining multiple third image generation models according to the first preset rule, the similarity between each third image generation model and the real face image can be judged by combining the cartoon face image obtained by each third image generation model based on the input vector, and the third image generation model corresponding to the cartoon face image with the highest similarity is taken as the target image generation model.

[0151] Step 608: Input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image.

[0152] Specifically, step 608 is the same as step 206, so I will not go into details here.

[0153] In this embodiment, a target image generation model combining different feature information from two image generation models can be obtained by replacing the parameters of the first image generation model with the parameters of the second image generation model. This ensures that the cartoon-style face image obtained by the target image generation model has the feature information of a real face image, thereby improving the similarity with the real face image.

[0154] As a preferred embodiment, the method of determining the target image generation model based on the third image generation model may include:

[0155] The second random vector is input into the third image generation model to obtain the second random face image;

[0156] Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated.

[0157] The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

[0158] Specifically, after obtaining multiple third image generation models according to the first preset rule, a second random vector can be input into each third image generation model to obtain a second random cartoonized face image corresponding to each third image generation model. Feature extraction is then performed on each second random cartoonized face image and a real face image. The second random vector can be a vector corresponding to the real face image, for example, but not limited to obtaining a second random vector corresponding to the real face image based on the aforementioned first image generation model; this will not be elaborated further here. It is understood that the feature extraction method can be, but is not limited to, inputting each second random cartoonized face image and the real face image into a trained face feature extraction model to obtain the face features corresponding to each face image. Here, the face features can be key features such as facial contours, nose, eyes, and mouth.

[0159] Furthermore, the feature extraction results of each second random cartoon face image can be compared with the feature extraction results of the real face image to calculate the similarity between each second random cartoon face image and the real face image. The similarity calculation method here can be, but is not limited to, first calculating the number of facial feature points in the second random cartoon face image that overlap with those in the real face image, and then calculating the ratio of this overlapping number of facial feature points to the total number of facial feature points in the real face image. It can be understood that a higher ratio indicates a higher similarity between the second random cartoon face image and the real face image, and the third image generation model corresponding to the second random cartoon face image with the highest similarity can be used as the target image generation model.

[0160] As another preferred embodiment, the target image generation model can be obtained by replacing the parameters of the first image generation model and the second image generation model.

[0161] According to the second preset rule, the third parameter of the second image generation model is replaced with the fourth parameter corresponding to the third parameter in the first image generation model to obtain the fourth image generation model;

[0162] The target image generation model is determined based on the fourth image generation model.

[0163] Specifically, after training the first image generation model to obtain the second image generation model, the third parameters in the second image generation model can be replaced with the fourth parameters in the first image generation model according to a second preset rule to obtain multiple fourth image generation models. Since the second image generation model is trained based on the first image generation model, the network structure of the first image generation model is the same as that of the second image generation model. Therefore, each parameter of the first image generation model corresponds to a parameter of the second image generation model. For example, if the parameters of the first image generation model are a1, b1, and c1, then the parameters of the second image generation model can be represented as a2, b2, and c2. It is understood that the second preset rule can be, but is not limited to, replacing any n third parameters in all parameters of the second image generation model with the n corresponding fourth parameters in the first image generation model. The total number of parameters in the second image generation model can be m, and m is greater than n. Taking the parameters of the first image generation model mentioned above as a1, b1, c1, and the parameters of the second image generation model as a2, b2, c2 as an example, the second preset rule can be to replace any two of the three third parameters of the second image generation model with the two fourth parameters of the first image generation model that correspond to each of the two third parameters. That is to say, three fourth image generation models with different parameters can be formed, which can be represented as a fourth image generation model including parameters a1, b1, c2, a fourth image generation model including parameters a1, b2, c1, and a fourth image generation model including parameters a2, b1, c1.

[0164] It is also understood that, optionally, the parameters used to represent identity features in the second image generation model can be replaced with the parameters used to represent identity features in the first image generation model, so that the resulting fourth image generation model can include the parameters used to represent identity features in the first image generation model and the parameters used to represent style and texture features in the second image generation model, thereby ensuring that the cartoonized face image obtained by the fourth image generation model has both the identity features of a real face image and the texture features of a cartoonized face.

[0165] Furthermore, the method for determining the target image generation model based on the fourth image generation model can be found in the above embodiments, and will not be elaborated further here.

[0166] Please see Figure 7 , Figure 7 A schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application is shown.

[0167] like Figure 7As shown, the image processing apparatus may include at least an acquisition module 701, a processing module 702, and a generation module 703, wherein:

[0168] The acquisition module 701 is used to acquire the face image to be processed and obtain the vector corresponding to the face image to be processed based on the first image generation model;

[0169] Processing module 702 is used to perform parameter replacement on the first image generation model and the second image generation model to obtain the target image generation model; the second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as the network structure of the first image generation model.

[0170] The generation module 703 is used to input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image.

[0171] In some possible embodiments, the acquisition module 701 includes:

[0172] The first acquisition unit is used to input the first random vector into the first image generation model to obtain the first random face image;

[0173] The first processing unit is used to construct a loss function based on the first random vector, the face image to be processed, and the random face image;

[0174] The second processing unit is used to optimize the loss function to obtain a vector corresponding to the face image to be processed.

[0175] In some possible embodiments, the processing module 702 includes:

[0176] The first generation unit is used to replace the first parameter of the first image generation model with the second parameter in the second image generation model corresponding to the first parameter according to the first preset rule, so as to obtain the third image generation model.

[0177] The first determining unit is used to determine the target image generation model based on the third image generation model.

[0178] In some possible embodiments, the first determining unit is specifically used for:

[0179] The second random vector is input into the third image generation model to obtain the second random face image;

[0180] Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated.

[0181] The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

[0182] In some possible embodiments, the processing module 702 includes:

[0183] The second generation unit is used to replace the third parameter of the second image generation model with the fourth parameter corresponding to the third parameter in the first image generation model according to the second preset rule, so as to obtain the fourth image generation model.

[0184] The second determining unit is used to determine the target image generation model based on the fourth image generation model.

[0185] In some possible embodiments, the second image generation model includes a first generation network and a second generation network;

[0186] The first generative network is used to generate random face images with pose angles that are the same as the target pose angles of real face images based on the input random vectors;

[0187] The second generator network is trained on sample images with known prediction values. The sample images include random face images with the same pose angle as the target pose angle of the real face image, real face images with the target pose angle, and real face images with pose angles other than the target pose angle. The random face images with the same pose angle as the target pose angle of the real face image are input into the second generator network to obtain the first prediction value. The real face images with the target pose angle are input into the second generator network to obtain the second prediction value. The real face images with the target pose angle are input into the second generator network to obtain the third prediction value.

[0188] The second image generation model is used to train the first generation network based on the predicted values ​​of random face images with the same pose angle as the target pose angle of the real face image obtained by the second generation network. This allows the trained first generation network to input random face images with the same pose angle as the target pose angle of the real face image obtained by the random vector into the second generation network to obtain target prediction values ​​belonging to a preset threshold range.

[0189] In some possible embodiments, the acquisition module 701 further includes:

[0190] The second acquisition unit is used to acquire the initial face image and extract the pose angle of the initial face image;

[0191] The third processing unit is used to take the initial face image and the pose angle of the initial face image as the face image to be processed.

[0192] Please see Figure 8 , Figure 8A schematic diagram of the structure of another image processing apparatus provided in an embodiment of this application is shown.

[0193] like Figure 8 As shown, the image processing device 800 may include at least one processor 801, at least one network interface 804, a user interface 803, a memory 805, and at least one communication bus 802.

[0194] The communication bus 802 can be used to realize the connection and communication of the above components.

[0195] The user interface 803 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0196] The network interface 804 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0197] The processor 801 may include one or more processing cores. The processor 801 connects to various parts within the electronic device 800 using various interfaces and lines. It executes instructions, programs, code sets, or instruction sets stored in the memory 805, and calls data stored in the memory 805 to perform various functions and process data within the routing device 800. Optionally, the processor 801 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 801 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 801 and may be implemented as a separate chip.

[0198] The memory 805 may include RAM or ROM. Optionally, the memory 805 may include a non-transitory computer-readable medium. The memory 805 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned processor 801. Figure 8 As shown, the memory 805, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an image processing application.

[0199] Specifically, the processor 801 can be used to call the image processing application stored in the memory 805 and perform the following operations:

[0200] Obtain the face image to be processed, and obtain the vector corresponding to the face image to be processed based on the first image generation model;

[0201] The parameters of the first image generation model and the second image generation model are replaced to obtain the target image generation model; the second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as that of the first image generation model.

[0202] The vector corresponding to the face image to be processed is input into the target image generation model to obtain the target face image.

[0203] In some possible embodiments, when the processor 801 obtains the vector corresponding to the face image to be processed based on the first image generation model, it is used to perform:

[0204] The first random vector is input into the first image generation model to obtain the first random face image;

[0205] A loss function is constructed based on the first random vector, the face image to be processed, and the random face image;

[0206] The loss function is optimized to obtain a vector corresponding to the face image to be processed.

[0207] In some possible embodiments, when the processor 801 performs parameter substitution on the first image generation model and the second image generation model to obtain the target image generation model, it is used to execute:

[0208] According to the first preset rule, the first parameter of the first image generation model is replaced with the second parameter in the second image generation model that corresponds to the first parameter, so as to obtain the third image generation model;

[0209] The target image generation model is determined based on the third image generation model.

[0210] In some possible embodiments, when processor 801 determines the target image generation model based on the third image generation model, it performs the following:

[0211] The second random vector is input into the third image generation model to obtain the second random face image;

[0212] Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated.

[0213] The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

[0214] In some possible embodiments, when the processor 801 performs parameter substitution on the first image generation model and the second image generation model to obtain the target image generation model, it is used to execute:

[0215] According to the second preset rule, the third parameter of the second image generation model is replaced with the fourth parameter corresponding to the third parameter in the first image generation model to obtain the fourth image generation model;

[0216] The target image generation model is determined based on the fourth image generation model.

[0217] In some possible embodiments, the second image generation model includes a first generation network and a second generation network;

[0218] The first generative network is used to generate random face images with pose angles that are the same as the target pose angles of real face images based on the input random vectors;

[0219] The second generator network is trained on sample images with known prediction values. The sample images include random face images with the same pose angle as the target pose angle of the real face image, real face images with the target pose angle, and real face images with pose angles other than the target pose angle. The random face images with the same pose angle as the target pose angle of the real face image are input into the second generator network to obtain the first prediction value. The real face images with the target pose angle are input into the second generator network to obtain the second prediction value. The real face images with the target pose angle are input into the second generator network to obtain the third prediction value.

[0220] The second image generation model is used to train the first generation network based on the predicted values ​​of random face images with the same pose angle as the target pose angle of the real face image obtained by the second generation network. This allows the trained first generation network to input random face images with the same pose angle as the target pose angle of the real face image obtained by the random vector into the second generation network to obtain target prediction values ​​belonging to a preset threshold range.

[0221] In some possible embodiments, when processor 801 acquires the face image to be processed, it performs the following:

[0222] Obtain the initial face image and extract the pose angles of the initial face image;

[0223] The initial face image and its pose angle are used as the face image to be processed.

[0224] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the above-described instructions. Figure 2 or Figure 5 or Figure 6 One or more steps in the illustrated embodiment. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0225] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

[0226] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0227] The embodiments described above are merely preferred embodiments of this application and are not intended to limit the scope of this application. Any modifications and improvements made by those skilled in the art to the technical solutions of this application without departing from the spirit of this application should fall within the protection scope defined by the claims of this application.

Claims

1. An image processing method, characterized in that, include: Obtain the face image to be processed, and obtain the vector corresponding to the face image to be processed based on the first image generation model; By replacing the parameters of the first image generation model and the second image generation model, the target image generation model is obtained. The second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as that of the first image generation model. The vector corresponding to the face image to be processed is input into the target image generation model to obtain the target face image; The second image generation model includes a first generation network and a second generation network; The first generator network is used to generate a random face image with the same pose angle as the target pose angle of the real face image based on the input random vector; The second generation network is trained on sample images with known prediction values. The sample images include random face images whose pose angles are the same as the target pose angles of real face images, real face images whose pose angles are the target pose angles, and real face images whose pose angles are not the target pose angles. The random face images whose pose angles are the same as the target pose angles of real face images are input into the second generation network to obtain a first prediction value. The real face images whose pose angles are the target pose angles are input into the second generation network to obtain a second prediction value. The real face images whose pose angles are not the target pose angles are input into the second generation network to obtain a third prediction value. The second image generation model is used to train the first generation network based on the predicted value of a random face image with the same pose angle as the target pose angle of the real face image obtained by the second generation network, so that the trained first generation network is input into the second generation network based on the random face image with the same pose angle as the target pose angle of the real face image obtained by the random vector to obtain a target prediction value belonging to a preset threshold range.

2. The method according to claim 1, characterized in that, The step of obtaining the vector corresponding to the face image to be processed based on the first image generation model includes: The first random vector is input into the first image generation model to obtain the first random face image; A loss function is constructed based on the first random vector, the face image to be processed, and the random face image; The loss function is optimized to obtain a vector corresponding to the face image to be processed.

3. The method according to claim 1, characterized in that, The step of replacing the parameters of the first image generation model and the second image generation model to obtain the target image generation model includes: According to the first preset rule, the first parameter of the first image generation model is replaced with the second parameter in the second image generation model that corresponds to the first parameter, so as to obtain the third image generation model; The target image generation model is determined based on the third image generation model.

4. The method according to claim 3, characterized in that, The step of determining the target image generation model based on the third image generation model includes: The second random vector is input into the third image generation model to obtain a second random face image; Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated. The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

5. The method according to claim 1, characterized in that, The step of replacing the parameters of the first image generation model and the second image generation model to obtain the target image generation model includes: According to the second preset rule, the third parameter of the second image generation model is replaced with the fourth parameter in the first image generation model that corresponds to the third parameter, to obtain the fourth image generation model; The target image generation model is determined based on the fourth image generation model.

6. The method according to claim 1, characterized in that, The process of acquiring the face image to be processed includes: Acquire an initial face image and extract the pose angle of the initial face image; The initial face image and its pose angle are used as the face image to be processed.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the face image to be processed and obtain the vector corresponding to the face image to be processed based on the first image generation model; The processing module is used to perform parameter replacement on the first image generation model and the second image generation model to obtain the target image generation model; The second image generation model is trained based on the first image generation model, and the network structure of the second image generation model is the same as that of the first image generation model. The generation module is used to input the vector corresponding to the face image to be processed into the target image generation model to obtain the target face image; The second image generation model includes a first generation network and a second generation network; The first generator network is used to generate a random face image with the same pose angle as the target pose angle of the real face image based on the input random vector; The second generation network is trained on sample images with known prediction values. The sample images include random face images whose pose angles are the same as the target pose angles of real face images, real face images whose pose angles are the target pose angles, and real face images whose pose angles are not the target pose angles. The random face images whose pose angles are the same as the target pose angles of real face images are input into the second generation network to obtain a first prediction value. The real face images whose pose angles are the target pose angles are input into the second generation network to obtain a second prediction value. The real face images whose pose angles are not the target pose angles are input into the second generation network to obtain a third prediction value. The second image generation model is used to train the first generation network based on the predicted value of a random face image with the same pose angle as the target pose angle of the real face image obtained by the second generation network, so that the trained first generation network is input into the second generation network based on the random face image with the same pose angle as the target pose angle of the real face image obtained by the random vector to obtain a target prediction value belonging to a preset threshold range.

8. The apparatus according to claim 7, characterized in that, The acquisition module includes: The first acquisition unit is used to input the first random vector into the first image generation model to obtain the first random face image; The first processing unit is configured to construct a loss function based on the first random vector, the face image to be processed, and the random face image; The second processing unit is used to optimize the loss function to obtain a vector corresponding to the face image to be processed.

9. The apparatus according to claim 7, characterized in that, The processing module includes: The first generation unit is used to replace the first parameter of the first image generation model with the second parameter in the second image generation model corresponding to the first parameter according to the first preset rule, so as to obtain the third image generation model. The first determining unit is used to determine the target image generation model based on the third image generation model.

10. The apparatus according to claim 9, characterized in that, The first determining unit is specifically used for: The second random vector is input into the third image generation model to obtain a second random face image; Feature extraction is performed on the second random face image and the real face image respectively, and the similarity between the feature extraction results of the second random face image and the feature extraction results of the real face image is calculated. The third image generation model corresponding to the second random face image with the highest similarity is used as the target image generation model.

11. The apparatus according to claim 7, characterized in that, The processing module includes: The second generation unit is used to replace the third parameter of the second image generation model with the fourth parameter corresponding to the third parameter in the first image generation model according to the second preset rule, so as to obtain the fourth image generation model. The second determining unit is used to determine the target image generation model based on the fourth image generation model.

12. The apparatus according to claim 7, characterized in that, The acquisition module further includes: The second acquisition unit is used to acquire an initial face image and extract the pose angle of the initial face image; The third processing unit is used to use the initial face image and the pose angle of the initial face image as the face image to be processed.

13. An image processing apparatus, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-6.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images