Image processing method and device, equipment and storage medium

By acquiring and using target deformation parameters to weight the reference image, a result image that conforms to the deformation effect of the original image object is generated, which solves the problem of lack of adaptability of the beauty of human faces in the prior art and improves human-computer interaction efficiency.

CN119048338APending Publication Date: 2024-11-29BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411146967.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing facial beauty technology lacks adaptability, resulting in the facial images after beauty that do not meet user needs. Users need to manually adjust them multiple times, reducing human-computer interaction efficiency.

Method used

By obtaining the target deformation parameters in the original image, the deformation diagram of the reference image is weighted to generate a result image that conforms to the deformation effect of the original image object, simplifying user operations and improving human-computer interaction efficiency.

Benefits of technology

Without the need for multiple manual adjustments from users, the deformation effect of the image matches the original image object, simplifying the operation process and improving human-computer interaction efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048338A_ABST
    Figure CN119048338A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and device, equipment and a storage medium, and belongs to the technical field of computers. According to the method, the at least one deformation graph of the reference image is weighted based on the at least one target deformation parameter of the original image to obtain the deformation graph of the original image, and the deformation graph of the reference image can be obtained based on the target deformation parameter used for indicating the target deformation degree of the upper part of the object in the original image. The deformation of the upper part of the object in the reference image is adjusted, that is, the deformation to be generated on the upper part of the object in the original image can be adjusted, so that the deformation effect of the result image obtained based on the deformation image processing of the original image can be matched with the object in the original image; therefore, the deformation effect of the result image meets the user demand, and the user does not need to manually adjust the result image for many times, so that the operation of obtaining the result image by the user is simplified, and the man-machine interaction efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method, device, equipment and storage medium. Background Art

[0002] With the development of Internet technology, face beautification has become a necessary function on mobile terminals, which can make faces look more beautiful. At present, the process of beautifying a face image is: obtaining multiple key points in the face image, and then adjusting the positions of the key points according to a fixed deformation direction and deformation strength to obtain the beautified face image. However, the above process uses the same deformation direction and deformation strength for all face images to be beautified, which lacks adaptability, resulting in the face image after beautification not meeting user needs, and further resulting in the user needing to manually adjust the face image multiple times, which makes the operation of the user obtaining the beautified face image more cumbersome, resulting in low human-computer interaction efficiency. Therefore, how to simplify the beautification process of face images to improve the efficiency of human-computer interaction is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present disclosure provides an image processing method, apparatus, device and storage medium, which can make the deformation effect of the result image obtained by processing the deformation map of the original image match the object in the original image, so that the deformation effect of the result image meets the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction. The technical solution of the present disclosure is as follows:

[0004] According to one aspect of an embodiment of the present disclosure, there is provided an image processing method, including:

[0005] Processing the original image to obtain at least one target deformation parameter of the original image, wherein the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of the object in the original image;

[0006] Based on the at least one target deformation parameter, weighting at least one deformation map of the reference image to obtain a deformation map of the original image, wherein the deformation map of the reference image is used to represent the deformation of any part of the object in the reference image, and the deformation map of the original image is used to represent the deformation to be occurred in at least one part of the object in the original image;

[0007] Based on the deformation map of the original image, the original image is processed to obtain a result image of the original image.

[0008] According to another aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, including:

[0009] A first processing unit is configured to process the original image to obtain at least one target deformation parameter of the original image, wherein the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of the object in the original image;

[0010] a weighting unit, configured to weight at least one deformation map of the reference image based on the at least one target deformation parameter to obtain a deformation map of the original image, wherein the deformation map of the reference image is used to represent a deformation of any part of the object in the reference image, and the deformation map of the original image is used to represent a deformation to be occurred in at least one part of the object in the original image;

[0011] The second processing unit is configured to process the original image based on the deformation map of the original image to obtain a result image of the original image.

[0012] In some embodiments, the weighting unit is configured to determine the deformation amount and deformation direction of at least one part on the object in the reference image based on at least one deformation map of the reference image; weight the deformation amount of at least one part on the object in the reference image based on the at least one target deformation parameter to obtain the target deformation amount of at least one part on the object in the original image; and generate the deformation map of the original image based on the target deformation amount and target deformation direction of the at least one part, wherein the target deformation direction of the at least one part is the deformation direction of at least one part on the object in the reference image.

[0013] In some embodiments, the second processing unit includes:

[0014] a determination subunit configured to determine a target deformation amount and a target deformation direction of at least one part of the object in the original image based on the deformation map of the original image;

[0015] The processing subunit is configured to process the original image based on the target deformation amount and target deformation direction of the at least one part to obtain the result image.

[0016] In some embodiments, the processing subunit is configured to determine the target deformation amount of the at least one part in the horizontal direction and the target deformation direction of the at least one part; based on the target deformation amount of the at least one part in the horizontal direction and the target deformation direction of the at least one part, offset the pixel points located at the at least one part in the original image to obtain the result image.

[0017] In some embodiments, the first processing unit is configured to process the original image based on a parameter generation model to obtain at least one target deformation parameter of the original image, and the parameter generation model is used to generate deformation parameters matching the object in the image based on the image.

[0018] In some embodiments, the apparatus further comprises:

[0019] An acquisition unit, configured to acquire a sample original image and a sample reference image;

[0020] A third processing unit is configured to process the sample original image based on a parameter generation model in a generative adversarial network to obtain a prediction result image, wherein the generative adversarial network includes the parameter generation model and a discriminant model, and the discriminant model is used to discriminate whether the image is true or false;

[0021] The training unit is configured to keep the model parameters of the parameter generation model unchanged, and train the discriminant model based on the sample reference image and the prediction result image; keep the model parameters of the discriminant model unchanged, and train the parameter generation model based on the sample original image.

[0022] In some embodiments, the third processing unit is configured to process the sample original image based on the parameter generation model to obtain at least one target sample deformation parameter of the sample original image, and the at least one target sample deformation parameter is used to indicate the target deformation degree of at least one part on the object in the sample original image; based on the at least one target sample deformation parameter, weight at least one deformation map of the sample reference image to obtain the deformation map of the sample original image, and the deformation map of the sample reference image is used to represent the deformation occurred in any part of the object in the sample reference image, and the deformation map of the sample original image is used to represent the deformation to be occurred in at least one part of the object in the sample original image; based on the deformation map of the sample original image, the sample original image is processed to obtain the prediction result image.

[0023] In some embodiments, the apparatus further comprises:

[0024] a spatial transformation unit configured to, when at least one of an object posture and an object size of the reference image and the original image is inconsistent, perform spatial transformation on at least one deformation map of the reference image based on the original image, so that at least one part of the object in the reference image corresponding to the at least one deformation map after spatial transformation is aligned with at least one part of the object in the original image;

[0025] The weighting unit is configured to weight the at least one deformation map after spatial transformation based on the at least one target deformation parameter to obtain the deformation map of the original image.

[0026] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0027] one or more processors;

[0028] A memory for storing program codes executable by the processor;

[0029] The processor is configured to execute the program code to implement the above-mentioned image processing method.

[0030] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When a program code in the computer-readable storage medium is executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned image processing method.

[0031] According to another aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned image processing method is implemented.

[0032] The disclosed embodiment provides an image processing method, which obtains a deformation map of the original image by weighting at least one deformation map of a reference image based on at least one target deformation parameter of the original image. The method can adjust the deformation of the part of the object in the reference image based on the target deformation parameter used to indicate the target deformation degree of the part of the object in the original image, that is, can adjust the deformation to be occurred in the part of the object in the original image, so that the deformation effect of the result image obtained by processing based on the deformation map of the original image can match the object in the original image, thereby making the deformation effect of the result image meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0033] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0035] Figure 1 It is a schematic diagram of an implementation environment of an image processing method according to an exemplary embodiment.

[0036] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0037] Figure 3 is a flowchart of another image processing method according to an exemplary embodiment.

[0038] Figure 4 It is a schematic diagram showing an image processing flow according to an exemplary embodiment.

[0039] Figure 5 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0040] Figure 6 is a block diagram of another image processing device according to an exemplary embodiment.

[0041] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0044] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the original images and reference images involved in this disclosure are obtained with full authorization.

[0045] The image processing method provided by the embodiments of the present disclosure can be performed by an electronic device. In some embodiments, the electronic device can be provided as a terminal or a server. When the electronic device is provided as a terminal, the operations performed by the image processing method can be implemented by the terminal; when it is provided as a server, the operations performed by the image processing method can be implemented by the server, and the server can receive the original image authorized by the terminal, and the server processes the original image; the server and the terminal can also interact to implement the operations performed by the image processing method; the terminal can also send an image processing request to the server, and the server performs image processing, and then feeds back the result of the image processing to the terminal, and the terminal outputs the result of the image processing.

[0046] Figure 1 FIG. 1 is a schematic diagram of an implementation environment of an image processing method according to an exemplary embodiment. Figure 1 The implementation environment specifically includes: a terminal 101 and a server 102. The terminal 101 can be connected to the server 102 via a wireless network or a wired network.

[0047] Terminal 101 may be at least one of a smart phone, a smart watch, a desktop computer, a laptop, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, and a laptop computer. A target application is installed or running on terminal 101. The target application is any type of application that can provide image processing functions, such as video applications, social applications, etc. Users can log in to the target application through terminal 101 to obtain the services provided by the target application. Server 102 may be a background server of the target application, used to provide background services for the target application.

[0048] The terminal 101 may generally refer to one of a plurality of terminals. This embodiment is described by taking the terminal 101 as an example. Those skilled in the art may know that the number of the above terminals may be more or less. For example, the above terminals may be several, or the above terminals may be dozens or hundreds, or a greater number. The embodiment of the present disclosure does not limit the number and device type of the terminals.

[0049] The server 102 is at least one of a single server, multiple servers, a cloud computing platform, and a virtualization center. Optionally, the number of the above servers may be more or less, which is not limited in the embodiments of the present disclosure. Of course, the server 102 may also include other functional servers to provide more comprehensive and diversified services. In some embodiments, the server 102 undertakes the main computing work, and the terminal 101 undertakes the secondary computing work; or, the server 102 undertakes the secondary computing work, and the terminal 101 undertakes the main computing work; or, the server 102 and the terminal 101 use a distributed computing architecture for collaborative computing. The server 102 can be connected to the terminal 101 and other terminals via a wireless network or a wired network. Optionally, the number of the above servers may be more or less, which is not limited in the embodiments of the present disclosure.

[0050] Figure 2 is a flowchart of an image processing method according to an exemplary embodiment. Figure 2 As shown, the method is performed by an electronic device and includes the following steps:

[0051] In step 201, the electronic device processes an original image to obtain at least one target deformation parameter of the original image, where the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of an object in the original image.

[0052] In the embodiments of the present disclosure, the original image is an image including at least one part of an object, and the object may be a person or an object. Taking the object as a person as an example, the original image may be a face image or a body image. When the original image is a face image, the at least one part includes at least one part of the face of the object, such as eyes, nose, mouth, temple, cheekbone, jaw, etc. When the original image is a body image, the at least one part includes not only parts of the face but also parts of the body of the object, such as arms, legs, torso, etc. The electronic device processes the original image and can obtain at least one target deformation parameter based on the characteristics of at least one part of the object in the original image. Among them, the at least one target deformation parameter is used to indicate the target deformation degree of at least one part. The target deformation degree is the deformation degree to be achieved for the part of the object in the original image.

[0053] In step 202, the electronic device weights at least one deformation map of the reference image based on at least one target deformation parameter to obtain a deformation map of the original image, where the deformation map of the reference image is used to represent the deformation that occurs to any part of the object in the reference image, and the deformation map of the original image is used to represent the deformation to be occurred to at least one part of the object in the original image.

[0054] In the disclosed embodiment, the reference image is an image including at least one part of an object, and at least one part of the reference image corresponds one-to-one to at least one part of the original image. That is, when the original image to be processed is a face image, the reference image is also a face image. In the process of processing the reference image, the electronic device can determine the deformation of at least one part of the object in the reference image during the processing, and then obtain at least one deformation map of the reference image. The deformation map is used to represent the deformation of any part of the object in the reference image, such as the object's temple bulging outwards, the object's eyes becoming larger, the object's lower jaw shrinking, etc.

[0055] Since the features of at least one part of the object in the original image are different from the features of at least one part of the object in the reference image, the deformation of any part of the object in the reference image is not suitable for direct application to the part of the object in the original image. Accordingly, in order to determine the deformation suitable for application to at least one part of the object in the original image, the electronic device can weight at least one deformation map of the reference image based on at least one target deformation parameter of the original image to obtain the deformation map of the original image. By weighting at least one deformation map of the reference image based on at least one target deformation parameter, the deformation of the part of the object in the reference image can be adjusted so that the deformation of the part of the object in the adjusted reference image is suitable for application to the part of the object in the original image.

[0056] In step 203, the electronic device processes the original image based on the deformation map of the original image to obtain a result image of the original image.

[0057] In an embodiment of the present disclosure, the electronic device processes the original image based on the deformation map of the original image, and can apply the deformation to be occurred on at least one part of the object in the original image represented by the deformation map to the original image to obtain a result image of the original image.

[0058] The disclosed embodiment provides an image processing method, which obtains a deformation map of the original image by weighting at least one deformation map of a reference image based on at least one target deformation parameter of the original image. The method can adjust the deformation of the part of the object in the reference image based on the target deformation parameter used to indicate the target deformation degree of the part of the object in the original image, that is, can adjust the deformation to be occurred in the part of the object in the original image, so that the deformation effect of the result image obtained by processing based on the deformation map of the original image can match the object in the original image, thereby making the deformation effect of the result image meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0059] In some embodiments, based on at least one target deformation parameter, weighting at least one deformation map of the reference image to obtain the deformation map of the original image includes:

[0060] Determine, based on at least one deformation map of the reference image, a deformation amount and a deformation direction of at least one portion of the object in the reference image;

[0061] Based on at least one target deformation parameter, weighting the deformation amount of at least one part of the object in the reference image to obtain a target deformation amount of at least one part of the object in the original image;

[0062] A deformation map of the original image is generated based on a target deformation amount and a target deformation direction of at least one part, wherein the target deformation direction of at least one part is a deformation direction of at least one part on the object in the reference image.

[0063] In the disclosed embodiment, by weighting the deformation amount of at least one part on the object represented by at least one deformation map of the reference image based on at least one target deformation parameter, the deformation amount of the part on the object in the reference image can be adjusted so that the adjusted deformation amount is the deformation amount that matches the part on the object in the original image, thereby improving the deformation effect of the result image obtained by processing the original image based on the deformation amount, so that the result image can better meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0064] In some embodiments, the original image is processed based on the deformation map of the original image to obtain a result image of the original image, including:

[0065] Determine a target deformation amount and a target deformation direction of at least one part of the object in the original image based on the deformation map of the original image;

[0066] Based on the target deformation amount and the target deformation direction of at least one part, the original image is processed to obtain a result image.

[0067] In the disclosed embodiment, by determining the target deformation amount and target deformation direction corresponding to the deformation to be performed on at least one part of the object in the original image, the terminal can process each part in the original image separately based on the target deformation amount and target deformation direction of each part, so that the deformation effect of each part in the processed result image matches the object in the original image, thereby achieving refined processing of the original image and improving the deformation effect of the processed result image, so that the user no longer needs to fine-tune the result image, thereby improving the efficiency of human-computer interaction.

[0068] In some embodiments, based on the target deformation amount and the target deformation direction of at least one part, the original image is processed to obtain a result image, including:

[0069] Determine a target deformation amount in a horizontal direction and a target deformation amount in a vertical direction of at least one part based on a target deformation amount and a target deformation direction of at least one part;

[0070] Based on the target deformation amount of at least one part in the horizontal direction and the target deformation amount in the vertical direction, the pixel points located at at least one part in the original image are offset to obtain a result image.

[0071] In the embodiment of the present disclosure, by offsetting the pixel points based on the target deformation amounts of the pixel points in the horizontal and vertical directions to obtain a result image, it is possible to achieve refined processing of the original image and improve the deformation effect of the result image, so that the user no longer needs to fine-tune the result image, thereby simplifying the operation of the user obtaining the result image and improving the efficiency of human-computer interaction.

[0072] In some embodiments, processing the original image to obtain at least one target deformation parameter of the original image includes:

[0073] Based on the parameter generation model, the original image is processed to obtain at least one target deformation parameter of the original image. The parameter generation model is used to generate deformation parameters matching the object in the original image based on the original image.

[0074] In the embodiment of the present disclosure, by utilizing a parameter generation model to obtain at least one target deformation parameter of the original image, at least one target deformation parameter that matches the object in the original image can be quickly and accurately determined, thereby reducing the amount of calculation required by the terminal to obtain the target deformation parameter and improving the processing efficiency of the original image, thereby improving the efficiency with which the user obtains the result image of the original image and improving the user experience.

[0075] In some embodiments, the training process of the parameter generation model includes:

[0076] Obtain a sample original image and a sample reference image;

[0077] Based on the parameter generation model in the generative adversarial network, the original sample image is processed to obtain the predicted result image. The generative adversarial network includes a parameter generation model and a discriminant model. The discriminant model is used to distinguish the true or false of the image.

[0078] Keep the model parameters of the parameter generation model unchanged, and train the discriminant model based on the sample reference image and the prediction result image;

[0079] Keep the model parameters of the discriminant model unchanged, and train the parameters to generate the model based on the sample original image.

[0080] In some embodiments, based on the parameter generation model in the generative adversarial network, the sample original image is processed to obtain a prediction result image, including:

[0081] Based on the parameter generation model, the sample original image is processed to obtain at least one target sample deformation parameter of the sample original image, where the at least one target sample deformation parameter is used to indicate a target deformation degree of at least one part of the object in the sample original image;

[0082] Based on at least one target sample deformation parameter, for at least one deformation map of the sample reference image, a deformation map of the sample original image is determined, wherein the deformation map of the sample reference image is used to represent the deformation of any part of the object in the sample reference image, and the deformation map of the sample original image is used to represent the deformation to be occurred in at least one part of the object in the sample original image;

[0083] Based on the deformation map of the sample original image, the sample original image is processed to obtain a prediction result image.

[0084] In some embodiments, the method further comprises:

[0085] When at least one of an object posture and an object size of the reference image and the original image is inconsistent, spatially transforming at least one deformation map of the reference image based on the original image so that at least one part of the object in the reference image corresponding to the at least one deformation map after the spatial transformation is aligned with at least one part of the object in the original image;

[0086] Based on at least one target deformation parameter, weighting at least one deformation map of the reference image to obtain a deformation map of the original image includes:

[0087] Based on at least one target deformation parameter, at least one deformation map after spatial transformation is weighted to obtain a deformation map of the original image.

[0088] In the disclosed embodiment, by performing spatial transformation on at least one deformation map of a reference image, the posture and size of a part in the deformation map after the spatial transformation can be made consistent with the posture and size of the corresponding part in the original image, and then when the deformation map of the reference image after the spatial transformation is processed based on the target deformation parameter, the deformation effect of the deformation to be occurred represented by the deformation map of the processed original image can be improved, thereby improving the deformation effect of the result image of the original image, so that the result image can better meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0089] Above Figure 2FIG. 1 is a flow chart of an image processing method of the present disclosure. The image processing solution provided by the present disclosure is further described below. Figure 3 is a flowchart of another image processing method according to an exemplary embodiment, the method is executed by a terminal, see Figure 3 , the method comprising:

[0090] In step 301, the terminal obtains an original image.

[0091] In the embodiment of the present disclosure, when the terminal runs an application program that provides an image processing function, in response to a user uploading an image in the application program, the terminal can use the image uploaded by the user as the original image to be processed. Alternatively, in response to a user shooting operation in the application program, the terminal can use the shot image as the original image to be processed. The original image is an image that includes at least one part of an object.

[0092] In step 302, the terminal processes the original image based on the parameter generation model to obtain at least one target deformation parameter of the original image. The parameter generation model is used to generate deformation parameters that match the object in the image based on the image, and the at least one target deformation parameter is used to indicate the target deformation degree of at least one part on the object in the original image.

[0093] In the disclosed embodiment, the parameter generation model is used to generate deformation parameters matching the object in the image based on the image. Accordingly, the terminal inputs the acquired original image into the parameter generation model, and can obtain at least one target deformation parameter matching the object in the original image. For example, at least one part of the object in the original image is the left temple, right temple, left cheekbone, right cheekbone, left mandible and right mandible of the face. The parameter generation model can generate the corresponding target deformation parameters for each part based on the features of these six parts in the original image. The target deformation parameter corresponding to each part is used to indicate the degree of deformation to be achieved by the part, that is, the target deformation degree. By using the parameter generation model to obtain at least one target deformation parameter of the original image, at least one target deformation parameter matching the object in the original image can be quickly and accurately determined, which reduces the amount of calculation for the terminal to obtain the target deformation parameter, improves the processing efficiency of the original image, and thereby improves the efficiency of the user to obtain the result image of the original image, thereby improving the user experience.

[0094] It should be noted that at least one part corresponding to at least one target deformation parameter of the original image is related to the image processing function selected by the user in the application. For example, when the image processing function is the face-slimming function, the at least one target deformation parameter obtained by the terminal from processing the original image may be the target deformation parameters of the six parts: the left temple, the right temple, the left cheekbone, the right cheekbone, the left mandible, and the right mandible. When the image processing function is the three-chamber and five-eye proportion adjustment function, the at least one target deformation parameter obtained by the terminal from processing the original image may be the target deformation parameters of the five parts: the left eye, the right eye, the nose, the mouth, and the forehead.

[0095] In some embodiments, in order to ensure the accuracy of the parameter generation model, the terminal can train the parameter generation model, and the training process of the parameter generation model includes:

[0096] (1) The terminal obtains a sample original image and a sample reference image. Taking the sample original image and the sample reference image as a face image as an example, the terminal can obtain an image of an "ordinary face" as a sample original image and obtain an image of a "good-looking face" (such as a model or celebrity's face) as a sample reference image.

[0097] (2) The terminal processes the sample original image based on the parameter generation model in the generative adversarial network to obtain a prediction result image. The generative adversarial network includes a parameter generation model and a discriminant model. The discriminant model is used to discriminate whether the image is true or false. The terminal can process the sample original image based on the parameter generation model to obtain at least one target sample deformation parameter of the sample original image. The at least one target sample deformation parameter is used to indicate the target deformation degree of at least one part of the object in the sample original image. Then, the terminal weights at least one deformation map of the sample reference image based on the at least one target sample deformation parameter to obtain a deformation map of the sample original image. The deformation map of the sample reference image is used to indicate the deformation of any part of the object in the sample reference image. The deformation map of the sample original image is used to indicate the deformation to be occurred in at least one part of the object in the sample original image. Finally, the terminal can process the sample original image based on the deformation map of the sample original image to obtain a prediction result image.

[0098] (3) The terminal keeps the model parameters of the parameter generation model unchanged, trains the discriminant model based on the sample reference image and the prediction result image, keeps the model parameters of the discriminant model unchanged, and trains the parameter generation model based on the sample original image. In the process of alternately training the parameter generation model and the discriminant model, when training the parameter generation model, the terminal can input the prediction result image generated by the parameter generation model into the discriminant model, and the discriminant model can generate a first discrimination result based on the prediction result image and the sample reference image, and back-propagate the first discrimination result to the parameter generation model through the loss function, and optimize the model parameters of the parameter generation model. When training the discriminant model, the parameter generation model is the parameter generation model optimized under this alternation, and the optimized parameter generation model generates the prediction result image again based on the sample original image, and the discriminant model generates a second discrimination result based on the prediction result image and the sample reference image, and back-propagates the second discrimination result to the discriminant model through the loss function, and optimizes the model parameters of the discriminant model. After multiple optimizations, if the preset convergence condition is met, the alternating iteration ends, and a trained parameter generation model is obtained.

[0099] In step 303, when at least one of the object posture and object size of the reference image is inconsistent with that of the original image, the terminal performs spatial transformation on at least one deformation map of the reference image based on the original image, and the deformation map of the reference image is used to represent the deformation of any part of the object in the reference image.

[0100] In an embodiment of the present disclosure, after acquiring the original image, the terminal can also determine the object posture and object size of the original image. Among them, the object posture includes at least one of the facial posture and body posture of the object in the original image. Taking the original image as a facial image as an example, the object posture is the facial posture, which may include the deflection angle of the object's face in the original image. The deflection angle is used to indicate the degree of deflection of the object's face. The more upright the object's face is in the original image, the smaller the deflection angle of the object's face is. The object size is used to reflect the size of the object in the original image. For example, the object size of the original image is 560*360 (pixels).

[0101] Since the face of the object in the reference image is usually frontal (the face is facing the camera), the object size is usually a fixed value, and the object in the original image may have a tilted head or a side head, so that the face of the object in the original image is deflected, and the object size of the original image is related to the distance of the object from the camera. Therefore, when at least one of the object posture and object size of the reference image and the original image is inconsistent, the terminal can perform spatial transformation on at least one deformation map of the reference image based on the object posture and object size of the original image, so that at least one part of the object in the reference image corresponding to the at least one deformation map after spatial transformation is aligned with at least one part of the object in the original image, that is, the terminal can obtain at least one deformation map that matches the object posture and object size of the original image. Among them, spatial transformation includes operations such as scaling, rotating, and translating the image.

[0102] It should be noted that the process of the terminal acquiring at least one deformation map of the reference image is the same as the above step 202, which will not be described in detail here.

[0103] In step 304, the terminal weights at least one deformation map of the spatially transformed reference image based on at least one target deformation parameter to obtain a deformation map of the original image, where the deformation map of the original image is used to represent the deformation to be occurred in at least one part of the object in the original image.

[0104] In the disclosed embodiment, the deformation map of the reference image is used to represent the deformation of any part of the object in the reference image. Since the features of at least one part of the object in the original image are different from the features of at least one part of the object in the reference image, the deformation of any part of the object in the reference image is not suitable for direct application to the part of the object in the original image, that is, the terminal processes the original image based on at least one deformation map of the reference image, and the deformation effect of the result image obtained does not meet the user's needs to a large extent, so that the user still needs to manually adjust the result image, resulting in low human-computer interaction efficiency. In order to solve this problem, the terminal can weight at least one deformation map of the reference image after spatial transformation based on at least one target deformation parameter of the original image, so as to adjust the deformation of at least one part of the object represented by at least one deformation map of the reference image, so that the deformation of the part of the object in the adjusted reference image is suitable for application to the part of the object in the original image, that is, the deformation to be occurred represented by the weighted deformation map of the original image is suitable for application to the deformation of the part of the object in the original image. By performing spatial transformation on at least one deformation map of a reference image, the posture and size of the parts in the deformation map after spatial transformation can be made consistent with the posture and size of the corresponding parts in the original image. Furthermore, when the deformation map of the reference image after spatial transformation is processed based on the target deformation parameter, the deformation effect of the deformation to be occurred represented by the deformation map of the processed original image can be improved, thereby improving the deformation effect of the result image of the original image, so that the result image can better meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0105] For example, when the eyes of an object in the original image are very small, the value of the target deformation parameter corresponding to the object's eyes obtained by the terminal after processing the original image is larger, so that the terminal can increase the deformation of the object's eyes in the reference image based on the target deformation parameter, that is, increase the magnification of the eyes.

[0106] In some embodiments, the terminal can weight the deformation amount of any part of the object in the reference image represented by the deformation map of the reference image based on the target deformation parameter. Accordingly, the terminal weights at least one deformation map of the reference image based on at least one target deformation parameter to obtain the deformation map of the original image, including: the terminal can determine the deformation information of at least one part of the object in the reference image based on at least one deformation map of the reference image, and the deformation information is used to indicate the deformation of the part, including the deformation amount and deformation direction of the part. That is, the terminal can determine the deformation amount and deformation direction of at least one part of the object in the reference image; then, since the target deformation parameter is used to indicate the target deformation degree of any part of the object in the original image, and the deformation amount of the part is used to reflect the deformation degree of the part, the terminal can weight the deformation amount of at least one part of the object in the reference image based on at least one target deformation parameter to obtain the deformation amount to be occurred in at least one part of the object in the original image, that is, the target deformation amount; finally, based on the target deformation amount and target deformation direction of at least one part of the object in the original image, the terminal can determine the deformation to be occurred in at least one part of the object in the original image, that is, the deformation map of the original image can be obtained. Among them, the target deformation direction of at least one part is the deformation direction of at least one part of the object in the reference image. By weighting the deformation amount of at least one part on the object represented by at least one deformation map of the reference image based on at least one target deformation parameter, the deformation amount of the part on the object in the reference image can be adjusted so that the adjusted deformation amount matches the deformation amount of the part on the object in the original image, thereby improving the deformation effect of the result image obtained by processing the original image based on the deformation amount, making the result image more in line with user needs, without the need for the user to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0107] In some embodiments, the deformation map of the original image is a 3-channel image. Based on the target deformation amount and target deformation direction of at least one part of the object in the original image, the terminal can determine the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of each pixel in the original image. Then, the terminal can generate the deformation map of the original image by taking the target deformation amount in the horizontal direction of the pixel as the value of the first channel, the target deformation amount in the vertical direction of the pixel as the value of the second channel, and the preset value as the value of the third channel.

[0108] In step 305, the terminal determines a target deformation amount and a target deformation direction of at least one part of the object in the original image based on the deformation map of the original image.

[0109] In the disclosed embodiment, the target deformation amount is the deformation amount that at least one part of the object in the original image is to achieve when the deformation occurs. The target deformation direction is the deformation direction of at least one part of the object in the original image when the deformation occurs. Since the deformation map of the original image is used to represent the deformation to be occurred in at least one part of the object in the original image. Therefore, based on the deformation map of the original image, the terminal can determine the deformation amount and deformation direction of at least one part of the object in the original image when the deformation occurs.

[0110] In step 306, the terminal processes the original image based on the target deformation amount and target deformation direction of at least one part to obtain a result image.

[0111] In the disclosed embodiment, the terminal processes the original image based on the target deformation amount and target deformation direction of at least one part, so that at least one part on the object in the original image is deformed by the target deformation amount of at least one part according to the target deformation direction of at least one part, so as to obtain a result image of the original image. By determining the target deformation amount and target deformation direction corresponding to the deformation to be occurred in at least one part on the object in the original image, the terminal can process each part in the original image separately based on the target deformation amount and target deformation direction of each part, so that the deformation effect of each part in the result image obtained by processing matches the object in the original image, and the refined processing of the original image is realized, and the deformation effect of the result image obtained by processing is improved, so that the user does not need to fine-tune the result image, thereby improving the efficiency of human-computer interaction.

[0112] In some embodiments, in the process of processing the original image to obtain the result image, the terminal can offset the pixel points in the original image based on the target deformation amount and target deformation direction of at least one part. Accordingly, the terminal can determine the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of at least one part based on the target deformation amount and target deformation direction of at least one part; then, the terminal can offset the pixel points located in at least one part in the original image based on the target deformation amount and target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of at least one part to obtain the result image. Among them, the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of at least one part include the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of the pixel points located in at least one part in the original image. Accordingly, for any part on the object in the original image, the terminal determines the target deformation amount and the target deformation direction of multiple pixel points located in the part in the original image based on the target deformation amount and the target deformation direction of the part. For any pixel point, the terminal can obtain the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of the pixel point by decomposing the target deformation amount along the target deformation direction. Then, the terminal can shift the pixel based on the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of the pixel to obtain the position of the pixel in the result image. By shifting the pixel based on the target deformation amount in the horizontal direction and the vertical direction of the pixel to obtain the result image, the original image can be processed in a refined manner, the deformation effect of the result image can be improved, and the user does not need to fine-tune the result image, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0113] For example, taking the pixel coordinates in the original image as (x, y), the horizontal deformation of the pixel as offset(x, y)[0], and the vertical deformation as offset(x, y)[1], after the terminal offsets the pixel, the pixel coordinates in the result image are (x+offset(x, y)[0], y+offset(x, y)[1]).

[0114] In order to better illustrate the image processing method of the present disclosure, Figure 4As shown, a schematic diagram of an image processing flow is shown. The original image is a face image. The terminal can perform spatial transformation on multiple deformation maps (standard spatial deformation basis) of the reference image according to the face posture, face size, etc. of the original image to obtain multiple deformation maps after spatial transformation (input spatial deformation basis). The terminal can also input the original image into the parameter generation model to obtain the target deformation parameters of the original image. Finally, the terminal can linearly weight the target deformation parameters of the original image and the multiple deformation maps after spatial transformation to obtain the deformation map of the original image, and apply it to the original image to obtain the final result image.

[0115] The disclosed embodiment provides an image processing method, which obtains a deformation map of the original image by weighting at least one deformation map of a reference image based on at least one target deformation parameter of the original image. The method can adjust the deformation of the part of the object in the reference image based on the target deformation parameter used to indicate the target deformation degree of the part of the object in the original image, that is, can adjust the deformation to be occurred in the part of the object in the original image, so that the deformation effect of the result image obtained by processing based on the deformation map of the original image can match the object in the original image, thereby making the deformation effect of the result image meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0116] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0117] Figure 5 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 5 As shown, the device includes: a first processing unit 501, a weighting unit 502 and a second processing unit 503.

[0118] The first processing unit 501 is configured to process the original image to obtain at least one target deformation parameter of the original image, where the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of the object in the original image;

[0119] A weighting unit 502 is configured to weight at least one deformation map of the reference image based on at least one target deformation parameter to obtain a deformation map of the original image, wherein the deformation map of the reference image is used to represent the deformation of any part of the object in the reference image, and the deformation map of the original image is used to represent the deformation to be occurred in at least one part of the object in the original image;

[0120] The second processing unit 503 is configured to process the original image based on the deformation map of the original image to obtain a result image of the original image.

[0121] In some embodiments, the weighting unit 502 is configured to determine the deformation amount and deformation direction of at least one part on the object in the reference image based on at least one deformation map of the reference image; weight the deformation amount of at least one part on the object in the reference image based on at least one target deformation parameter to obtain the target deformation amount of at least one part on the object in the original image; and generate a deformation map of the original image based on the target deformation amount and target deformation direction of at least one part, wherein the target deformation direction of at least one part is the deformation direction of at least one part on the object in the reference image.

[0122] In some embodiments, Figure 6 FIG. 1 is a block diagram of another image processing device according to an exemplary embodiment. Figure 6 As shown, the second processing unit 503 includes:

[0123] A determination subunit 601 is configured to determine a target deformation amount and a target deformation direction of at least one part of the object in the original image based on the deformation map of the original image;

[0124] The processing subunit 602 is configured to process the original image based on the target deformation amount and target deformation direction of at least one part to obtain a result image.

[0125] In some embodiments, the processing subunit 602 is configured to determine a target deformation amount in the horizontal direction and a target deformation amount in the vertical direction of at least one part based on the target deformation amount and target deformation direction of at least one part; based on the target deformation amount in the horizontal direction and the target deformation amount in the vertical direction of at least one part, offset the pixel points located at at least one part in the original image to obtain a result image.

[0126] In some embodiments, the first processing unit 501 is configured to process the original image based on a parameter generation model to obtain at least one target deformation parameter of the original image, and the parameter generation model is used to generate deformation parameters matching the object in the original image based on the original image.

[0127] In some embodiments, see Figure 6 , the device further comprises:

[0128] An acquisition unit 504 is configured to acquire a sample original image and a sample reference image;

[0129] The third processing unit 505 is configured to process the sample original image based on the parameter generation model in the generative adversarial network to obtain a prediction result image, where the generative adversarial network includes a parameter generation model and a discriminant model, and the discriminant model is used to discriminate the authenticity of the image;

[0130] The training unit 506 is configured to keep the model parameters of the parameter generation model unchanged, and train the discriminant model based on the sample reference image and the prediction result image; keep the model parameters of the discriminant model unchanged, and train the parameter generation model based on the sample original image.

[0131] In some embodiments, the third processing unit 505 is configured to process the sample original image based on the parameter generation model to obtain at least one target sample deformation parameter of the sample original image, and the at least one target sample deformation parameter is used to indicate the target deformation degree of at least one part on the object in the sample original image; based on the at least one target sample deformation parameter, determine the deformation map of the sample original image for at least one deformation map of the sample reference image, and the deformation map of the sample reference image is used to represent the deformation occurred in any part of the object in the sample reference image, and the deformation map of the sample original image is used to represent the deformation to be occurred in at least one part of the object in the sample original image; based on the deformation map of the sample original image, process the sample original image to obtain a prediction result image.

[0132] In some embodiments, see Figure 6 , the device further comprises:

[0133] A spatial transformation unit 507 is configured to, when at least one of an object posture and an object size of the reference image and the original image is inconsistent, spatially transform at least one deformation map of the reference image based on the original image, so that at least one part of the object in the reference image corresponding to the at least one deformation map after the spatial transformation is aligned with at least one part of the object in the original image;

[0134] The weighting unit 502 is configured to weight at least one deformation map after the spatial transformation based on at least one target deformation parameter to obtain a deformation map of the original image.

[0135] The disclosed embodiment provides an image processing device, which obtains a deformation map of the original image by weighting at least one deformation map of a reference image based on at least one target deformation parameter of the original image. The device can adjust the deformation of the part of the object in the reference image based on the target deformation parameter used to indicate the target deformation degree of the part of the object in the original image, that is, it can adjust the deformation to be occurred in the part of the object in the original image, so that the deformation effect of the result image obtained by processing based on the deformation map of the original image can match the object in the original image, thereby making the deformation effect of the result image meet the user's needs, without the user having to manually adjust the result image multiple times, thereby simplifying the user's operation of obtaining the result image and improving the efficiency of human-computer interaction.

[0136] It should be noted that the image processing device provided in the above embodiment is only illustrated by the division of the above functional units. In actual applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the electronic device is divided into different functional units to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiment and the image processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0137] Regarding the image processing device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0138] Figure 7 700 is a block diagram of an electronic device according to an exemplary embodiment. Generally, the electronic device 700 includes: a processor 701 and a memory 702 .

[0139] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0140] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one program code, which is used to be executed by the processor 701 to implement the image processing method provided by the method embodiment of the present disclosure.

[0141] In some embodiments, the electronic device 700 may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707 and a power supply 708.

[0142] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0143] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 704 can communicate with other electronic devices through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in the present disclosure.

[0144] The display screen 705 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to collect touch signals on the surface or above the surface of the display screen 705. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 705 can be one, and the front panel of the electronic device 700 is set; in other embodiments, the display screen 705 can be at least two, which are respectively set on different surfaces of the electronic device 700 or are folded; in some other embodiments, the display screen 705 can be a flexible display screen, which is set on the curved surface or folded surface of the electronic device 700. Even, the display screen 705 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 705 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0145] The camera assembly 706 is used to capture images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the electronic device, and the rear camera is arranged on the back of the electronic device. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize the panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0146] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing, or input them into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the electronic device 700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0147] The power supply 708 is used to power various components in the electronic device 700. The power supply 708 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 708 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0148] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the electronic device 700, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0149] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 702 including instructions, and the instructions can be executed by a processor 701 of an electronic device 700 to complete the above-mentioned image processing method. Alternatively, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0150] A computer program product comprises a computer program, and when the computer program is executed by a processor, the image processing method is implemented.

[0151] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0152] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, It is characterized in that The method comprises: Processing the original image to obtain at least one target deformation parameter of the original image, wherein the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of the object in the original image; Based on the at least one target deformation parameter, weighting at least one deformation map of the reference image to obtain a deformation map of the original image, wherein the deformation map of the reference image is used to represent the deformation of any part of the object in the reference image, and the deformation map of the original image is used to represent the deformation to be occurred in at least one part of the object in the original image; Based on the deformation map of the original image, the original image is processed to obtain a result image of the original image.

2. The image processing method according to claim 1, It is characterized in that The step of weighting at least one deformation map of the reference image based on the at least one target deformation parameter to obtain the deformation map of the original image comprises: Determine, based on at least one deformation map of the reference image, a deformation amount and a deformation direction of at least one part of the object in the reference image; Based on the at least one target deformation parameter, weighting the deformation amount of at least one part of the object in the reference image to obtain a target deformation amount of at least one part of the object in the original image; A deformation map of the original image is generated based on the target deformation amount and target deformation direction of the at least one part, wherein the target deformation direction of the at least one part is the deformation direction of the at least one part on the object in the reference image.

3. The image processing method according to claim 1, It is characterized in that The step of processing the original image based on the deformation map of the original image to obtain a result image of the original image includes: Determining a target deformation amount and a target deformation direction of at least one part of an object in the original image based on the deformation map of the original image; The original image is processed based on the target deformation amount and the target deformation direction of the at least one part to obtain the result image.

4. The image processing method according to claim 3, It is characterized in that The processing of the original image based on the target deformation amount and the target deformation direction of the at least one part to obtain the result image includes: Determining a target deformation amount in a horizontal direction and a target deformation amount in a vertical direction of the at least one part based on the target deformation amount and the target deformation direction of the at least one part; Based on the target deformation amount of the at least one part in the horizontal direction and the target deformation amount in the vertical direction, the pixel points located at the at least one part in the original image are offset to obtain the result image.

5. The image processing method according to claim 1, It is characterized in that The processing of the original image to obtain at least one target deformation parameter of the original image includes: The original image is processed based on a parameter generation model to obtain at least one target deformation parameter of the original image, wherein the parameter generation model is used to generate deformation parameters matching the object in the image based on the image.

6. The image processing method according to claim 5, It is characterized in that The training process of the parameter generation model includes: Obtain a sample original image and a sample reference image; Based on the parameter generation model in the generative adversarial network, the sample original image is processed to obtain a prediction result image, wherein the generative adversarial network includes the parameter generation model and the discriminant model, and the discriminant model is used to discriminate the authenticity of the image; Keeping the model parameters of the parameter generation model unchanged, training the discriminant model based on the sample reference image and the prediction result image; The model parameters of the discriminant model are kept unchanged, and the parameter generation model is trained based on the sample original image.

7. The image processing method according to claim 6, It is characterized in that The parameter generation model in the generative adversarial network is used to process the sample original image to obtain a prediction result image, including: Based on the parameter generation model, the sample original image is processed to obtain at least one target sample deformation parameter of the sample original image, wherein the at least one target sample deformation parameter is used to indicate a target deformation degree of at least one part of an object in the sample original image; Based on the at least one target sample deformation parameter, weighting at least one deformation map of the sample reference image to obtain a deformation map of the sample original image, wherein the deformation map of the sample reference image is used to represent the deformation of any part of the object in the sample reference image, and the deformation map of the sample original image is used to represent the deformation to be occurred in at least one part of the object in the sample original image; Based on the deformation map of the sample original image, the sample original image is processed to obtain the prediction result image.

8. The image processing method according to claim 1, It is characterized in that The method further comprises: In a case where at least one of an object posture and an object size of the reference image and the original image is inconsistent, spatially transforming at least one deformation map of the reference image based on the original image so that at least one part of the object in the reference image corresponding to the at least one deformation map after the spatial transformation is aligned with at least one part of the object in the original image; The step of weighting at least one deformation map of the reference image based on the at least one target deformation parameter to obtain the deformation map of the original image comprises: Based on the at least one target deformation parameter, the at least one deformation map after the spatial transformation is weighted to obtain a deformation map of the original image.

9. An image processing device, It is characterized in that The device comprises: A first processing unit is configured to process the original image to obtain at least one target deformation parameter of the original image, wherein the at least one target deformation parameter is used to indicate a target deformation degree of at least one part of the object in the original image; a weighting unit, configured to weight at least one deformation map of the reference image based on the at least one target deformation parameter to obtain a deformation map of the original image, wherein the deformation map of the reference image is used to represent a deformation of any part of the object in the reference image, and the deformation map of the original image is used to represent a deformation to be occurred in at least one part of the object in the original image; The second processing unit is configured to process the original image based on the deformation map of the original image to obtain a result image of the original image.

10. An electronic device, It is characterized in that The electronic device comprises: one or more processors; a memory for storing program code executable by the processor; The processor is configured to execute the program code to implement the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, It is characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the image processing method as described in any one of claims 1 to 8.

12. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 8 is implemented.