Image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed image which comprises a target object original image; performing three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, the attitude of the first three-dimensional model being consistent with the attitude of the target object in the original image of the target object; based on the target three-dimensional stylized model, carrying out stylization processing on the first three-dimensional model to obtain a second three-dimensional model of the target object; performing projection processing on the second three-dimensional model to obtain a stylized image of the target object; and fusing the stylized image with the to-be-processed image to obtain a processed image. According to the invention, stylization processing can be carried out on images of other parts except the face.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In recent years, with the advancement of research methods and the collection of relevant data sets, facial stylization has become a popular research field and application direction. Facial stylization is the process of converting facial avatars into specific styles, such as common sketch portrait styles, cartoon image (animation) styles, oil painting styles, etc. However, the existing technology lacks stylization processing technology for other parts besides the face, which makes it difficult to meet user needs. Summary of the invention

[0003] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides an image processing method, device, electronic device and storage medium.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] Acquire an image to be processed, wherein the image to be processed includes an original image of a target object;

[0006] Performing three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object;

[0007] Based on the target three-dimensional stylized model, stylizing the first three-dimensional model to obtain a second three-dimensional model of the target object;

[0008] Projecting the second three-dimensional model to obtain a stylized image of the target object;

[0009] The stylized image is fused with the image to be processed to obtain a processed image.

[0010] In a second aspect, the present disclosure further provides an image processing device, comprising:

[0011] An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes an original image of a target object;

[0012] A reconstruction module, configured to perform three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object;

[0013] A stylization module, configured to perform stylization processing on the first three-dimensional model based on a target three-dimensional stylized model to obtain a second three-dimensional model of the target object;

[0014] A projection module, used for projecting the second three-dimensional model to obtain a stylized image of the target object;

[0015] A fusion module is used to fuse the stylized image with the image to be processed to obtain a processed image.

[0016] In a third aspect, the present disclosure further provides an electronic device, the electronic device comprising:

[0017] one or more processors;

[0018] A storage device for storing one or more programs;

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described above.

[0020] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the image processing method as described above when executed by a processor.

[0021] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0022] The technical solution provided in the embodiment of the present application is to perform three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, and the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object; based on the target three-dimensional stylized model, the first three-dimensional model is stylized to obtain a second three-dimensional model of the target object; the second three-dimensional model is projected to obtain a stylized image of the target object; the stylized image is fused with the image to be processed to obtain a processed image. In essence, the original image of the target object is stylized with the help of the target three-dimensional stylized model, which can fill the gap of the current lack of stylized processing technology for other parts except the face. And compared with the solution of stylizing the original image of the target object with the help of a stylized image, since the three-dimensional stylized model carries more features, it can make the obtained processed image more natural and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0025] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0026] Figure 2 is a structural schematic diagram of an image processing device in an embodiment of the present disclosure;

[0027] Figure 3 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] Figure 1 This is a flowchart of an image processing method provided in an embodiment of the present disclosure. This embodiment is applicable to the case where image stylization processing is performed in a client. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a terminal, specifically including but not limited to a smart phone, a PDA, a tablet computer, a wearable device with a display screen, a desktop computer, a laptop computer, an all-in-one machine, a smart home device, etc. Alternatively, this embodiment is applicable to the case where image stylization processing is performed in a server. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server.

[0031] like Figure 1 As shown, the method may specifically include:

[0032] S110, obtaining an image to be processed, where the image to be processed includes an original image of a target object.

[0033] The original image may be, for example, an image that has not been subjected to stylization processing and is two-dimensional.

[0034] The target object may be, for example, something that is desired to be stylized. In one embodiment, the target object may be a body part that can make large movements such as rotation and bending. For example, the target object is other parts other than the face, such as the neck, limbs, etc.

[0035] The image to be processed may be, for example, an original image that is desired to be stylized. Optionally, the image to be processed includes an original image of a target object and an original image of a candidate object. The candidate object may be, for example, other things in the image to be processed except the target object. For example, if an image to be processed is an image of a cat standing in front of a table. If the cat's neck is taken as the target object, the candidate object is other things in the image to be processed except the cat's neck. Such as the cat's head, limbs, torso and table.

[0036] The image to be processed may specifically be an image captured by an image acquisition module in the terminal or an image downloaded from the network.

[0037] S120, performing three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object; the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object.

[0038] Since the target object can make large movements such as rotation and bending, if the movement of the target object in the original image of the target object is not considered and stylization is directly performed, the details of the target object in the obtained image will be inaccurate and unnatural.

[0039] The essence of this step is to reconstruct a first three-dimensional model of the target object based on the original image of the target object. The reconstructed first three-dimensional model includes the posture information of the target object. Compared with the original image of the target object, the first three-dimensional model has one more dimension, which carries richer personalized features and can describe the target object in more detail and accuracy.

[0040] Since the target object (such as the neck, limbs, etc.) can perform various actions, the posture of the target object can reflect the specific action performed by the current target object.

[0041] The posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object, which means that the action performed by the first three-dimensional model is the same as the action performed by the target object in the original image of the target object.

[0042] There are many methods for implementing this step, and this application does not limit this. Exemplarily, the implementation method of this step includes: determining the pose data of the target object based on the original image of the target object; obtaining a sample three-dimensional model of the target object; using the pose data of the target object, adjusting the sample three-dimensional model to obtain a first three-dimensional model of the target object. The pose data of the target object is data used to describe the posture presented by the target object in the original image of the target object. Optionally, the pose data of the target object may include an MVP (Model-View-Projection) matrix. Among them, the MVP matrix can convert a number of 3D model data into a view volume space through three-step transformations of the model matrix, the view matrix, and the projection matrix.

[0043] The sample three-dimensional model of the target object can be, for example, a pre-built three-dimensional model for simulating the target object. The sample three-dimensional model is a universal model. Its posture is fixed. That is, if there are multiple images to be processed, and the target objects in different images to be processed are the same but have different postures, the same sample three-dimensional model can be used to combine the position and posture data of the target objects in each image to be processed to obtain the first three-dimensional model corresponding to the target objects in each image to be processed.

[0044] S130: Based on the target three-dimensional stylized model, stylize the first three-dimensional model to obtain a second three-dimensional model of the target object.

[0045] The target 3D stylized model is a pre-designed 3D model with a specific artistic style. The target 3D stylized model includes style feature information and shape feature information of the target style.

[0046] The specific art style may be a sketch portrait style, a cartoon image (animation) style, a pencil drawing style, or an oil painting style, etc. The style of the target three-dimensional stylized model is the style that is desired to be added to the target object in the image to be processed, that is, the style that is desired to be possessed by the target object in the processed image obtained by using the image processing solution.

[0047] Optionally, the pose of the target three-dimensional stylized model is consistent with the pose of the first three-dimensional model. This setting can reduce the difficulty of executing this step.

[0048] If the posture of the target three-dimensional stylized model is consistent with the posture of the first three-dimensional model, optionally, before this step, the method also includes: obtaining an original target three-dimensional stylized model; and adjusting the original target three-dimensional stylized model using the posture data of the target object to obtain the target three-dimensional stylized model.

[0049] The original target 3D stylized model is a universal target 3D stylized model, and its posture is fixed. The posture of the original target 3D stylized model may be consistent with or inconsistent with the posture of the first 3D model.

[0050] Exemplarily, if in an image to be processed, the torso of the animal is facing forward, the head is twisted, facing its right rear, and the whole body turns its head. If the target object is the neck, since the neck is the key to turning the head, it will be in a rotated state. Assume that the posture of the original target three-dimensional stylized model is in a non-rotated state. When executing this step, it is necessary to use the posture data parsed from the original image of the target object; adjust the posture of the original target three-dimensional stylized model, and then obtain the target three-dimensional stylized model. The posture of the target three-dimensional stylized model is in a rotated state, and the rotation direction and rotation angle are consistent with the rotation direction and rotation angle of the target object in the original image of the target object. In this way, the purpose of making the posture of the target three-dimensional stylized model consistent with the posture of the first three-dimensional model can be achieved. And, in this way, when designing the original target three-dimensional stylized model, there is no need to consider the subsequent actual usage scenarios, which can reduce the difficulty of designing the original target three-dimensional stylized model.

[0051] The purpose of this step is to superimpose the target 3D stylized model with the first 3D model, so that the target style features are added to the first 3D model. The first 3D model with the target style features added is the second 3D model.

[0052] There are many methods for implementing this step, and this application does not limit this. Exemplarily, the implementation method of this step includes: based on the stylization processing submodule of the target three-dimensional stylized model, stylizing the first three-dimensional model; based on the deformation processing submodule of the target three-dimensional stylized model, deforming the first three-dimensional model after stylization.

[0053] The first three-dimensional model is stylized so as to superimpose the style features of the target three-dimensional stylized model on the stylized first three-dimensional model. Specifically, the method may include using a rendering engine to create a material on the first three-dimensional model. For example, a hair system is used to create hair on the first three-dimensional model to achieve the purpose of stylizing the first three-dimensional model, so that the obtained second three-dimensional model presents the material effect of, for example, an animal neck.

[0054] The first three-dimensional model after stylization is deformed, so as to superimpose the shape features of the target three-dimensional stylized model on the first three-dimensional model after stylization. In one embodiment, optionally, the target three-dimensional stylized model includes a plurality of first nodes, and the first three-dimensional model includes a plurality of second nodes; the first nodes and the second nodes correspond one to one; the first three-dimensional model after stylization is deformed, including: adjusting the positions of at least some of the second nodes in the first three-dimensional model after stylization according to the corresponding relationship between the first nodes and the second nodes and the positions of the first nodes.

[0055] Optionally, if the 3D model is a 3D mesh model, the common vertices of adjacent meshes can be used as nodes. In addition, if the 3D model is regarded as a collection of spatial points, the nodes can also be pre-specified spatial points in the spatial points constituting the 3D module. The nodes can be points on the surface of the 3D model or points inside the 3D model.

[0056] In some scenarios, optionally, the target three-dimensional stylized model is constructed based on the sample three-dimensional model, so that the correspondence between the first node in the target three-dimensional stylized model and the second node in the first three-dimensional model can be quickly determined without having to match the first node in the target three-dimensional stylized model with the second node in the first three-dimensional model one by one.

[0057] Exemplarily, if the first node a1 in the target three-dimensional stylized model has a corresponding relationship with the second node a2 in the first three-dimensional model, the coordinates of the second node in the world coordinate system can be adjusted based on the coordinates of the first node a1 in the world coordinate system, so that the adjusted first three-dimensional model carries the shape features of the target three-dimensional stylized model.

[0058] In one embodiment, the positions of at least some of the second nodes in the first three-dimensional model are adjusted based on the correspondence between the first node and the second node, and the positions of each first node. Specifically, it may include: for any second node, based on the position of the first node corresponding to the second node, determining the target position coordinates of each second node in the first three-dimensional model; based on the target position coordinates of each second node, adjusting the position of each second node in the first three-dimensional model so that after adjustment, each second node is located at the target position coordinate corresponding to it.

[0059] Optionally, after the first three-dimensional model after stylization is deformed, the method may further include: extracting illumination information from the image to be processed; and performing lighting processing on the first three-dimensional model after deformation according to the illumination information. In this way, the fusion effect of the stylized image of the target object in the subsequently formed processed image and the image to be processed can be more reasonable and realistic.

[0060] S140: Project the second three-dimensional model to obtain a stylized image of the target object.

[0061] If at the time of capturing the image to be processed, the relative position relationship between the image capture device (the image capture device can be a real image capture device or a virtual image capture device) and the target object is the first relative position relationship. The essence of this step is to use the virtual image capture device to capture the image of the second three-dimensional model, and at the time of image capture, the relative position relationship between the virtual image capture device and the second three-dimensional model is still the first relative position relationship. The acquisition result of the virtual image capture device capturing the image of the second three-dimensional model is a stylized image of the target object. The stylized image is two-dimensional.

[0062] S150: Fusing the stylized image with the image to be processed to obtain a processed image.

[0063] The essence of this step is to paste the stylized image back to the image to be processed to obtain a processed image. Optionally, the stylized image can be pasted back to the image to be processed by using Laplace fusion.

[0064] Furthermore, in order to ensure that the processed image is more realistic, optionally, S150 includes, based on the original image of the target object, performing color migration and deformation processing on the stylized image to obtain a second target object image; and fusing the second target object image with the image to be processed to obtain a processed image.

[0065] The purpose of color migration of the stylized image is to make the color of the stylized image after migration consistent or close to the color of the original image of the target object.

[0066] Performing deformation processing on the stylized image may exemplarily include: adjusting the positions of the contour points of the target object in the stylized image so that the thickness of the target object is consistent with the thickness of the target object presented in the original image of the target object; and / or, based on the original image of the target object, truncating the connection between the target object and other parts in the stylized image so that the length of the target object is consistent with the length of the target object presented in the original image of the target object.

[0067] On the basis of the above technical solution, optionally, S150 also includes: determining an occlusion relationship between the candidate object and the target object in the image to be processed; and based on the occlusion relationship, fusing the second target object image with the image to be processed to obtain a processed image.

[0068] As mentioned above, the candidate objects may be, for example, other things in the image to be processed except the target object. For example, if the image to be processed is an image of an animal showing a specific action, if the target object is the neck, the candidate objects may be the clothes worn by the animal, or other parts of the animal (such as the chin, limbs, etc.).

[0069] By setting the occlusion relationship as the basis, the second target object image is fused with the image to be processed to obtain a processed image. In essence, when performing image fusion, the positional relationship of different objects in space is considered, which can make the obtained processed image more reasonable.

[0070] Based on the above technical solution, optionally, after S150, the method further includes: blurring the boundary area between the second target object image and the candidate object, so as to make the second target object image and the candidate object transition naturally and make the processed image more realistic.

[0071] The technical solution provided by the embodiment of the present application is to perform three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, and the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object; based on the target three-dimensional stylized model, the first three-dimensional model is stylized to obtain a second three-dimensional model of the target object; the second three-dimensional model is projected to obtain a stylized image of the target object; the stylized image is fused with the image to be processed to obtain a processed image. In essence, the original image of the target object is stylized with the help of a target three-dimensional stylized model that is consistent with the posture of the first three-dimensional model, which can fill the gap of the current lack of stylized processing technology for other parts except the face. And compared with the solution of stylizing the original image of the target object with the help of a stylized image, since the three-dimensional stylized model carries more features, it can make the obtained processed image more natural and realistic.

[0072] In addition, since the first three-dimensional model includes the target object posture information, it can ensure that the processed image obtained accurately, naturally and realistically depicts the details of the target object.

[0073] On the basis of the above technical solutions, optionally, the images to be processed and the stylized images mentioned above can be used as training data to train the neural network model so that the neural network model has stylized processing functions. The trained neural network model can be subsequently applied to the field of video processing or live broadcasting. Specifically, the image of each frame in the video can be used as the image to be processed, and the target stylized features can be added to it one by one using the trained neural network model.

[0074] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0075] Figure 2 Schematic diagram of the structure of an image processing device in an embodiment of the present disclosure. The image processing device provided in the embodiment of the present disclosure can be configured in a client or in a server. Figure 2 , the image processing device specifically comprises:

[0076] An acquisition module 310 is used to acquire an image to be processed, wherein the image to be processed includes an original image of a target object;

[0077] A reconstruction module 320 is used to perform three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object;

[0078] A stylization module 330, configured to perform stylization processing on the first three-dimensional model based on a target three-dimensional stylized model to obtain a second three-dimensional model of the target object;

[0079] A projection module 340 is used to project the second three-dimensional model to obtain a stylized image of the target object;

[0080] The fusion module 350 is used to fuse the stylized image with the image to be processed to obtain a processed image.

[0081] Furthermore, the reconstruction module is used to:

[0082] Based on the original image of the target object, determining the position and posture data of the target object;

[0083] Obtaining a sample three-dimensional model of the target object;

[0084] The sample three-dimensional model is adjusted using the position and posture data of the target object to obtain a first three-dimensional model of the target object.

[0085] Furthermore, the stylization module is used to:

[0086] Based on the stylization processing submodule of the target three-dimensional stylized model, stylization processing is performed on the first three-dimensional model;

[0087] Based on the deformation processing submodule of the target three-dimensional stylized model, deformation processing is performed on the first three-dimensional model after the stylized processing.

[0088] Furthermore, the target three-dimensional stylized model includes a plurality of first nodes, the first three-dimensional model includes a plurality of second nodes; the first nodes and the second nodes correspond one to one; and the stylization module is used to:

[0089] According to the correspondence between the first nodes and the second nodes and the positions of the first nodes, the positions of at least some of the second nodes in the stylized first three-dimensional model are adjusted.

[0090] Furthermore, the fusion module is used to:

[0091] Based on the original image of the target object, performing color migration and deformation processing on the stylized image to obtain a second target object image;

[0092] The second target object image is fused with the image to be processed to obtain a processed image.

[0093] Furthermore, the fusion module is used to:

[0094] Determining an occlusion relationship between a candidate object and the target object in the image to be processed;

[0095] Based on the occlusion relationship, the second target object image is fused with the image to be processed to obtain a processed image.

[0096] Furthermore, the fusion module is further used to: fuse the second target object image with the image to be processed to obtain a processed image, and then perform blur processing on the boundary area between the second target object image and the candidate object.

[0097] The image processing device provided in the embodiment of the present disclosure can execute the steps executed by the client or server in the image processing method provided in the embodiment of the method of the present disclosure, and has the execution steps and beneficial effects, which will not be repeated here.

[0098] Figure 3 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 3, which shows a schematic diagram of the structure of an electronic device 1000 suitable for implementing the embodiment of the present disclosure. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0099] like Figure 3 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 to a random access memory (RAM) 1003 to implement the image processing method of the embodiment described in the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0100] Typically, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or by wire to exchange information. Although Figure 3 The electronic device 1000 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0101] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart, thereby implementing the image processing method as described above. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 1009, or installed from a storage device 1008, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0102] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated information signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0103] In some embodiments, the client and the server may communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any known or future developed network.

[0104] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0105] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0106] Acquire an image to be processed, wherein the image to be processed includes an original image of a target object;

[0107] Performing three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object;

[0108] Based on the target three-dimensional stylized model, stylizing the first three-dimensional model to obtain a second three-dimensional model of the target object;

[0109] Projecting the second three-dimensional model to obtain a stylized image of the target object;

[0110] The stylized image is fused with the image to be processed to obtain a processed image.

[0111] Optionally, when the above one or more programs are executed by the electronic device, the electronic device may also execute other steps described in the above embodiments.

[0112] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0113] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0114] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0115] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0116] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0117] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

[0118] one or more processors;

[0119] A memory for storing one or more programs;

[0120] When the one or more programs are executed by the one or more processors, the one or more processors implement any image processing method provided in the present disclosure.

[0121] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the image processing method as described in any one of the present disclosure is implemented.

[0122] The embodiment of the present disclosure further provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the image processing method as described above is implemented.

[0123] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0124] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes an original image of a target object; Performing three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object; Based on the target three-dimensional stylized model, stylizing the first three-dimensional model to obtain a second three-dimensional model of the target object; Projecting the second three-dimensional model to obtain a stylized image of the target object; The stylized image is fused with the image to be processed to obtain a processed image.

2. The method according to claim 1, characterized in that The three-dimensional reconstruction of the original image of the target object to obtain a first three-dimensional model of the target object includes: Based on the original image of the target object, determining the position and posture data of the target object; Obtaining a sample three-dimensional model of the target object; The sample three-dimensional model is adjusted using the position and posture data of the target object to obtain a first three-dimensional model of the target object.

3. The method according to claim 2, characterized in that The performing stylization processing on the first three-dimensional model based on the target three-dimensional stylized model includes: Based on the stylization processing submodule of the target three-dimensional stylized model, stylization processing is performed on the first three-dimensional model; Based on the deformation processing submodule of the target three-dimensional stylized model, deformation processing is performed on the first three-dimensional model after the stylized processing.

4. The method according to claim 3, characterized in that The target three-dimensional stylized model includes a plurality of first nodes, the first three-dimensional model includes a plurality of second nodes; the first nodes correspond to the second nodes one by one; and the deformation processing of the stylized first three-dimensional model includes: According to the correspondence between the first nodes and the second nodes and the positions of the first nodes, the positions of at least some of the second nodes in the stylized first three-dimensional model are adjusted.

5. The method according to claim 1, characterized in that The step of fusing the stylized image with the image to be processed to obtain a processed image includes: Based on the original image of the target object, performing color migration and deformation processing on the stylized image to obtain a second target object image; The second target object image is fused with the image to be processed to obtain a processed image.

6. The method according to claim 5, characterized in that The fusing the second target object image with the image to be processed to obtain a processed image further includes: Determining an occlusion relationship between a candidate object and the target object in the image to be processed; Based on the occlusion relationship, the second target object image is fused with the image to be processed to obtain a processed image.

7. The method according to claim 6, characterized in that After fusing the second target object image with the image to be processed to obtain a processed image, the method further includes: A blur process is performed on the boundary area between the second target object image and the candidate object.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring an image to be processed, wherein the image to be processed includes an original image of a target object; A reconstruction module, configured to perform three-dimensional reconstruction on the original image of the target object to obtain a first three-dimensional model of the target object, wherein the posture of the first three-dimensional model is consistent with the posture of the target object in the original image of the target object; A stylization module, configured to perform stylization processing on the first three-dimensional model based on a target three-dimensional stylized model to obtain a second three-dimensional model of the target object; A projection module, used for projecting the second three-dimensional model to obtain a stylized image of the target object; A fusion module is used to fuse the stylized image with the image to be processed to obtain a processed image.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images