An image processing method, apparatus, device and storage medium

Abstract

The present disclosure relates to an image processing method, device, equipment and storage medium. The method comprises the following steps: determining a source style basis and a target style basis corresponding to an original two-dimensional object in a to-be-processed image, the source style basis being used to represent a style corresponding to the original two-dimensional object, and the target style basis being used to represent a style required to be converted by the original two-dimensional object, each style being used to represent a shape feature of a corresponding object; performing three-dimensional reconstruction processing on the two-dimensional object by using a three-dimensional reconstruction network corresponding to the source style basis and a three-dimensional reconstruction network corresponding to the target style basis, respectively, to obtain a source three-dimensional object and a target three-dimensional object; and performing style conversion processing on the to-be-processed image according to the source three-dimensional object and the target three-dimensional object, to obtain a target image including a target two-dimensional object, so that the target two-dimensional object is biased towards the target style while not losing the source style, thereby improving the natural degree of two-dimensional object deformation and improving the optimization effect.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of internet technology, image processing of two-dimensional objects has increasingly attracted attention and importance from various industries. Related technologies involve extracting key point information of the two-dimensional object from the image to be processed; splitting the two-dimensional object into parts; using corresponding keypoint transfer networks to transfer key points of different parts of the two-dimensional object to obtain target transfer key point information; simultaneously performing 3D reconstruction processing on the two-dimensional object to obtain the 3D model to be processed; and adjusting the 3D model to be processed using the key point information and the target transfer key point information of each part to obtain the target 3D model. However, this image processing method leads to inconsistent deformation at the connections between different parts of the overall virtual object, resulting in the loss of the unique characteristics of the target virtual object itself. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, device, and storage medium to at least solve the problems of loss of the inherent characteristics of the target virtual object and inconsistent deformation at the connection points of different parts obtained in related technologies. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0005] Determine the source style base and target style base of the original two-dimensional object in the image to be processed; the source style base is used to characterize the style corresponding to the original two-dimensional object, the target style base is used to characterize the style that the original two-dimensional object needs to be transformed, and each style is used to characterize the shape features of the corresponding object;

[0006] Using the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base, the 2D object is subjected to 3D reconstruction processing to obtain the source 3D object and the target 3D object; wherein the source 3D object is the 3D object corresponding to the source style base, and the target 3D object is the 3D object corresponding to the target style base;

[0007] Based on the source 3D object and the target 3D object, the image to be processed is subjected to style conversion processing to obtain a target image including the target 2D object; the target 2D object is a 2D object with the same style as the target 3D object.

[0008] In some exemplary embodiments,

[0009] Before performing 3D reconstruction processing on the 2D object using the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base to obtain the source 3D object and the target 3D object, the method further includes:

[0010] Determine the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base.

[0011] In some exemplary embodiments,

[0012] The step of performing style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image containing the target 2D object includes:

[0013] The source 3D object and the target 3D object are projected onto the 2D space where the 2D object is located, respectively, to obtain a first projection and a second projection; the first projection is the projection of the source 3D object in the 2D space, and the second projection is the projection of the target 3D object in the 2D space;

[0014] The conversion parameters are determined based on the first projection and the second projection;

[0015] Based on the conversion parameters, the image to be processed is converted to obtain the target image containing the target two-dimensional object.

[0016] In some exemplary embodiments, the step of performing style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image including the target 2D object includes:

[0017] The source 3D object is migrated to the target 3D object by a preset distance to obtain the migrated 3D object;

[0018] The migrated 3D object is projected into a 2D space to obtain the target image containing the target 2D object.

[0019] In some exemplary embodiments, the 3D reconstruction network corresponding to the source style base includes a source encoding network and a first fitting network, and the 3D reconstruction network corresponding to the target style base includes a target encoding network and a second fitting network; the step of performing 3D reconstruction processing on the 2D object using the source 3D reconstruction network and the target 3D reconstruction network respectively to obtain the source 3D object and the target 3D object includes:

[0020] The source coding network is used to perform feature extraction processing on the original two-dimensional object to obtain the source feature parameters corresponding to the source style base; the source feature parameters are used to characterize the image features associated with the source two-dimensional object and the source style base.

[0021] The target coding network is used to perform feature extraction processing on the original two-dimensional object to obtain the target feature parameters corresponding to the target style base; the target feature parameters are used to characterize the image features associated with the two-dimensional object and the target style base.

[0022] Based on the first fitting network, parameter fitting processing is performed on the source feature parameters, the source style base and the preset expression base to obtain the source 3D object corresponding to the source style base;

[0023] Based on the second fitting network, parameter fitting processing is performed on the target feature parameters, the target style base, and the preset expression base to obtain the target 3D object corresponding to the target style base.

[0024] In some exemplary embodiments, the method includes:

[0025] Obtain sample two-dimensional objects from sample images; the sample two-dimensional objects are two-dimensional objects corresponding to preset style bases; the preset style bases include the source style base and the target style base;

[0026] The sample two-dimensional object is input into a preset neural network corresponding to a preset style base for feature extraction processing to obtain the sample feature parameters corresponding to the preset style base.

[0027] The sample feature parameters, the preset style base, and the general expression base are subjected to object prediction processing to obtain a predicted two-dimensional object;

[0028] Based on the sample two-dimensional object and the predicted two-dimensional object, determine the loss information of the two-dimensional object corresponding to the preset style base;

[0029] A preset neural network is trained based on the loss information of the two-dimensional object corresponding to the preset style base to obtain the three-dimensional reconstruction network corresponding to the preset style base.

[0030] In some embodiments, the method further includes:

[0031] Scan the three-dimensional objects in multiple sample images under the preset style base to obtain multiple three-dimensional meshes corresponding to the preset style base;

[0032] Based on the plurality of three-dimensional meshes corresponding to the preset style base, the shape features under the preset style base are determined, and the shape features under the preset style base are used as the preset style base.

[0033] In some embodiments, each 3D mesh contains multiple feature points, and determining the shape features under the preset style base based on the multiple 3D meshes corresponding to the preset style base, and using the shape features under the preset style base as the preset style base, includes:

[0034] Based on multiple feature points in each three-dimensional mesh corresponding to the preset style base, the mean three-dimensional mesh under the preset style base and the first preset number of shape features corresponding to each three-dimensional mesh are obtained;

[0035] Determine the feature values ​​of the first preset number of shape features;

[0036] Arrange the first preset number of shape features according to the magnitude of the feature values;

[0037] Based on the arrangement order of the first preset number of shape features, the first preset number of shape features are subjected to dimensionality reduction processing to obtain the second preset number of shape features corresponding to each three-dimensional mesh, and the second preset number of shape features are used as the preset style base.

[0038] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:

[0039] The style base determination module is configured to determine the source style base and the target style base corresponding to a two-dimensional object in the image to be processed; the source style base and the target style base are used to characterize the shape features of the corresponding object.

[0040] The reconstruction processing module is configured to perform three-dimensional reconstruction processing on the two-dimensional object based on the source style base and the target style base to obtain the source three-dimensional object and the target three-dimensional object;

[0041] The style transfer processing module is configured to perform style transfer processing on the two-dimensional object based on the source three-dimensional object and the target three-dimensional object to obtain the target two-dimensional object.

[0042] In some exemplary embodiments, the reconstruction processing module may include:

[0043] The network acquisition unit is configured to acquire the source 3D reconstruction network corresponding to the source style base and the target 3D reconstruction network corresponding to the target style base;

[0044] The 3D reconstruction unit is configured to perform 3D reconstruction processing on the 2D object using the source 3D reconstruction network and the target 3D reconstruction network respectively, to obtain the source 3D object and the target 3D object.

[0045] In some exemplary embodiments, the style transfer processing module may include:

[0046] The first projection unit is configured to project the source 3D object and the target 3D object onto the 2D space where the 2D object is located, respectively, to obtain the first projection and the second projection.

[0047] The conversion parameter determination unit is configured to determine conversion parameters based on the first projection and the second projection;

[0048] The conversion processing unit is configured to perform conversion processing on the two-dimensional object based on the conversion parameters to obtain the target two-dimensional object.

[0049] In some exemplary embodiments, the style transfer processing module may include:

[0050] The migration unit is configured to migrate the source 3D object to the target 3D object by a preset distance to obtain the migrated 3D object;

[0051] The second projection unit is configured to project the migrated 3D object into a 2D space to obtain the target 2D object.

[0052] In some exemplary embodiments, the source 3D reconstruction network includes a source encoding network and a first fitting network, and the target 3D reconstruction network includes a target encoding network and a second fitting network; the 3D reconstruction unit includes:

[0053] The feature parameter processing unit is configured to perform feature extraction processing on the two-dimensional object using the source coding network and the target coding network respectively, to obtain the source feature parameters corresponding to the source style base and the target feature parameters corresponding to the target style base;

[0054] The first parameter fitting processing unit is configured to perform parameter fitting processing on the source feature parameters, the source style base and the preset expression base based on the first fitting network to obtain the source 3D object corresponding to the source style base.

[0055] The second parameter fitting processing unit is configured to perform parameter fitting processing on the target feature parameters, the target style base, and the preset expression base based on the second fitting network to obtain the target 3D object corresponding to the target style base.

[0056] In some embodiments, the device may further include:

[0057] The sample data acquisition module is configured to acquire sample two-dimensional objects in a sample image; the sample two-dimensional objects are two-dimensional objects corresponding to a preset style base.

[0058] The feature extraction module is configured to input the sample two-dimensional object into a preset neural network corresponding to a preset style base for feature extraction processing to obtain sample feature parameters corresponding to the preset style base;

[0059] The prediction module is configured to perform object prediction processing on the sample feature parameters, the preset style base, and the general expression base to obtain a preset two-dimensional object;

[0060] The loss information determination module is configured to determine the loss information of the two-dimensional object corresponding to the preset style base based on the sample two-dimensional object and the predicted two-dimensional object;

[0061] The training processing module is configured to train a corresponding preset neural network based on the loss information of the two-dimensional object corresponding to the preset style base, so as to obtain a three-dimensional reconstruction network corresponding to the preset style base.

[0062] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0063] processor;

[0064] Memory used to store the processor's executable instructions;

[0065] The processor is configured to execute the instructions to implement the aforementioned image processing method.

[0066] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, comprising:

[0067] When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the aforementioned image processing method.

[0068] Implementing this application will have the following beneficial effects:

[0069] By determining the source style base and target style base corresponding to the two-dimensional object in the image to be processed, and performing three-dimensional reconstruction processing on the two-dimensional object based on the source style base and target style base, a source three-dimensional object and a target three-dimensional object are obtained. The source three-dimensional object has a high correlation with the source style to which the two-dimensional object belongs, and the target three-dimensional object obtained by three-dimensional reconstruction based on the target style base has a high correlation with the target style. Based on the source three-dimensional object and the target three-dimensional object, style transfer is performed on the two-dimensional object to obtain the target two-dimensional object. This allows the target two-dimensional object to be biased towards the target style without losing the source style, thereby improving the naturalness of the deformation of the two-dimensional object and enhancing the optimization effect. Attached Figure Description

[0070] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0071] Figure 1 This is a schematic diagram illustrating an implementation environment according to an exemplary embodiment.

[0072] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0073] Figure 3 This is a flowchart illustrating an exemplary embodiment of a three-dimensional reconstruction process of a two-dimensional object based on a source style base and a target style base, to obtain a source three-dimensional object and a target three-dimensional object.

[0074] Figure 4 This is a flowchart illustrating a three-dimensional reconstruction using a three-dimensional reconstruction network according to an exemplary embodiment.

[0075] Figure 5 This is a flowchart illustrating, according to an exemplary embodiment, a method for performing style transfer processing on an image to be processed based on a source 3D object and a target 3D object to obtain a target image containing the target 2D object.

[0076] Figure 6 This is a flowchart illustrating, according to an exemplary embodiment, a method for performing style transfer processing on an image to be processed based on a source 3D object and a target 3D object to obtain a target 2D object.

[0077] Figure 7 This is a flowchart illustrating a three-dimensional reconstruction network training method according to an exemplary embodiment.

[0078] Figure 8 This is a flowchart illustrating the construction of a preset style base according to an exemplary embodiment.

[0079] Figure 9 This is a schematic diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0080] Figure 10 This is a schematic diagram of a three-dimensional reconstruction network training device according to an exemplary embodiment.

[0081] Figure 11 This is a block diagram illustrating an electronic device for training an image processing or 3D reconstruction network according to an exemplary embodiment. Detailed Implementation

[0082] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0083] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0084] Please see Figure 1 The diagram illustrates an implementation environment in an exemplary embodiment of this disclosure, which may include:

[0085] At least one terminal 01 and at least one server 02. The at least one terminal 01 can communicate with the at least one server 02 via a network.

[0086] In an optional embodiment, terminal 01 can send image processing instructions to server 02, which may include data for determining source style bases and target style bases. Terminal 01 may be, but is not limited to, electronic devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. The operating system running on terminal 01 may include, but is not limited to, Android, iOS, Linux, Windows, and Unix.

[0087] In an optional embodiment, server 02 can process the image to be processed based on the image processing instructions provided by terminal 01 to obtain a style-transformed target 2D image, and then send the target 2D image to terminal 01. Optionally, server 02 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0088] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the image processing method provided in this disclosure.

[0089] In the embodiments described in this specification, the terminal 01 and the server 02 can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0090] It should be noted that the following diagram illustrates one possible sequence of steps, and it is not strictly required to follow this order. Some steps can be performed in parallel without interdependence. The user information (including but not limited to user device information, user personal information, user behavior information, etc.) and data (including but not limited to data used for display, training data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0091] Figure 2 This is a flowchart illustrating an image processing method according to an exemplary embodiment. Figure 2 As shown, the image processing method includes the following steps:

[0092] In step S21, the source style base and target style base corresponding to the original two-dimensional object in the image to be processed are determined.

[0093] In the embodiments of this specification, the image to be processed can refer to an image containing an original two-dimensional object. Specifically, the image to be processed can be an image acquired by the terminal and sent to the server, or it can be an image selected by the user from a database; this disclosure does not limit this.

[0094] In the embodiments of this specification, the source style base and the target style base are used to characterize the shape features of the corresponding objects. For example, they may include shape features such as the size of multiple parts of the original two-dimensional object and the curvature of lines. Specifically, the source style base and the target style base can be at least one of multiple style bases, where each style base can refer to the shape features common to multiple objects of the corresponding style. For example, a style base can be the shape features common to multiple objects in a region. Specifically, the shape features can be represented in the form of shape basis vectors. In practical applications, the style base can be a combination of the mean two-dimensional object and the shape features of multiple objects of the corresponding style. Specifically, the mean two-dimensional object can be adjusted based on the shape features to obtain the style base. The source style base is used to characterize the style corresponding to the original two-dimensional object, the target style base is used to characterize the style to be transformed from the original two-dimensional object, and each style is used to characterize the shape features of the corresponding object.

[0095] In the embodiments described in this specification, the source style base can characterize the shape features corresponding to a two-dimensional object in the image to be processed. In some embodiments, the corresponding source style base can be determined by determining the associated geographical location of the image to be processed, etc. The target style base can be a set style base used to characterize the style to be transformed from the original two-dimensional object.

[0096] In practical applications, the terminal can determine the target style base based on the user's selection of the target style and generate image processing instructions based on the target style base. The server can determine the source style base based on the terminal's geographical location, parse the image processing instructions, and determine the target style base.

[0097] In step S22, the three-dimensional reconstruction network corresponding to the source style base and the three-dimensional reconstruction network corresponding to the target style base are used to perform three-dimensional reconstruction processing on the two-dimensional object to obtain the source three-dimensional object and the target three-dimensional object.

[0098] In the embodiments of this specification, performing 3D reconstruction processing on a 2D object based on a source style base and a target style base can involve obtaining a source 3D reconstruction network corresponding to the source style base and a target 3D reconstruction network corresponding to the target style base, and then using the source 3D reconstruction network and the target 3D reconstruction network to perform 3D reconstruction processing on the 2D object respectively, to obtain a source 3D object and a target 3D object. Here, the source 3D object is the 3D object corresponding to the source style base, and the target 3D object is the 3D object corresponding to the target style base.

[0099] In the embodiments of this specification, the source 3D reconstruction network corresponding to the source style basis can be a 3D reconstruction network trained based on the source style basis, and the target 3D reconstruction network corresponding to the target style basis can be a 3D reconstruction network trained based on the target style basis. Using the source 3D reconstruction network to perform 3D reconstruction processing on a 2D object can highlight the style of the source 3D object itself.

[0100] Figure 3 This is a flowchart illustrating an exemplary embodiment of a three-dimensional reconstruction process of a two-dimensional object based on a source style base and a target style base, to obtain a source three-dimensional object and a target three-dimensional object. Specifically, it may include:

[0101] In step S31, the source 3D reconstruction network corresponding to the source style base and the target 3D reconstruction network corresponding to the target style base are obtained.

[0102] In the embodiments of this specification, different style bases can correspond to different 3D reconstruction networks. Different style bases can be used to train 3D reconstruction networks corresponding to different styles in advance. After determining the source style base and the target style base, the 3D reconstruction network corresponding to the source style base can be used as the source 3D reconstruction network, and the 3D reconstruction network corresponding to the target style base can be used as the target 3D reconstruction network, based on the correspondence between the style base and the 3D reconstruction network.

[0103] In step S32, the two-dimensional object is reconstructed into a three-dimensional object using the source three-dimensional reconstruction network and the target three-dimensional reconstruction network respectively, to obtain the source three-dimensional object and the target three-dimensional object.

[0104] In the embodiments described in this specification, the source 3D object and the target 3D object obtained from the 3D reconstruction process have one more dimension than the 2D object. The source 3D reconstruction network and the target 3D reconstruction network can typically employ regression models, such as the ResNet101 network.

[0105] By obtaining the source 3D reconstruction network corresponding to the source style base and the target 3D reconstruction network corresponding to the target style base, the 2D object is reconstructed using the source 3D reconstruction network and the target 3D reconstruction network respectively, resulting in the source 3D object and the target 3D object. After determining the source style and the target style, the corresponding 3D reconstruction network can be quickly determined. Since the 3D reconstruction network corresponding to the preset style base is pre-trained, the efficiency of 3D reconstruction of the source 3D object and the target 3D object can be improved, while the reusability of the 3D reconstruction network in image processing can be enhanced.

[0106] In some embodiments, the source 3D reconstruction network may include a source encoding network and a first fitting network, and the target 3D reconstruction network may include a target encoding network and a second fitting network. Based on this, Figure 4 This is a flowchart illustrating an exemplary embodiment of performing 3D reconstruction processing on a 2D object using a source 3D reconstruction network and a target 3D reconstruction network to obtain a source 3D object and a target 3D object. Specifically, it may include:

[0107] In step S41, the source coding network and the target coding network are used to perform feature extraction processing on the two-dimensional object to obtain the source feature parameters corresponding to the source style base and the target feature parameters corresponding to the target style base.

[0108] In the embodiments of this specification, the source coding network and the target coding network can perform feature extraction processing on a two-dimensional object. Specifically, the two-dimensional object can be input into the source coding network to obtain shape, expression, and pose parameters optimized by the source style base; and the two-dimensional object can be input into the target coding network to obtain shape, expression, and pose parameters optimized by the target style base. Since the source coding network and the target coding network are trained based on the source style base and the target style base respectively, during feature extraction, the source coding network can highlight the shape features represented by the source style base, and the target coding network can highlight the shape features represented by the target style base. That is, the source feature parameters are more biased towards the source style base, and the target feature parameters are more biased towards the target style base, thereby strengthening the style corresponding to the source and target feature parameters.

[0109] In step S42, the source feature parameters, source style base and preset expression base are fitted by the first fitting network to obtain the source 3D object corresponding to the source style base.

[0110] In the embodiments of this specification, the source style basis can be a basis vector representing the shape features of a two-dimensional object, the preset expression basis can be a basis vector representing preset expression features, and the source feature parameters can be the feature matrix corresponding to the source style basis of the two-dimensional object. The source feature parameters, source style basis, and preset expression basis are fitted using a first fitting network. This can be achieved by performing matrix transformation on the feature matrix corresponding to the source style basis of the two-dimensional object based on the basis vectors of the shape features corresponding to the source style and the basis vectors of the preset expression features, to obtain a first transformation matrix. The first transformation matrix is ​​then used to transform the feature points in the mean three-dimensional object to obtain the source three-dimensional object corresponding to the source style basis.

[0111] In step S43, the target feature parameters, target style base, and preset expression base are fitted using the second fitting network to obtain the target 3D object corresponding to the target style base.

[0112] In the embodiments of this specification, the target style basis can be a basis vector representing the shape features of the target style, the preset expression basis can be the same as the preset expression basis in the source 3D object generation process, and the target feature parameters can be the feature matrix corresponding to the target style. The second fitting network can perform matrix transformation processing on the feature matrix corresponding to the source style basis of the 2D object based on the basis vector of the shape features corresponding to the target style and the basis vector of the preset expression features to obtain a second transformation matrix, and use the second transformation matrix to transform the feature points in the mean 3D object to obtain the target 3D object corresponding to the target style basis.

[0113] By using source coding networks and target coding networks to extract features from two-dimensional objects, the source feature parameters corresponding to the source style base and the target feature parameters corresponding to the target style base can be obtained, which can improve the accuracy of the source feature parameters corresponding to the source style base and the target feature parameters corresponding to the target style base.

[0114] In step S23, style conversion processing is performed on the image to be processed based on the source 3D object and the target 3D object to obtain a target 2D object including the target 2D object.

[0115] In the embodiments described in this specification, style conversion processing of a two-dimensional object requires both a source three-dimensional object obtained by enhancing the source style and a target three-dimensional object. This allows for style conversion processing of the target two-dimensional object without losing the characteristics of the source style. The target two-dimensional object is a two-dimensional object with the same style as the target three-dimensional object.

[0116] By determining the source style base and target style base corresponding to the two-dimensional object in the image to be processed, and performing three-dimensional reconstruction processing on the two-dimensional object based on the source style base and target style base, a source three-dimensional object and a target three-dimensional object are obtained. The source three-dimensional object has a high correlation with the source style to which the two-dimensional object belongs, and the target three-dimensional object obtained by three-dimensional reconstruction based on the target style base has a high correlation with the target style. Based on the source three-dimensional object and the target three-dimensional object, style transfer is performed on the two-dimensional object to obtain the target two-dimensional object. This allows the target two-dimensional object to be biased towards the target style without losing the source style, thereby improving the naturalness of the deformation of the two-dimensional object and enhancing the optimization effect.

[0117] Figure 5 This is a flowchart illustrating, according to an exemplary embodiment, a method for performing style transfer processing on an image to be processed based on a source 3D object and a target 3D object to obtain a target image containing the target 2D object. For example... Figure 5 As shown, it can specifically include:

[0118] In step S51, the source 3D object and the target 3D object are projected onto the 2D space where the 2D object is located, respectively, to obtain the first projection and the second projection.

[0119] In the embodiments of this specification, the two-dimensional space where the two-dimensional object resides can be a plane formed by the X-axis and the Y-axis. Projecting the source three-dimensional object and the target three-dimensional object onto the plane formed by the X-axis and the Y-axis can convert the three-dimensional coordinates of key points on the source three-dimensional object and the target three-dimensional object into two-dimensional coordinates based on the X-axis and the Y-axis.

[0120] In step S52, the transformation parameters are determined based on the first projection and the second projection.

[0121] In the embodiments of this specification, the first projection and the second projection can be composed of multiple key points, and offset information can be determined based on the key points in the first projection and the key points in the second projection. Optionally, the key points in the first projection and the second projection can be classified according to their type, for example, according to shape type and expression type, to obtain a key point set corresponding to the shape type and a key point set corresponding to the expression type. It should be noted that a key point in the first projection or the second projection can be one of the first key point set or the second key point set, or it can be in both the first key point set and the second key point set. The offset information is determined based on the key points in the first projection and the key points in the second projection. A first conversion parameter can be determined based on the first key point set corresponding to the shape type in the first projection and the second key point set corresponding to the shape type in the second projection, and a second conversion parameter can be determined based on the third key point set corresponding to the expression type in the first projection and the fourth key point set corresponding to the shape type in the second projection. The final conversion parameter is obtained based on the first conversion parameter and the second conversion parameter. Determining transformation parameters using key points corresponding to shape and expression types in the first and second projections can improve the correlation between transformation parameters and shape / expression, avoiding unnatural transformations caused by excessively dense offset information when determining transformation parameters. The first projection is the projection of the source 3D object into 2D space, and the second projection is the projection of the target 3D object into 2D space.

[0122] Optionally, the key points in the first and second projections can be divided according to the parts of the 2D object. For example, if the 2D object is a 2D face image, the key points in the first and second projections can be divided according to facial features to obtain key point sets corresponding to different parts. Determining the offset information based on the key points in the first and second projections can be achieved by using the key point set corresponding to each part in the first projection and the key point set corresponding to the corresponding part in the second projection to determine transformation parameters corresponding to multiple parts, and then using these transformation parameters as the final transformation parameters.

[0123] In step S53, the image to be processed is transformed based on the transformation parameters to obtain a target image containing the target two-dimensional object.

[0124] In the embodiments of this specification, a grid of the same size as the image to be processed can be generated based on the image to be processed containing the two-dimensional object. The deformation field of the grid can be adjusted using transformation parameters, and the adjusted two-dimensional object can be used as the target two-dimensional object.

[0125] The source 3D object and the target 3D object are projected onto the 2D space where the 2D object is located, respectively, to obtain the first projection and the second projection. The transformation parameters are determined based on the first projection and the second projection. The 2D object is then transformed based on the transformation parameters to obtain the target 2D object. By using the projection method, the features of the source 3D object and the target 3D object in the 2D space can be represented. Furthermore, the transformation parameters are determined based on the projections of the source 3D object and the target 3D object in the 2D space. This can improve the smoothness of each part during the 2D object transformation process, reduce the processing load, and improve processing efficiency. At the same time, it can also preserve the style of the source 2D object to a certain extent, improving the naturalness of the image processing.

[0126] Figure 6 This is a flowchart illustrating, according to an exemplary embodiment, a method for performing style transfer processing on an image to be processed based on a source 3D object and a target 3D object to obtain a target image containing the target 2D object. Specifically, it may include:

[0127] In step S61, the source 3D object is migrated to the target 3D object by a preset distance to obtain the migrated 3D object.

[0128] In the embodiments of this specification, migrating a source 3D object to a target 3D object a preset distance can be achieved by determining key points in the migrating 3D object based on key points in both the source and target 3D objects. Specifically, the coordinates of each key point in the source and target 3D objects can be determined, and the average of the coordinates of every two corresponding key points in both objects can be taken. The point corresponding to the coordinates of this average can be used as the key point of the migrating 3D object, and the migrating 3D object can be determined based on these key points.

[0129] In step S62, the migrated 3D object is projected onto a 2D space to obtain the target 2D object.

[0130] By migrating the source 3D object to the target 3D object, a migrated 3D object is obtained. The migrated 3D object is then projected into a 2D space to obtain the target 2D object. Since the migrated object is a 3D object, the accuracy of the migration process can be improved.

[0131] Figure 7 This is a flowchart illustrating a 3D reconstruction network training method according to an exemplary embodiment. Specifically, the method may include:

[0132] In step S71, the sample two-dimensional object in the sample image is obtained.

[0133] In the embodiments of this specification, the sample two-dimensional object can be a two-dimensional object corresponding to multiple preset style bases. A two-dimensional object corresponding to a preset style base can refer to a two-dimensional object with a preset style. Sample two-dimensional objects with multiple preset styles can be collected in advance. The preset style base can include, but is not limited to, a source style base and a target style base.

[0134] In step S72, the sample two-dimensional object is input into the preset neural network corresponding to the preset style base for feature extraction processing to obtain the sample feature parameters corresponding to the preset style base.

[0135] In the embodiments of this specification, the preset style base can be one of multiple style bases, and the preset neural network corresponding to each style base can be the same neural network. When training the preset neural networks corresponding to different style bases, the parameters in the preset neural networks can be adjusted according to different styles to obtain the 3D reconstruction networks corresponding to different style bases.

[0136] In step S73, object prediction processing is performed on the sample feature parameters, preset style base, and general expression base to obtain the predicted two-dimensional object.

[0137] In the embodiments of this specification, the preset style basis can be a shape basis corresponding to a preset style. The shape basis can be used to determine the shape information of a three-dimensional object. Specifically, the shape basis can be one or more sets of basis vectors representing the shape of a three-dimensional object. The preset style basis can be one of multiple preset style bases to which the sample two-dimensional object belongs. The general expression basis can be a basis vector of a preset expression.

[0138] In a specific embodiment, the sample two-dimensional object is input into a preset neural network corresponding to a preset style base for feature extraction processing to obtain sample feature parameters corresponding to the preset style base. This can be achieved by constructing a deep neural network with multiple residual layers as the generator. In the residual block structure of this multi-layer residual layer, the sample two-dimensional object x is taken as input and learned through multiple convolutional layers to obtain sample features F(x). The sample two-dimensional object x and sample features F(x) are superimposed to obtain F(x)+x, and F(x)+x is used as the output of the residual block structure. The gradient can propagate through both F(x) and x, thus avoiding the gradient vanishing phenomenon as the neural network deepens. Finally, the generator can regress to obtain the sample feature parameters y1. Object prediction processing is then performed on the sample feature parameters, the preset style base, and the general expression base to obtain the predicted two-dimensional object. This can be achieved by using multiple fully connected layers to discriminate the regression results of the generator. Specifically, the discriminator's input can include two types of data. The first type of data can be the sample feature parameters y1 obtained from the generator's regression, and the second type of data can be the feature parameters y2 of a real two-dimensional object. These feature parameters y2 include a preset style base and a general expression base. Correspondingly, the discriminator's output also has two types: 0 and 1, representing whether the discriminator determines whether the input comes from the sample feature parameters regressed by the generator or from the second type of data obtained from a real two-dimensional object with a preset style, respectively. The discriminator can learn the feature parameters y2 of the real two-dimensional object, and adversarial generative learning can be performed using the generator and discriminator.

[0139] In step S74, based on the sample two-dimensional object and the predicted two-dimensional object, the loss information of the two-dimensional object corresponding to the preset style base is determined.

[0140] In the embodiments of this specification, the loss information of the two-dimensional object corresponding to the preset style base can be generated at the output of the discriminator D, or it can be the sum of the cross-entropy of the real two-dimensional object and the sample two-dimensional object.

[0141] In step S75, a preset neural network is trained based on the loss information of the two-dimensional object corresponding to the preset style base to obtain the three-dimensional reconstruction network corresponding to the preset style base.

[0142] In the embodiments of this specification, the adversarial process between the generator and the discriminator can be a process of cross-training the generator and the discriminator. Process one: The generator G can be fixed, and the discriminator D can be trained. In this stage, the training objective of the discriminator D is to maximize the loss information. Process two: The discriminator D can be fixed, and the generator G can be trained. In this stage, the objective of the generator G is to minimize the loss information. Under the framework of cross-training the generator G and the discriminator D, the generator G can learn how to generate network parameters that conform to a preset style base and a general expression base.

[0143] In the training process of the 3D reconstruction network, 2D objects with a preset style are selected as sample 2D objects, and a preset style base is also introduced for object prediction processing. This ensures that the 3D reconstruction network fully learns the style base corresponding to the 2D objects during training. At the same time, by combining the learning with the general expression base, the correlation between the 3D objects reconstructed by the 3D reconstruction network and the preset style base can be effectively improved, thereby greatly improving the image processing effect in the subsequent image processing system.

[0144] Figure 8 This is a flowchart illustrating the construction of a preset style base according to an exemplary embodiment. The preset style base can be a statistical model constructed based on a laser scanning dataset of multiple experimental objects. Specifically, before training a 3D reconstruction network, a preset style base can be constructed in advance, which may include:

[0145] In step S81, three-dimensional objects in multiple sample images under a preset style base are scanned to obtain multiple three-dimensional meshes corresponding to the preset style base.

[0146] In the embodiments of this specification, three-dimensional objects in multiple sample images can be scanned. The scanned three-dimensional objects are in point cloud state. Since the number of points is different, the point cloud of each three-dimensional object can be registered into a mesh state with a consistent topology to obtain a three-dimensional mesh corresponding to multiple three-dimensional objects.

[0147] In step S82, shape features under the preset style base are determined based on multiple three-dimensional meshes corresponding to the preset style base, and the shape features under the preset style base are used as the preset style base.

[0148] In the embodiments described in this specification, each 3D object's corresponding 3D mesh may contain multiple feature points. Based on this, the linear weighted sum of the 3D meshes in multiple 3D objects can be used to represent the 3D object model corresponding to a preset style base.

[0149] In step S83, a preset style base is obtained based on shape features.

[0150] In the embodiments of this specification, shape features (shape basis) can be obtained from the linear weighted expression of the three-dimensional mesh of multiple three-dimensional objects, and the shape features can be used as a preset style basis.

[0151] Taking a scan of 200 3D objects as an example, with each 3D mesh containing 53,490 points, the 3D mesh S for each 3D object... i This can be expressed as the following formula:

[0152] S i= (x1, y1, z1, x2, y2, z2, ..., xn, yn, zn), n = 53490, i ∈ [1, 200] and i is a natural number;

[0153] Where x, y, z represent the three-dimensional mesh S i The three-dimensional coordinates of each point in the graph.

[0154] The average 3D object S corresponding to the preset style base model This can be expressed as the following formula:

[0155]

[0156] Here, ui represents the mesh coefficient corresponding to each 3D object.

[0157] By applying a linear weight to these 200 3D objects, we can obtain:

[0158]

[0159] Among them, S mean This represents a 3D object representing the mean of multiple preset style bases, where Pid(i) represents the preset style base.

[0160] The preset style base constructed in the above manner can be matched with the preset style of multiple sample images. By scanning the 3D objects in multiple sample images under the preset style base, multiple 3D meshes corresponding to the preset style base can be obtained. Based on these multiple 3D meshes, the mean 3D mesh and shape features under the preset style base can be determined, thus obtaining the preset style base. This can improve the accuracy of the preset style base, and further greatly improve the accuracy of the 3D reconstruction network training process and image processing process.

[0161] In some exemplary embodiments, each 3D mesh may contain multiple feature points. Determining the mean 3D mesh and shape features under the preset style base based on multiple 3D meshes corresponding to the preset style base may include: obtaining the mean 3D mesh and a first preset number of shape features corresponding to each 3D mesh based on multiple feature points in each 3D mesh corresponding to the preset style base; determining the feature values ​​of the first preset number of shape features; arranging the first preset number of shape features according to the magnitude of the feature values; and performing dimensionality reduction processing on the first preset number of shape features based on the arrangement order of the first preset number of shape features to obtain a second preset number of shape features corresponding to each 3D mesh.

[0162] In the embodiments of this specification, determining the feature values ​​of a first preset number of shape features can be achieved by calculating the feature values ​​of the shape features of different parts of a 3D object, further summing the feature values ​​of the shape features of each part to obtain the feature value corresponding to each 3D object, sorting the feature values ​​corresponding to the 3D objects, selecting the first preset number of feature values, and determining the 3D objects corresponding to the first preset number of feature values. Specifically, the first preset number can be less than the number of the multiple 3D objects. For example, in the dimensionality reduction process S... model This can be expressed as the following formula:

[0163] That is, the dimensionality of the result after linearly weighting 200 three-dimensional objects is reduced to 99.

[0164] By performing dimensionality reduction on the results of linear weighting of multiple 3D objects, errors caused by the non-orthogonal correlation between basis vectors can be avoided, thus improving the accuracy of the preset style basis results.

[0165] Figure 9 This is a schematic diagram illustrating an image processing apparatus according to an exemplary embodiment. Figure 9 As shown, the device may include:

[0166] The style base determination module 91 is configured to determine the source style base and the target style base of the original two-dimensional object in the image to be processed; the source style base is used to characterize the style corresponding to the original two-dimensional object, the target style base is used to characterize the style that the original two-dimensional object needs to be transformed, and each style is used to characterize the shape features of the corresponding object;

[0167] The reconstruction processing module 92 is configured to use the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base to perform 3D reconstruction processing on the 2D object respectively, to obtain the source 3D object and the target 3D object; wherein the source 3D object is the 3D object corresponding to the source style base, and the target 3D object is the 3D object corresponding to the target style base.

[0168] The style transfer processing module 93 is configured to perform style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image including the target 2D object; the target 2D object is a 2D object with the same style as the target 3D object.

[0169] By determining the source style base and target style base corresponding to the two-dimensional object in the image to be processed, and performing three-dimensional reconstruction processing on the two-dimensional object based on the source style base and target style base, a source three-dimensional object and a target three-dimensional object are obtained. The source three-dimensional object has a high correlation with the source style to which the two-dimensional object belongs, and the target three-dimensional object obtained by three-dimensional reconstruction based on the target style base has a high correlation with the target style. Based on the source three-dimensional object and the target three-dimensional object, style transfer is performed on the two-dimensional object to obtain the target two-dimensional object. This allows the target two-dimensional object to be biased towards the target style without losing the source style, thereby improving the naturalness of the deformation of the two-dimensional object and enhancing the optimization effect.

[0170] In some exemplary embodiments, the device may further include:

[0171] The network determination module is configured to determine the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base.

[0172] In some exemplary embodiments, the style transfer processing module may include:

[0173] The projection unit is configured to project the source 3D object and the target 3D object onto the 2D space where the 2D object is located, respectively, to obtain a first projection and a second projection; the first projection is the projection of the source 3D object in the 2D space, and the second projection is the projection of the target 3D object in the 2D space;

[0174] The conversion parameter determination unit is configured to determine conversion parameters based on the first projection and the second projection;

[0175] The conversion parameter determination unit is configured to determine conversion parameters based on the first projection and the second projection;

[0176] The conversion processing unit is configured to perform conversion processing on the image to be processed based on the conversion parameters to obtain the target image containing the target two-dimensional object.

[0177] In some exemplary embodiments, the style transfer processing module may include:

[0178] The migration unit is configured to migrate the source 3D object to the target 3D object by a preset distance to obtain the migrated 3D object;

[0179] The second projection unit is configured to project the migrated three-dimensional object into a two-dimensional space to obtain the target image containing the target two-dimensional object.

[0180] In some exemplary embodiments, the source 3D reconstruction network includes a source encoding network and a first fitting network, and the target 3D reconstruction network includes a target encoding network and a second fitting network; the 3D reconstruction unit includes:

[0181] The feature parameter processing unit is configured to perform feature extraction processing on the original two-dimensional object using the source coding network to obtain the source feature parameters corresponding to the source style base; the source feature parameters are used to characterize the image features associated with the source two-dimensional object and the source style base.

[0182] The target feature parameter determination unit is configured to perform feature extraction processing on the original two-dimensional object using the target coding network to obtain the target feature parameters corresponding to the target style base; the target feature parameters are used to characterize the image features associated with the two-dimensional object and the target style base.

[0183] The source 3D object determination unit is configured to perform parameter fitting processing on the source feature parameters, the source style base and the preset expression base based on the first fitting network to obtain the source 3D object corresponding to the source style base.

[0184] The target 3D object determination unit is configured to perform parameter fitting processing on the target feature parameters, the target style base, and the preset expression base based on the second fitting network to obtain the target 3D object corresponding to the target style base.

[0185] Figure 10 This is a schematic diagram illustrating a three-dimensional reconstruction network training apparatus according to an exemplary embodiment. Figure 10 As shown, the device may include:

[0186] The sample data acquisition module 101 is configured to acquire sample two-dimensional objects in a sample image; the preset style base includes the source style base and the target style base;

[0187] The feature extraction module 102 is configured to input the sample two-dimensional object into a preset neural network corresponding to a preset style base for feature extraction processing to obtain sample feature parameters corresponding to the preset style base.

[0188] Prediction module 103 is configured to perform object prediction processing on the sample feature parameters, the preset style base and the general expression base to obtain a preset two-dimensional object;

[0189] The loss information determination module 104 is configured to determine the loss information of the two-dimensional object corresponding to the preset style base based on the sample two-dimensional object and the predicted two-dimensional object;

[0190] The training processing module 105 is configured to train a corresponding preset neural network based on the loss information of the two-dimensional object corresponding to the preset style base, so as to obtain a three-dimensional reconstruction network corresponding to the preset style base.

[0191] In some exemplary embodiments, the device may further include:

[0192] The scanning module is configured to scan three-dimensional objects in multiple sample images under the preset style base to obtain multiple three-dimensional meshes corresponding to the preset style base;

[0193] The feature determination module is configured to determine the mean three-dimensional mesh and shape features under the preset style base based on the plurality of three-dimensional meshes corresponding to the preset style base;

[0194] The style base determination module is configured to obtain the preset style base based on the shape features.

[0195] In some exemplary embodiments, each 3D mesh contains multiple feature points, and the feature determination module includes:

[0196] The shape feature determination unit is configured to obtain the mean three-dimensional grid under the preset style base and the first preset number of shape features corresponding to each three-dimensional grid based on multiple feature points in each three-dimensional grid corresponding to the preset style base.

[0197] The feature value determination unit is configured to determine the feature values ​​of the first preset number of shape features;

[0198] The sorting unit is configured to arrange the first preset number of shape features according to the magnitude of the feature values;

[0199] The dimensionality reduction processing unit is configured to perform dimensionality reduction processing on the first preset number of shape features based on the arrangement order of the first preset number of shape features, to obtain a second preset number of shape features corresponding to each three-dimensional mesh.

[0200] Figure 11 This is a block diagram illustrating an electronic device for training an image processing or 3D reconstruction network according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 11As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an image processing method or a 3D reconstruction network training method.

[0201] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0202] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the multimedia location information processing method as described in the embodiments of this disclosure.

[0203] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the image processing method or the 3D reconstruction network training method of the present disclosure. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0204] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the image processing method or the 3D reconstruction network training method of the present disclosure embodiments.

[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0206] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0207] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Determine the source style base and target style base of the original two-dimensional objects in the image to be processed; The source style base is used to characterize the style corresponding to the original two-dimensional object, the target style base is used to characterize the style that the original two-dimensional object needs to be transformed into, and each style is used to characterize the shape features of the corresponding object; Using the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base, the 2D object is subjected to 3D reconstruction processing to obtain the source 3D object and the target 3D object; wherein, the source 3D object is the 3D object corresponding to the source style base, and the target 3D object is the 3D object corresponding to the target style base; Based on the source 3D object and the target 3D object, the image to be processed is subjected to style conversion processing to obtain a target image including the target 2D object; the target 2D object is a 2D object with the same style as the target 3D object. Before performing 3D reconstruction processing on the 2D object using the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base to obtain the source 3D object and the target 3D object, the method further includes: A regression model is used to determine the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base.

2. The method according to claim 1, characterized in that, The step of performing style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image containing the target 2D object includes: The source 3D object and the target 3D object are projected onto the 2D space where the 2D object is located, respectively, to obtain a first projection and a second projection; the first projection is the projection of the source 3D object in the 2D space, and the second projection is the projection of the target 3D object in the 2D space; Based on the first projection and the second projection, determine the transformation parameters; Based on the conversion parameters, the image to be processed is converted to obtain the target image containing the target two-dimensional object.

3. The method according to claim 1, characterized in that, The step of performing style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image including the target 2D object includes: The source 3D object is migrated to the target 3D object by a preset distance to obtain the migrated 3D object; The migrated 3D object is projected into a 2D space to obtain the target image containing the target 2D object.

4. The method according to claim 1, characterized in that, The 3D reconstruction network corresponding to the source style base includes a source encoding network and a first fitting network, and the 3D reconstruction network corresponding to the target style base includes a target encoding network and a second fitting network; the step of performing 3D reconstruction processing on the 2D object using the source 3D reconstruction network and the target 3D reconstruction network respectively to obtain the source 3D object and the target 3D object includes: The source coding network is used to perform feature extraction processing on the original two-dimensional object to obtain the source feature parameters corresponding to the source style base; The source feature parameters are used to characterize the image features associated with the original two-dimensional object and the source style base; The target coding network is used to perform feature extraction processing on the original two-dimensional object to obtain the target feature parameters corresponding to the target style base; the target feature parameters are used to characterize the image features associated with the two-dimensional object and the target style base. Based on the first fitting network, parameter fitting processing is performed on the source feature parameters, the source style base and the preset expression base to obtain the source 3D object corresponding to the source style base; Based on the second fitting network, parameter fitting processing is performed on the target feature parameters, the target style base, and the preset expression base to obtain the target 3D object corresponding to the target style base.

5. The method according to claim 1, characterized in that, The method further includes: Obtain sample two-dimensional objects from sample images; the sample two-dimensional objects are two-dimensional objects corresponding to preset style bases; the preset style bases include the source style base and the target style base; The sample two-dimensional object is input into a preset neural network corresponding to a preset style base for feature extraction processing to obtain the sample feature parameters corresponding to the preset style base. The sample feature parameters, the preset style base, and the general expression base are subjected to object prediction processing to obtain a predicted two-dimensional object; Based on the sample two-dimensional object and the predicted two-dimensional object, determine the loss information of the two-dimensional object corresponding to the preset style base; Based on the loss information of the two-dimensional object corresponding to the preset style base, a corresponding preset neural network is trained to obtain the three-dimensional reconstruction network corresponding to the preset style base.

6. The method according to claim 5, characterized in that, The method further includes: Scan the three-dimensional objects in multiple sample images under the preset style base to obtain multiple three-dimensional meshes corresponding to the preset style base; Based on the plurality of three-dimensional meshes corresponding to the preset style base, the shape features under the preset style base are determined, and the shape features under the preset style base are used as the preset style base.

7. The method according to claim 6, characterized in that, Each 3D mesh contains multiple feature points. The step of determining the shape features under the preset style base based on the multiple 3D meshes corresponding to the preset style base, and using the shape features under the preset style base as the preset style base, includes: Based on multiple feature points in each three-dimensional mesh corresponding to the preset style base, the mean three-dimensional mesh under the preset style base and the first preset number of shape features corresponding to each three-dimensional mesh are obtained. Determine the feature values ​​of the first preset number of shape features; Arrange the first preset number of shape features according to the magnitude of the feature values; Based on the arrangement order of the first preset number of shape features, the first preset number of shape features are subjected to dimensionality reduction processing to obtain the second preset number of shape features corresponding to each three-dimensional mesh, and the second preset number of shape features are used as the preset style base.

8. An image processing apparatus, characterized in that, The device includes: The style base determination module is configured to determine the source style base and target style base of the original two-dimensional object in the image to be processed; the source style base is used to characterize the style corresponding to the original two-dimensional object, the target style base is used to characterize the style that the original two-dimensional object needs to be transformed, and each style is used to characterize the shape features of the corresponding object; The reconstruction processing module is configured to use the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base to perform 3D reconstruction processing on the 2D object respectively, to obtain the source 3D object and the target 3D object; wherein the source 3D object is the 3D object corresponding to the source style base, and the target 3D object is the 3D object corresponding to the target style base; The style transfer processing module is configured to perform style transfer processing on the image to be processed based on the source 3D object and the target 3D object to obtain a target image including the target 2D object; the target 2D object is a 2D object with the same style as the target 3D object. Before performing 3D reconstruction processing on the 2D object using the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base to obtain the source 3D object and the target 3D object, the method further includes: A regression model is used to determine the 3D reconstruction network corresponding to the source style base and the 3D reconstruction network corresponding to the target style base.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the image processing method as described in any one of claims 1 to 7.

Images