Image processing method, device, computer and readable storage medium

Abstract

Embodiments of the present application disclose an image processing method and device, a computer and a readable storage medium. The method comprises: acquiring an image sample, identifying planar coordinate information of a key part object in the image sample, and identifying spatial part information of the key part object; in an initial spatial part reconstruction model, identifying spatial contour coordinate information of the spatial part information, performing rasterization processing on the spatial part information to obtain projection camera parameters, performing planar projection processing on the spatial contour coordinate information based on the projection camera parameters to obtain projection coordinate information; generating a first loss function according to the planar coordinate information and the projection coordinate information, and generating a second loss function according to the spatial part information; and performing parameter adjustment on the initial spatial part reconstruction model based on the first loss function and the second loss function to obtain a spatial part reconstruction model. The present application can improve the accuracy of image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an image processing method and device, computer and readable storage medium. BACKGROUND

[0002] With the development of three-dimensional (3-dimension, 3D) scanning technology and the progress of face reconstruction technology, the application of face reconstruction is more and more widely. Among them, face reconstruction is a process of restoring the three-dimensional structure of the face in the two-dimensional image of the face. Generally, sparse face 3D feature points of the face in the image are identified as supervision information and true value 3D feature points. Based on the face 3D feature points and the true value 3D feature points, the face in the image is supervised and adjusted to generate a 3D face model corresponding to the face. Due to the difference between the two-dimensional and three-dimensional levels, there may be a difference between the 3D face model and the face in the image, resulting in low modeling accuracy of the 3D face model. SUMMARY

[0003] The embodiments of the present application provide an image processing method, device, computer and readable storage medium, which can improve the accuracy and detection efficiency of image processing.

[0004] The embodiments of the present application provide an image processing method, device, computer and readable storage medium, which can improve the accuracy and detection efficiency of image processing.

[0005] An image sample is obtained, planar coordinate information of a key part object in the image sample is identified, and spatial part information of the key part object is identified.

[0006] In the initial spatial part reconstruction model, the spatial contour coordinate information of the spatial part information is identified, the spatial part information is rasterized to obtain projection camera parameters, the spatial contour coordinate information is planar projected based on the projection camera parameters to obtain projection coordinate information.

[0007] A first loss function is generated according to the planar coordinate information and the projection coordinate information, and a second loss function is generated according to the spatial part information.

[0008] The initial spatial part reconstruction model is adjusted in parameters based on the first loss function and the second loss function to obtain a spatial part reconstruction model. The spatial part reconstruction model is used for re-rendering processing of a target key part object in a target image.

[0009] The embodiments of the present application provide an image processing device, which comprises:

[0010] A sample acquisition module is configured to acquire an image sample.

[0011] A plane coordinate identification module is configured to identify plane coordinate information of a key part object in an image sample.

[0012] A part identification module is configured to identify spatial part information of the key part object.

[0013] A spatial coordinate identification module is configured to identify spatial contour coordinate information of the spatial part information in an initial spatial part reconstruction model.

[0014] An image projection module is configured to rasterize the spatial part information to obtain projection camera parameters, perform plane projection processing on the spatial contour coordinate information based on the projection camera parameters, and obtain projection coordinate information.

[0015] A first loss generation module is configured to generate a first loss function based on the plane coordinate information and the projection coordinate information.

[0016] A second loss generation module is configured to generate a second loss function based on the spatial part information.

[0017] A model adjustment module is configured to adjust parameters of the initial spatial part reconstruction model based on the first loss function and the second loss function to obtain a spatial part reconstruction model, and the spatial part reconstruction model is configured to perform re-rendering processing on a target key part object in a target image.

[0018] The plane coordinate identification module includes:

[0019] A plane analysis unit is configured to identify plane part features of the key part object in the image sample, analyze the plane part features, and obtain plane contour points corresponding to the key part object.

[0020] A plane mask identification unit is configured to identify a plane mask image of the key part object from the image sample based on the plane contour points.

[0021] A plane coordinate acquisition unit is configured to acquire the plane coordinate information of the key part object from the plane mask image.

[0022] The plane mask identification unit includes:

[0023] A contour connection subunit is configured to connect the plane contour points to obtain a plane contour line and generate an initial mask image based on the image sample.

[0024] A pixel update subunit is configured to update a value of a first pixel point located in a first region indicated by the plane contour line in the initial mask image to a first default pixel value and update a value of a second pixel point located in a second region indicated by the plane contour line to a second default pixel value. The first region refers to a region surrounded by the plane contour line, and the second region refers to a region in the initial mask image other than the first region.

[0025] A plane mask determination sub-unit is configured to determine the updated initial mask image as a plane mask image of the key part object.

[0026] The plane coordinate acquisition unit comprises:

[0027] An image scanning sub-unit is configured to perform line-by-line scanning on the plane mask image to obtain pixel values of mask pixel points included in each plane pixel row in the plane mask image; the plane mask image comprises d plane pixel rows, and d is a positive integer;

[0028] A representation determination sub-unit is configured to obtain target mask pixel points in the i-th plane pixel row whose pixel values belong to the contour pixel range, and determine a representation coordinate corresponding to the i-th plane pixel row according to a plane pixel column in the i-th plane pixel row where the target mask pixel points are located and the i-th plane pixel row; i is a positive integer less than or equal to d;

[0029] A coordinate determination sub-unit is configured to determine plane coordinate information of the key part object according to the representation coordinates corresponding to the d plane pixel rows, respectively.

[0030] The space coordinate identification module comprises:

[0031] A plane acquisition unit is configured to identify a plane part feature of the key part object in the image sample, analyze the plane part feature, and obtain a plane contour point corresponding to the key part object;

[0032] A trapezoid generation unit is configured to obtain a starting contour point and a terminal contour point of the plane contour point, and generate a trapezoidal mask image corresponding to the image sample according to the starting contour point, the terminal contour point, a first image vertex, and a second image vertex of the image sample;

[0033] A rendering acquisition unit is configured to obtain a rendering surface mask image of the space part information in the initial space part reconstruction model.

[0034] A space determination unit is configured to perform merging processing on the trapezoidal mask image and the rendering surface mask image to obtain a space contour line of the key part object, and determine space contour coordinate information corresponding to the space part information based on the space contour line.

[0035] The image projection module comprises:

[0036] A patch acquisition unit is configured to perform rasterization on the space part information by using a differentiable renderer to obtain patch information of each space pixel point in the space part information.

[0037] A parameter generation unit is configured to generate projection camera parameters corresponding to the patch information by using spherical harmonics.

[0038] The projection determination unit is configured to perform planar projection processing on the spatial part information based on the projection camera parameters to obtain projected coordinate information.

[0039] The part recognition module comprises:

[0040] The parameter acquisition unit is configured to acquire spatial part parameters of the key part object in the initial spatial part reconstruction model.

[0041] The average acquisition unit is configured to acquire an average part model.

[0042] The spatial adjustment unit is configured to add the spatial part parameters to the average part model to obtain spatial part information of the key part object.

[0043] The spatial part parameters comprise shape parameters.

[0044] The spatial adjustment unit comprises:

[0045] The shape acquisition subunit is configured to acquire identity parameters and part attribute parameters in the shape parameters, and to identify identity basis vectors and part attribute basis vectors between the key part object and the average part model.

[0046] The shape offset subunit is configured to generate identity offset information according to the identity parameters and the identity basis vectors, and to generate part attribute offset information according to the part attribute parameters and the part attribute basis vectors.

[0047] The shape generation subunit is configured to acquire an average shape part model in the average part model, to add the identity offset information and the part attribute offset information in the average shape part model, and to generate spatial shape information of the key part object.

[0048] The information determination subunit is configured to determine spatial part information of the key part object according to the spatial shape information.

[0049] The projection camera parameters comprise spatial part parameters and illumination parameters.

[0050] When performing planar projection processing on the spatial contour coordinate information based on the projection camera parameters to obtain the projected coordinate information, the image projection module comprises:

[0051] The scale recognition unit is configured to identify scale change information between the key part object and the spatial shape information, and to acquire a projection matrix; the projection matrix is a matrix used for projecting the key part object from space to a plane.

[0052] The direction obtaining unit is configured to identify part direction information of the key part object, and obtain direction offset information between the part direction information and reference direction information; the reference direction information refers to direction information of the key part object indicated by the average part model;

[0053] The plane generating unit is configured to determine an initial plane image of the key part object according to the spatial shape information, the scale change information, the projection matrix and the direction offset information.

[0054] The position offset unit is configured to identify position offset information between the key part object in the image sample and a reference position point, and add the position offset information in the initial plane image to obtain plane projection information; the reference position point refers to a reference pixel point in the image sample.

[0055] The light rendering unit is configured to perform simulated light rendering on the plane projection information by using the light parameter to obtain projection coordinate information.

[0056] The spatial part parameter includes a texture parameter.

[0057] The spatial adjusting unit includes:

[0058] The texture offset sub-unit is configured to obtain texture base data, and determine a product of the texture base data and the texture parameter as texture offset information.

[0059] The texture generating sub-unit is configured to obtain average texture information in the average part model, add the texture offset information to the average texture information to obtain spatial texture information.

[0060] The information determining sub-unit is further configured to determine spatial part information of the key part object according to the spatial texture information.

[0061] The second loss function includes an L2 loss function.

[0062] The second loss generating module includes:

[0063] The feature point obtaining unit is configured to identify spatial feature points of the key part object in the image sample in the initial spatial part reconstruction model, and obtain true value feature points corresponding to the image sample.

[0064] The first loss generating unit is configured to determine the L2 loss function according to difference information between the spatial feature points and the true value feature points.

[0065] The second loss function includes a pixel loss function.

[0066] The second loss generating module includes:

[0067] The re-rendering unit is configured to re-render the spatial part information by using a differentiable renderer to obtain a re-rendered image.

[0068] The pixel difference unit is configured to obtain pixel difference information between a re-rendered pixel point of the re-rendered image and a sample pixel point of the image sample.

[0069] The second loss generation unit is configured to identify skin detection data of the image sample, and determine a pixel loss function based on a dot product result between the pixel difference information and the skin detection data.

[0070] The device further includes:

[0071] The material rendering module is configured to obtain a first material image, identify a first key part object in the first material image by using a spatial part reconstruction model, re-render the first key part object to obtain a planar key part image corresponding to the first key part object.

[0072] The target adjustment module is configured to identify a target key part object in a target image, adjust the target key part object in the target image based on the planar key part image to obtain an updated key part object, and generate a second material image based on the updated key part object. A similarity between a planar contour of the updated key part object and a planar contour of the planar key part image is greater than or equal to a contour similarity threshold.

[0073] The embodiment of the present application provides a computer device, which includes a processor, a memory, and an input / output interface.

[0074] The processor is connected with the memory and the input / output interface, respectively. The input / output interface is configured to receive and output data. The memory is configured to store a computer program. The processor is configured to call the computer program, so that the computer device including the processor executes the image processing method in the embodiment of the present application.

[0075] The embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor, so that a computer device including the processor executes the image processing method in the embodiment of the present application.

[0076] The computer program product or the computer program includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional manners in the aspect of the embodiments of the present application. In other words, the computer instructions are executed by the processor to implement the method provided in various optional manners in the aspect of the embodiments of the present application.

[0077] The embodiments of the present application are implemented, and the following beneficial effects are achieved:

[0078] In the embodiments of the present application, an image sample is obtained, planar coordinate information of a key part object in the image sample is identified, and spatial part information of the key part object is identified. In an initial spatial part reconstruction model, spatial contour coordinate information of the spatial part information is identified, the spatial part information is rasterized to obtain projection camera parameters, the spatial contour coordinate information is planar projected based on the projection camera parameters to obtain projection coordinate information. A first loss function is generated according to the planar coordinate information and the projection coordinate information, and a second loss function is generated according to the spatial part information. The initial spatial part reconstruction model is adjusted in parameters based on the first loss function and the second loss function to obtain a spatial part reconstruction model. The spatial part reconstruction model is used for re-rendering processing of a target key part object in a target image. Through the above process, when the image sample is spatially reconstructed, the projection coordinate information obtained after the key part object in the image sample is re-rendered is compared with the planar coordinate information directly identified from the key part object in the image sample. The spatial part reconstruction model is adjusted in parameters according to the comparison result. The projection coordinate information can be used to represent the contour information of the key part object in the image sample after spatial reconstruction and planar re-rendering. The planar coordinate information is used to represent the planar contour of the key part object in the image sample. Therefore, the contour of the model after spatial reconstruction of the key part object can be constrained to the planar contour of the key part object, so that the contour after re-rendering of the model after spatial reconstruction can be as close as possible to the original planar contour, thereby reducing the ambiguity between the 2D contour on the image level and the 3D contour of the three-dimensional structure, and effectively improving the spatial reconstruction accuracy of the key part object and the accuracy of image processing. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below only constitute some of the embodiments of the present application, and do not constitute all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0080] Figure 1 is a network interaction architecture for image processing provided by an embodiment of the present application;

[0081] Figure 2 is a schematic diagram of an image processing scene provided by an embodiment of the present application;

[0082] Figure 3 is a method flowchart for image processing provided by an embodiment of the present application;

[0083] Figure 4 is a schematic diagram of a first loss function generation scene provided by an embodiment of the present application;

[0084] Figure 5 is a specific flowchart of an image processing method provided by an embodiment of the present application;

[0085] Figure 6 is a schematic diagram of a planar coordinate information acquisition scene provided by an embodiment of the present application;

[0086] Figure 7 is a schematic diagram of a spatial reconstruction training scene provided by an embodiment of the present application;

[0087] Figure 8 is a schematic diagram of a trapezoidal mask image generation scene provided by an embodiment of the present application;

[0088] Figure 9 is a schematic diagram of a part reconstruction scene provided by an embodiment of the present application;

[0089] Figure 10 is a schematic diagram of an image processing device provided by an embodiment of the present application;

[0090] Figure 11 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0091] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments only constitute some of the embodiments of the present application, and do not constitute all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0092] In the embodiments of the present application, please refer to Figure 1 , Figure 1 is a network interaction architecture diagram for image processing provided by the embodiments of the present application, and the embodiments of the present application can be implemented by a computer device 101. Wherein, the computer device 101 can be used to adjust a spatial part reconstruction model, and optionally, the computer device can be a terminal device, or a device corresponding to an application platform, etc., which is not limited here. Wherein, the computer device 101 can interact with one or at least two target devices (such as target device 102a, target device 102b and target device 102c, etc.), and optionally, the computer device 101 can obtain an image sample from any one or more target devices, or obtain an image sample from the computer device 101, adjust parameters of an initial spatial part reconstruction model based on the image sample, and generate a spatial part reconstruction model; or, parameters used to identify spatial part information of the image sample can be adjusted, so that the spatial part information obtained after the image sample is processed by spatial reconstruction can tend to be as close as possible to the plane coordinate information of the image sample itself when projected to a plane dimension, etc. Optionally, the computer device 101 can directly obtain an image sample, adjust parameters of an initial spatial part reconstruction model based on the image sample, and generate a spatial part reconstruction model; or, the computer device 101 can receive a spatial reconstruction request of a target device for an image sample, process the image sample by spatial reconstruction based on the spatial reconstruction request, adjust spatial part information based on plane coordinate information and projection coordinate information obtained by projecting the spatial part information and key part object of the image sample, and obtain a final image reconstruction result, and send the image reconstruction result to the target device which sends the spatial reconstruction request; or, the computer device 101 can obtain an image sample, adjust parameters of an initial spatial part reconstruction model based on the image sample, obtain a spatial part reconstruction model, and add the spatial part reconstruction model to an application platform, etc. Optionally, the computer device 101 can also implement the scheme in the present application in other application scenarios, which is not limited here.

[0093] Specifically, please refer to Figure 2 , Figure 2 is an image processing scene diagram provided by the embodiments of the present application. As Figure 2As shown, the computer device can acquire an image sample 201, identify planar coordinate information of a key part object in the image sample 201, which can be considered as coordinate information corresponding to the key part object in the image sample 201 on a two-dimensional plane, such as coordinate information of planar pixel points or planar contour points of the key part object, and the like. Further, the computer device can identify spatial part information of the key part object, and optionally, the computer device can perform spatial reconstruction processing on the key part object in the image sample to obtain the spatial part information of the key part object. The computer device can identify spatial contour coordinate information of the spatial part information in an initial spatial part reconstruction model, perform rasterization processing on the spatial part information to obtain projection camera parameters, perform planar projection processing on the spatial contour coordinate information based on the projection camera parameters to obtain projection coordinate information. The computer device can generate a first loss function based on the projection coordinate information and the planar coordinate information, generate a second loss function based on the spatial part information, and perform parameter adjustment on the initial spatial part reconstruction model based on the first loss function and the second loss function to generate a spatial part reconstruction model. Wherein, the computer device can constrain and adjust the spatial part information obtained after spatial reconstruction of the key part object in the image sample and the planar coordinate information based on the first loss function, so that the spatial part information approaches the planar coordinate information in the planar dimension; and can perform parameter adjustment on the spatial reconstruction process of the key part object based on the second loss function, so that the spatial part information obtained by performing spatial reconstruction processing on the key part object is more accurate, and thus the accuracy of the spatial reconstruction process of the key part object in the planar and spatial dimensions can be improved.

[0094] It can be understood that the target device mentioned in the embodiments of the present application can be a computer device, and the computer device in the embodiments of the present application includes but is not limited to a terminal device or a server. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to a mobile phone, a tablet computer, a desktop computer, a notebook computer, a palm computer, a vehicle-mounted device, an augmented reality / virtual reality (AR / VR) device, a head-mounted display, a smart television, a wearable device, a smart speaker, a digital camera, a camera, and other mobile internet devices (MID) with network access capability, or terminal devices in scenes such as trains, ships, and aircraft. Among them, the server mentioned above can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road cooperation, content delivery networks (CDN), and big data and artificial intelligence platforms, and other basic cloud computing services.

[0095] Optionally, the data involved in the embodiments of the present application can be stored in the computer device, or can be stored based on cloud storage technology or a blockchain network, which is not limited herein.

[0096] Optionally, the present application can use machine learning technology in the field of artificial intelligence to realize the image denoising process.

[0097] Among them, artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0098] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, automatic driving, intelligent transportation and several other major directions.

[0099] Machine learning (ML) is a multi-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It is a discipline that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and rule-based learning. For example, the parameter adjustment of the initial spatial part reconstruction model and the use of the spatial part reconstruction model in this application.

[0100] Further, please refer to Figure 3 , Figure 3 is a flowchart of an image processing method provided by an embodiment of the present application. As shown in Figure 3 , the image processing process includes the following steps:

[0101] Step S301, acquiring an image sample, identifying the planar coordinate information of the key part object in the image sample, and identifying the spatial part information of the key part object.

[0102] In the embodiment of the present application, the computer device can obtain an image sample. The computer device can obtain the image sample from an image library, or from a target device, or from the Internet, cloud storage space, or a blockchain network, without limitation. The image sample refers to an image including a key part object, that is, the image including the key part object is determined as the image sample. Alternatively, the computer device can obtain an image sample, detect whether the image sample includes a key part object, delete the image sample if it does not include a key part object, and identify the planar coordinate information of the key part in the image sample if it includes a key part object. The key part object refers to a part object having a spatial structure and being displayable in a plane, for example, the key part object can be a face object, a hand object, or a shoulder object, without limitation. Further, the computer device can identify the planar contour points of the key part object in the image sample, and determine the planar coordinate information of the key part object based on the planar contour points. Further, the computer device can perform spatial reconstruction processing on the key part object in the image sample to obtain spatial part information corresponding to the key part object. Specifically, the computer device can perform spatial reconstruction processing on the key part object in the image sample in an initial spatial part reconstruction model to obtain the spatial part information of the key part object. The spatial part information can be regarded as a spatial model for representing the spatial structure and multi-level data information of the key part object.

[0103] For example, please refer to Figure 4 , Figure 4 is a first loss function generation scene diagram provided by the embodiment of the present application. As shown in Figure 4 , the computer device can obtain an image sample 401, identify the planar contour points 4011 corresponding to the key part object in the image sample 401, identify the planar mask image 402 of the key part object from the image sample according to the planar contour points 4011, and obtain the planar coordinate information of the key part object from the planar mask image 402. Further, the computer device can identify the spatial part information 403 of the key part object.

[0104] In step S302, the spatial contour coordinate information of the spatial part information is identified in the initial spatial part reconstruction model, the spatial part information is rasterized to obtain projection camera parameters, the spatial contour coordinate information is planar projected based on the projection camera parameters to obtain projection coordinate information.

[0105] In the embodiment of the present application, the computer device can identify the rendering mask image corresponding to the spatial part information in the initial spatial part reconstruction model, and the size of the rendering mask image is the same as that of the image sample. The contour of the key part object is constrained on the rendering mask image to obtain the spatial contour coordinate information of the spatial part information. Further, the spatial part information can be rasterized by using the differentiable renderer to obtain the projection camera parameter, which is a parameter for projecting the spatial part information from space to a plane. Specifically, the spatial contour coordinate information can be projected to a plane based on the projection camera parameter to obtain the projection coordinate information, which can be used to represent the plane coordinate information obtained by re-rendering the spatial part information.

[0106] Specifically, as shown in Figure 4 The computer device can generate the trapezoidal mask image 404 corresponding to the image sample 401 according to the plane contour point 4011, obtain the rendering mask image 405 of the spatial part information 403 by using the differentiable renderer, integrate the trapezoidal mask image 404 and the rendering mask image 405 to obtain the spatial contour coordinate information corresponding to the spatial part information 403. Further, the spatial part information 403 can be rasterized by using the differentiable renderer to obtain the projection camera parameter, and the spatial contour coordinate information can be projected to a plane based on the projection camera parameter to obtain the projection coordinate information.

[0107] In step S303, the first loss function is generated according to the plane coordinate information and the projection coordinate information, and the second loss function is generated according to the spatial part information.

[0108] In the embodiment of the present application, the computer device can generate the first loss function according to the plane coordinate information and the projection coordinate information. The first loss function can be a square loss function, a mean loss function or a cross-entropy loss function between the plane coordinate information and the projection coordinate information, which is not limited herein. For example, the computer device can obtain the difference between the coordinate information of the plane coordinate point and the coordinate information of the projection coordinate point based on the position correspondence between the plane coordinate point of the plane coordinate information and the projection coordinate point of the projection coordinate information, and generate the first loss function according to the difference between the coordinate information of the plane coordinate point and the coordinate information of the projection coordinate point. As shown in Figure 4As shown, the computer device can generate a first loss function according to the planar coordinate information and the projection coordinate information. The computer device can generate a second loss function based on the spatial part information and the spatial part ground truth corresponding to the spatial part information. The first loss function can constrain the spatial part information of the key part object after spatial reconstruction in terms of planar contour, that is, the spatial part information can be more close to the planar coordinate information of the key part object itself after being re-rendered into a planar image, reducing the influence of the ambiguity between the planar contour and the spatial contour of the spatial structure on spatial reconstruction, etc. The second loss function can represent the accuracy of spatial reconstruction of the key part object.

[0109] In step S304, the initial spatial part reconstruction model is adjusted in parameters based on the first loss function and the second loss function, to obtain a spatial part reconstruction model.

[0110] In the embodiments of the present application, the computer device can adjust the initial spatial part reconstruction model in parameters based on the first loss function and the second loss function, to obtain a spatial part reconstruction model. Optionally, the computer device can perform spatial reconstruction processing again on the key part object based on the spatial part reconstruction model, that is, steps S301 to S304 are performed again to optimize the spatial part reconstruction model, until the first loss function and the second loss function satisfy the parameter optimization condition, to obtain an optimized spatial part reconstruction model. The spatial part reconstruction model can be used for spatial reconstruction processing of a target key part object in a target image, and can be used for re-rendering processing of a target key object.

[0111] In the embodiments of the present application, the computer device can obtain an image sample, adjust the initial spatial part reconstruction model in parameters based on the planar coordinate information of the key part object in the image sample and the projection coordinate information corresponding to the spatial part information obtained after spatial reconstruction processing, to constrain the contour of the spatial part information in the planar dimension, reduce the difference between the spatial part information in the planar dimension and the planar coordinate information, and further reduce the ambiguity between the contour of the spatial part information in the spatial structure and the contour of the key part object in the planar level, effectively improving the spatial reconstruction accuracy of the key part object and the accuracy of image processing.

[0112] Further, referring to Figure 5 , Figure 5 is a specific flowchart of an image processing method provided by the embodiments of the present application. As shown in Figure 5 , the method comprises the following processes:

[0113] In step S501, an image sample is obtained, and planar coordinate information of a key part object in the image sample is identified.

[0114] In the embodiment of the present application, the computer device can obtain an image sample, identify a key part object in the image sample, obtain a planar contour point of the key part object in the image sample, and determine planar coordinate information of the key part object based on the planar contour point. Specifically, the computer device can identify a planar part feature of the key part object in the image sample, analyze the planar part feature to obtain a planar contour point corresponding to the key part object, identify a planar mask image of the key part object from the image sample according to the planar contour point, and obtain the planar coordinate information of the key part object from the planar mask image.

[0115] Further, when obtaining the planar mask image, the computer device can connect the planar contour points to obtain a planar contour line, generate an initial mask image based on the image sample, update a value of a first pixel point located in a first region indicated by the planar contour line in the initial mask image to a first default pixel value, update a value of a second pixel point located in a second region indicated by the planar contour line to a second default pixel value, and determine the updated initial mask image as the planar mask image of the key part object. Specifically, refer to Figure 6 Figure 6 is a planar coordinate information acquisition scene diagram provided by the embodiment of the present application. As shown in Figure 6 , the computer device can obtain an image sample 601, identify a planar contour point 6011 corresponding to a key part object in the image sample 601, connect the planar contour point 6011 to obtain a planar contour line 602, and generate an initial mask image 603 based on the image sample 601 and the planar contour line 602. The computer device updates a value of a first pixel point located in a first region 6031 indicated by the planar contour line 602 in the initial mask image 603 to a first default pixel value, updates a value of a second pixel point located in a second region 6032 indicated by the planar contour line 602 to a second default pixel value, and determines the updated initial mask image as a planar mask image 604 of the key part object. The planar mask image 604 includes an updated first region 6041 and an updated second region 6042. For example, assuming that the first default pixel value is 1, indicating white, and the second default pixel value is 0, indicating black, the planar mask image 604 can be obtained, the updated first region 6041 included in the planar mask image 604 is white, and the updated second region 6042 is black. The first default pixel value and the second default pixel value can also be other pixel values, which are not limited herein. Optionally, the first default pixel value and the second default pixel value are different. Further, the computer device can perform row-by-row traversal on the planar mask image to obtain planar coordinate information of the key object.​

[0116] Furthermore, when obtaining the planar coordinate information of key objects from the planar mask image, the computer device can scan the planar mask image line by line to obtain the pixel values ​​of the mask pixels included in each planar pixel row of the planar mask image; the planar mask image includes d planar pixel rows, where d is a positive integer. The target mask pixel whose pixel value belongs to the contour pixel range in the i-th planar pixel row is obtained. Based on the planar pixel column in the i-th planar pixel row and the i-th planar pixel row, the representation coordinates corresponding to the i-th planar pixel row are determined; i is a positive integer less than or equal to d. Here, the planar pixel column can be used to represent the column coordinates of the target mask pixel in the planar mask image, and the i-th planar pixel row can be used to represent the row coordinates of the target mask pixel in the planar mask image. For example, in the 5th plane pixel row, the plane pixel column where the target mask pixel is located is the 10th to the 21st plane pixel column. Then, the representation coordinates corresponding to the 5th plane pixel row can be considered to be (5, 10), (5, 11), ... to (5, 21). Alternatively, the representation coordinates corresponding to the 5th plane pixel row can be considered to be (5, 10) and (5, 21). Optionally, if there is no target mask pixel in the i-th planar pixel row, the representation coordinates corresponding to the i-th planar pixel row are determined to be empty; if there is one target mask pixel in the i-th planar pixel row, the coordinates of that target mask pixel are determined as the representation coordinates corresponding to the i-th planar pixel row; if there are at least two target mask pixels in the i-th planar pixel row, the coordinates corresponding to the at least two target mask pixels can be combined to form the representation coordinates corresponding to the i-th planar pixel row, or the coordinates of the target mask pixel with the smallest column coordinate and the target mask pixel with the largest column coordinate among the at least two target mask pixels can be determined as the representation coordinates corresponding to the i-th planar pixel row. Similarly, the representation coordinates corresponding to d planar pixel rows can be obtained, and the computer device can determine the planar coordinate information of the key part object based on the representation coordinates corresponding to d planar pixel rows. Optionally, this planar coordinate information can be considered as two-dimensional coordinate information.

[0117] Step S502: Obtain the spatial location parameters of the key part object.

[0118] In this embodiment, the computer device can obtain spatial part parameters of key part objects in the initial spatial part reconstruction model. These spatial part parameters may include shape parameters, texture parameters, and pose parameters, etc. See also... Figure 7 , Figure 7 This is a schematic diagram of a spatial reconstruction training scenario provided in an embodiment of this application. For example... Figure 7As shown, the computer device can extract the spatial part parameters of the key part object in the image sample 701 in the initial spatial part reconstruction model, including but not limited to shape parameters, texture parameters, and posture parameters, etc.

[0119] In step S503, the spatial part information of the key part object is obtained by performing spatial reconstruction processing on the key part object based on the spatial part parameters.

[0120] In the embodiments of the present application, the computer device can obtain an average part model, add the spatial part parameters to the average part model to perform spatial reconstruction processing on the key part object, and obtain the spatial part information of the key part object. Specifically, the spatial part parameters include shape parameters. When the spatial part parameters are added to the average part model to obtain the spatial part information of the key part object, the computer device can obtain the identity parameters and part attribute parameters in the shape parameters, identify the identity basis vector and part attribute basis vector between the key part object and the average part model; generate identity offset information according to the identity parameters and the identity basis vector, and generate part attribute offset information according to the part attribute parameters and the part attribute basis vector; obtain an average shape part model in the average part model, add the identity offset information and the part attribute offset information to the average shape part model to generate the spatial shape information of the key part object; and determine the spatial part information of the key part object according to the spatial shape information. The generation process of the spatial shape information can be referred to formula (1) as shown:

[0121]

[0122] In formula (1), M represents the spatial shape information, represents the average shape part model. Optionally, the average shape part model can be a model composed of at least two model point clouds. The computer device can perform information adding processing on each model pixel point based on the identity offset information and the part attribute offset information to obtain the spatial shape information of the key part object. Wherein, N represents the number of points of the at least two model point clouds, uN represents that each model point cloud in the N model point clouds has u dimensions, and u is a positive integer. That is, the spatial shape information here refers to u-dimensional shape information. For example, if u is 3, the spatial shape information can refer to three-dimensional shape information, etc., and R represents the data domain. id represents the identity parameters, and A id represents the identity basis vector. Optionally, A id The data range of A is used to represent that the identity basis vector is an m id dimensional data; and exp represents the part attribute parameters, and A expis referred to as a part attribute base vector. Optionally, A exp The data range of A is denoted as to indicate that the identity base vector is an m exp dimensional data. Optionally, the computer device can multiply the identity parameter with the identity base vector to obtain identity offset information, and multiply the part attribute parameter with the part attribute base vector to obtain part attribute offset information. Optionally, the computer device can directly obtain the general average shape part model and the identity base vector and the part attribute base vector; or can predict the average shape part model and the identity base vector through the basic part model, and obtain the part attribute base vector through the part attribute library. The average shape part model, the identity base vector and the part attribute base vector are used as a reference for spatial reconstruction of the key part object, and can be considered to form a basic part information. On the basis of the basic part information, the characteristics of the key part object are added to obtain the spatial shape information of the key part object, that is, the average shape part model, the identity base vector and the part attribute base vector are equivalent to a reference. Therefore, the acquisition method of the average shape part model, the identity base vector and the part attribute base vector is not limited to the above acquisition method. The part attribute parameter refers to the parameter of the related attribute of the key part object. For example, if the key part object is a face object, the part attribute parameter can include but is not limited to expression parameters (such as eye expression parameters, mouth expression parameters and nose expression parameters, etc.); if the key part object is a hand object, the part attribute parameter can include but is not limited to hand action parameters and hand basic attribute parameters. The hand action parameter can represent the action of the hand object, such as a fist action and a stretching action, etc. The hand basic attribute parameter can be used to represent the basic data of the hand object, such as the palm display state and the number of fingers, etc.

[0123] The space part parameter can include a texture parameter. When the space part parameter is added to the average part model to obtain the spatial part information of the key part object, the computer device can obtain texture base data, determine the product of the texture base data and the texture parameter as texture offset information, add the texture offset information to the average texture information in the average part model to obtain spatial texture information, and determine the spatial part information of the key part object according to the spatial texture information. The generation process of the spatial texture information can be seen from formula 2:

[0124]

[0125] In formula 2, C represents the spatial texture information. The computer device can obtain the average texture information in the average part model Wherein, B is used to represent texture base data, T is used to represent texture parameters, which are used to represent color and texture information of u model point clouds corresponding to the key part object. Optionally, the computer device can multiply the texture base data B and the texture parameters T to obtain texture offset information, wherein the texture offset information can be represented by a matrix.

[0126] Optionally, the computer device can determine the spatial part information of the key part object according to the spatial shape information, the spatial texture information and the spatial pose information of the key part object, wherein the spatial pose information is determined according to the pose parameters, including but not limited to scaling information and part direction information.

[0127] Optionally, the spatial part information can be considered as u-dimensional part information, such as three-dimensional part information. Figure 7 As shown in FIG. 7, the computer device can perform spatial reconstruction on the key part object based on the spatial part parameters in the differentiable renderer in the initial spatial part reconstruction model, to obtain the spatial part information 7021 of the key part object.

[0128] In step S504, the spatial contour coordinate information is determined based on the planar contour points and the spatial part information.

[0129] In the embodiments of the present application, the computer device can generate a trapezoidal mask image according to the planar contour points, obtain a rendering surface mask image of the spatial part information by using the differentiable renderer, and determine the spatial contour coordinate information based on the trapezoidal mask image and the rendering surface mask image. Specifically, the computer device can identify the planar part features of the key part object in the image sample, analyze the planar part features to obtain the planar contour points corresponding to the key part object, and optionally, the planar contour points obtained when generating the planar coordinate information can be directly obtained; obtain the starting contour point and the ending contour point of the planar contour points, and generate a trapezoidal mask image corresponding to the image sample according to the starting contour point, the ending contour point, the first image vertex and the second image vertex of the image sample. In the initial spatial part reconstruction model, the rendering surface mask image of the spatial part information is obtained, and optionally, the rendering surface mask image can be a color image, and the pixel information of the key part object in u dimensions can be represented by the color value information of the pixel points in the rendering surface mask image. The trapezoidal mask image and the rendering surface mask image are merged to obtain the spatial contour line of the key part object, and the spatial contour coordinate information corresponding to the spatial part information is determined based on the spatial contour line.

[0130] For example, please refer to Figure 8 , Figure 8 is a trapezoidal mask image generation scene schematic diagram provided by an embodiment of the present application. As Figure 8As shown, after the computer device obtains the planar contour point 8011 of the key part object in the image sample 801, the computer device obtains the starting contour point 8021 and the ending contour point 8022 of the planar contour point 8011, and generates a trapezoidal mask image 803 corresponding to the image sample 801 according to the starting contour point 8021, the ending contour point 8022, a first image vertex 8023 and a second image vertex 8024. The generation process of the trapezoidal mask image 803 is the same as that of the planar mask image. The trapezoidal mask image 803 includes a third region 8031 and a fourth region 8032. The third region 8031 is a region surrounded by the starting contour point 8021, the ending contour point 8022, the first image vertex 8023 and the second image vertex 8024. The value of a third pixel point in the third region 8031 is a third default pixel value. The size of the trapezoidal mask image 803 is the same as that of the planar mask image. The fourth region 8032 refers to a region in the trapezoidal mask image 803 other than the third region 8031. The value of a fourth pixel point in the fourth region 8032 is a fourth default pixel value. The third default pixel value is different from the fourth default pixel value. Optionally, the third default pixel value can be the same as the first default pixel value, and the fourth default pixel value can be the same as the second default pixel value. Optionally, the default pixel values can be different from each other.

[0131] The computer device can perform intersection processing on the trapezoidal mask image and the rendering surface mask image to determine a space rendering region. Specifically, the computer device can perform overlap processing on the trapezoidal mask image and the rendering surface mask image, and determine an intersection part between the third region and a rendering surface region in the overlapped trapezoidal mask image and the rendering surface mask image as the space rendering region. The rendering surface region refers to a region composed of pixel points in the rendering surface mask image whose pixel values satisfy the rendering pixel range. The computer device can perform line-by-line scanning on the space rendering region to obtain space contour coordinate information corresponding to the space part information. Specifically, the computer device can obtain pixel coordinates of pixel points included in the space rendering region, and determine the pixel coordinates of the pixel points included in the space rendering region as the space contour coordinate information corresponding to the space part information. Alternatively, the computer device can obtain a space contour line from the space rendering region, and determine pixel coordinates of pixel points corresponding to the space contour line as the space contour coordinate information corresponding to the space part information.

[0132] In step S505, rasterization processing is performed on the space part information to obtain the projection camera parameter.

[0133] In the embodiments of the present application, the computer device can rasterize the spatial part information to obtain the projection camera parameter, wherein the projection camera parameter can include but is not limited to the spatial part parameter and the illumination parameter, etc., and optionally, the illumination parameter can be considered as the spatial part parameter. Wherein, the computer device can obtain the spatial part parameter and the illumination parameter generated when the key part object is spatially reconstructed in the initial spatial part reconstruction model, and determine the projection camera parameter according to the spatial part parameter and the illumination parameter. Alternatively, the computer device can use a differentiable renderer to rasterize the spatial part information to obtain the patch information of each spatial pixel point in the spatial part information, wherein the spatial pixel point is wrapped by a triangular patch, that is, the spatial pixel point corresponds to at least two patch vertices, and the vertex information corresponding to each patch vertex constitutes the patch information of the spatial pixel point; and a spherical harmonic function is used to generate the projection camera parameter corresponding to the patch information. Wherein, the spherical harmonic function is used to simulate the projection illumination. The generation process of the vertex information can be seen from formula ③:

[0134]

[0135] In formula ③, v k represents the vertex information of the kth patch vertex, which is used to represent the radiance (such as color, etc.) and other information of the kth patch vertex, and k is a positive integer. C k represents the vertex texture information corresponding to the kth patch vertex in the spatial texture information; E represents the number of spherical harmonics, and one spherical harmonic corresponds to at least two spherical harmonic functions, wherein f() represents the spherical harmonic basis function, and Lap represents the spherical harmonic function parameter; n k represents the normal of the kth patch vertex. The computer device can use the spherical harmonic basis function to obtain the first vertex information of the patch vertex, weight process the first vertex information based on the spherical harmonic function parameter to generate the second vertex information of the patch vertex, add the vertex texture information of the patch vertex in the second vertex information to obtain the vertex information of the patch vertex. Similarly, the vertex information of at least two patch vertices corresponding to the spatial pixel point can be obtained, and the vertex information of at least two patch vertices is combined to constitute the patch information corresponding to the spatial pixel point. Optionally, if the spatial pixel point corresponds to at least two triangular patches, the triangular patch closest to the simulated projection lens is obtained as the triangular patch used for subsequent processing of the spatial pixel point. For example, the spatial pixel point 1 corresponds to the triangular patch 1, the triangular patch 2 and the triangular patch 3, wherein the distance between the triangular patch 2 and the simulated projection lens is less than the distance between the triangular patch 1 and the simulated projection lens, and less than the distance between the triangular patch 3 and the simulated projection lens, and the triangular patch 2 is determined as the final triangular patch corresponding to the spatial pixel point 1. The spherical harmonic function is used to determine the projection relationship between the patch information and the spatial pixel point, and the projection camera parameter is determined based on the projection relationship.

[0136] Optionally, this projection relationship can be expressed as shown in formula ④:

[0137]

[0138] In formula ④, A triangular facet is used to represent a spatial pixel (x, y), with vertices v1, v2, and v3. α is used to represent the centroid of the triangular facet. k This is used to represent the vertex v of the triangle when the centroid of the triangle is obtained. k The weight.

[0139] Step S506: Determine the projection coordinate information based on the spatial contour coordinate information and the projection camera parameters.

[0140] In this embodiment, a computer device can perform planar projection processing on spatial part information based on projection camera parameters to obtain projected coordinate information. Specifically, it performs planar projection processing on the spatial contour coordinate information generated based on the spatial part information to obtain projected coordinate information. The projection camera parameters include spatial part parameters and illumination parameters. When performing planar projection processing on the spatial contour coordinate information based on the projection camera parameters to obtain projected coordinate information, the computer device can identify the scale change information between the key part object and the spatial shape information, and obtain a projection matrix; the projection matrix is ​​a matrix used to project the key part object from space onto a plane. It identifies the part orientation information of the key part object and obtains the directional offset information between the part orientation information and the reference orientation information; the reference orientation information refers to the orientation information of the key part object indicated by the average part model. Based on the spatial shape information, scale change information, projection matrix, and directional offset information, an initial planar image of the key part object is determined. The positional offset information between the key part object in the image sample and the reference position point is identified, and the positional offset information is added to the initial planar image to obtain planar projection information; the reference position point refers to the reference pixel point in the image sample. The planar projection information is simulated using lighting parameters to obtain projection coordinate information, such as... Figure 7 Optionally, the computer device can generate a re-rendered image 703 based on the spatial location information 7021 and the projected image 7022, and can generate a first loss function and a second loss function based on the re-rendered image 703, the planar coordinate information, and the projected coordinate information. The process of generating the planar projection information can be seen in formula ⑤:

[0141] M p =s*p r *dir*M+δ ⑤

[0142] wherein, in formula (5), M p s is used to represent scale change information (may be used to represent zoom information), p r is a projection matrix, dir is used to represent direction offset information, M is spatial shape information, and δ is used to represent position offset information, which can represent the position information of the key part object in the image sample. Wherein, the direction (i.e., reference direction information) and size corresponding to the key part object indicated by the average part model can be considered fixed. On the basis of the average part model, the spatial shape information is subjected to scale change and direction adjustment, the adjusted spatial shape information is projected to a plane based on the projection matrix, an initial plane image is obtained, the position of the initial plane image is adjusted based on the position offset information, and plane projection information is obtained, so that the position of the key part object obtained after projection is close to the position of the key part object in the image sample. Optionally, the computer device can add spatial texture information to the initial plane image. Further, the computer device can use the lighting parameter to simulate light rendering on the plane projection information to obtain projection coordinate information.

[0143] Step S507: generating a first loss function according to the plane coordinate information and the projection coordinate information.

[0144] In the embodiments of the present application, the computer device can generate a first loss function according to the difference information of the plane coordinate information and the projection coordinate information. The difference information can be used to represent the square of the difference, the average of the difference, or the cross entropy between the plane coordinate information and the projection coordinate information, which is not limited herein. Optionally, the computer device can generate the first loss function according to the L2 loss function between the plane coordinate information and the projection coordinate information.

[0145] Step S508: generating a second loss function according to the spatial part information.

[0146] In the embodiments of the present application, the computer device can generate a second loss function according to the spatial part information, and the number of the second loss function is one or at least two. Optionally, the second loss function includes an L2 loss function; when generating the second loss function according to the spatial part information, the computer device can identify the spatial feature points of the key part object in the image sample in the initial spatial part reconstruction model, obtain the ground truth feature points corresponding to the image sample; and determine the L2 loss function according to the difference information between the spatial feature points and the ground truth feature points. Optionally, the computer device can determine the L2 loss function according to the difference information between the shape data corresponding to the spatial feature points and the shape data corresponding to the ground truth feature points, wherein the shape data corresponding to the spatial feature points is determined according to the spatial shape information; or the computer device can determine the L2 loss function according to the difference information between the texture data corresponding to the spatial feature points and the texture data corresponding to the ground truth feature points, wherein the texture data corresponding to the spatial feature points can be determined according to the spatial texture information; or the computer device can determine the L2 loss function according to the difference information between the shape data corresponding to the spatial feature points and the shape data corresponding to the ground truth feature points, and the difference information between the texture data corresponding to the spatial feature points and the texture data corresponding to the ground truth feature points, without limitation herein.

[0147] Optionally, the second loss function includes a pixel loss function. The computer device can perform re-rendering processing on the spatial part information by using a differentiable renderer to obtain a re-rendered image; obtain pixel difference information between re-rendered pixels of the re-rendered image and sample pixels of the image sample; identify skin detection data of the image sample, and determine the pixel loss function based on a dot product result between the pixel difference information and the skin detection data. The pixel loss function can be seen from formula (6) as follows:

[0148]

[0149] In formula (6), the re-rendered image is denoted as I', the image sample is denoted as I, the dot product processing is denoted as, and the skin detection data is denoted as Sk. The computer device can obtain the pixel difference information between the re-rendered pixels of the re-rendered image I' and the sample pixels of the image sample I, identify the skin detection data Sk of the image sample I, and can take the skin detection data Sk as the weight of the pixel difference information, perform dot product processing on the pixel difference information and the skin detection data to obtain a pixel loss, and optionally, the pixel loss can be integrated to generate a pixel loss function. Wherein, ~ is used to represent that the integration processing can be mean value processing, summation processing or square difference processing, without limitation herein.

[0150] Optionally, the pixel loss function can also be seen from formula (7) as follows:

[0151]

[0152] In the formula (VI), H is used to represent a region in which a key part object is located in a re-rendered image, ||…|| is used to represent an L2 norm, the computer device can obtain a jth L2 norm between a jth sample pixel point of the image sample and a jth re-rendered pixel point in the re-rendered image, j is a positive integer, obtain a jth skin point data in the skin detection data Sk, weight the jth L2 norm by the jth skin point data, obtain a jth pixel difference value, sum all the pixel difference values to obtain a rendering difference value, obtain a sum of the skin point data included in the skin detection data, and determine a ratio between the rendering difference value and the sum of the skin point data included in the skin detection data as the pixel loss function.

[0153] Optionally, the second loss function can include a visual loss function or a feature loss function, etc., which is not limited herein. For example, the computer device can obtain sample features corresponding to the image sample, obtain re-rendered features corresponding to the re-rendered image, and generate a feature loss function according to a cosine distance between the sample features and the re-rendered features.

[0154] The second loss function can be any one or at least two of the loss functions mentioned above.

[0155] In step S509, the model is trained based on the first loss function and the second loss function to generate a spatial part reconstruction model.

[0156] In the embodiments of the present application, the computer device can train the model based on the first loss function and the second loss function to generate a spatial part reconstruction model. Alternatively, the computer device can adjust the spatial part parameters based on the first loss function and the second loss function to obtain optimized spatial part information, that is, to optimize the spatial reconstruction of the key part object, so that the spatial reconstruction result of the key part object is optimized in three-dimensional structure and two-dimensional contour, thereby improving the accuracy of spatial reconstruction.

[0157] Further, the computer device can obtain a first material image, identify a first key part object in the first material image through the spatial part reconstruction model, perform re-rendering processing on the first key part object to obtain a planar key part image corresponding to the first key part object, identify a target key part object in a target image, adjust the target key part object in the target image based on the planar key part image to obtain an updated key part object, and generate a second material image based on the updated key part object; a similarity between a planar contour of the updated key part object and a planar contour of the planar key part image is greater than or equal to a contour similarity threshold.

[0158] For example, see Figure 9 , Figure 9 is a part reconstruction scene schematic diagram provided by an embodiment of the present application. As shown in Figure 9 , the computer device can acquire a first material image 901, wherein the first material image 901 can be deleted or updated, etc. The computer device can identify a first key part object in the first material image, perform re-rendering processing on the first key part object, and obtain a planar key part image corresponding to the first key part object. A target key part object in a target image 902 is identified, the target key part object in the target image 902 is adjusted based on the planar key part image, an updated key part object is obtained, a second material image 903 is generated based on the updated key part object, wherein the second material image 903 can retain shape, texture and pose information, etc. of the first key part object in the first material image 901, and can also retain contour information of the first key part object in the first material image 901, improving the accuracy of image processing. Optionally, the number of the first material image 901 can be one or at least two, the computer device responds to an update request for the target key part object in the target image, updates the target key part object based on the first material image in the image selected state, and generates a second material image; if there is no first material image in the image selected state, a second material image corresponding to each first material image can be generated, and the second material image is displayed.

[0159] In the present application, each image (such as image samples, first material images and target images, etc.) is used after obtaining the copyright of the image.

[0160] In the embodiment of the present application, an image sample is obtained, planar coordinate information of a key part object in the image sample is identified, and spatial part information of the key part object is identified. In an initial spatial part reconstruction model, spatial contour coordinate information of the spatial part information is identified, the spatial part information is rasterized to obtain projection camera parameters, the spatial contour coordinate information is planar projected based on the projection camera parameters to obtain projection coordinate information. A first loss function is generated according to the planar coordinate information and the projection coordinate information, and a second loss function is generated according to the spatial part information. The initial spatial part reconstruction model is adjusted in parameters based on the first loss function and the second loss function to obtain a spatial part reconstruction model. The spatial part reconstruction model is used for re-rendering processing of a target key part object in a target image. Through the above process, when the image sample is spatially reconstructed, the projection coordinate information of the key part object in the image sample obtained after re-rendering is compared with the planar coordinate information of the key part object in the image sample directly identified, and the spatial part reconstruction model is adjusted in parameters according to the comparison result. The projection coordinate information can be used to represent the contour information of the key part object in the image sample after spatial reconstruction and planar re-rendering, and the planar coordinate information is used to represent the planar contour of the key part object in the image sample. Therefore, the contour of the model after spatial reconstruction of the key part object can be constrained to the planar contour of the key part object as much as possible, so that the contour after re-rendering of the model after spatial reconstruction can be as close to the original planar contour as possible, thereby reducing the ambiguity between the 2D contour on the image level and the 3D contour of the three-dimensional structure, and effectively improving the spatial reconstruction accuracy of the key part object and the accuracy of image processing.

[0161] Further, please refer to Figure 10 , Figure 10 is a schematic diagram of an image processing device provided by an embodiment of the present application. The image processing device can be a computer program (including program code, etc.) running in a computer device, for example, the image processing device can be an application software; the device can be used to execute the corresponding steps in the method provided by the embodiments of the present application. As shown in Figure 10 , the image processing device 1000 can be used in the computer device in the embodiments corresponding to Figure 3 , specifically, the device can include: a sample acquisition module 11, a planar coordinate identification module 12, a part identification module 13, a spatial coordinate identification module 14, an image projection module 15, a first loss generation module 16, a second loss generation module 17, and a model adjustment module 18.

[0162] The sample acquisition module 11 is configured to acquire an image sample.

[0163] The planar coordinate identification module 12 is configured to identify planar coordinate information of a key part object in the image sample.

[0164] a part recognition module 13, configured to recognize spatial part information of a key part object;

[0165] a spatial coordinate recognition module 14, configured to recognize spatial contour coordinate information of the spatial part information in an initial spatial part reconstruction model;

[0166] an image projection module 15, configured to rasterize the spatial part information to obtain projection camera parameters, perform planar projection processing on the spatial contour coordinate information based on the projection camera parameters, and obtain projection coordinate information;

[0167] a first loss generation module 16, configured to generate a first loss function according to the planar coordinate information and the projection coordinate information;

[0168] a second loss generation module 17, configured to generate a second loss function according to the spatial part information;

[0169] a model adjustment module 18, configured to perform parameter adjustment on the initial spatial part reconstruction model based on the first loss function and the second loss function, to obtain a spatial part reconstruction model; the spatial part reconstruction model is used for performing re-rendering processing on a target key part object in a target image.

[0170] The planar coordinate recognition module 12 comprises:

[0171] a planar analysis unit 121, configured to recognize planar part features of the key part object in the image sample, analyze the planar part features, and obtain planar contour points corresponding to the key part object;

[0172] a planar mask recognition unit 122, configured to recognize a planar mask image of the key part object from the image sample according to the planar contour points;

[0173] a planar coordinate acquisition unit 123, configured to acquire planar coordinate information of the key part object from the planar mask image.

[0174] The planar mask recognition unit 122 comprises:

[0175] a contour connection sub-unit 1221, configured to connect the planar contour points to obtain a planar contour line, and generate an initial mask image based on the image sample;

[0176] a pixel updating sub-unit 1222, configured to update a value of a first pixel point located in a first region indicated by the planar contour line in the initial mask image to a first default pixel value, and update a value of a second pixel point located in a second region indicated by the planar contour line to a second default pixel value; the first region refers to a region surrounded by the planar contour line, and the second region refers to a region other than the first region in the initial mask image.

[0177] The plane mask determination sub-unit 1223 is configured to determine the updated initial mask image as a plane mask image of the key part object.

[0178] The plane coordinate acquisition unit 123 comprises:

[0179] The image scanning sub-unit 1231 is configured to perform line-by-line scanning on the plane mask image to obtain pixel values of mask pixel points included in each plane pixel row in the plane mask image; the plane mask image comprises d plane pixel rows, and d is a positive integer.

[0180] The representation determination sub-unit 1232 is configured to obtain target mask pixel points in the i th plane pixel row, whose pixel values belong to the contour pixel range, and determine a representation coordinate corresponding to the i th plane pixel row according to a plane pixel column in the i th plane pixel row where the target mask pixel points are located and the i th plane pixel row; i is a positive integer less than or equal to d.

[0181] The coordinate determination sub-unit 1233 is configured to determine the plane coordinate information of the key part object according to the representation coordinates corresponding to the d plane pixel rows respectively.

[0182] The space coordinate identification module 14 comprises:

[0183] The plane acquisition unit 141 is configured to identify a plane part feature of the key part object in the image sample, analyze the plane part feature, and obtain a plane contour point corresponding to the key part object.

[0184] The trapezoid generation unit 142 is configured to obtain a starting contour point and a terminal contour point of the plane contour point, and generate a trapezoidal mask image corresponding to the image sample according to the starting contour point, the terminal contour point, a first image vertex and a second image vertex of the image sample.

[0185] The rendering acquisition unit 143 is configured to obtain a rendering surface mask image of the space part information in the initial space part reconstruction model.

[0186] The space determination unit 144 is configured to perform merging processing on the trapezoidal mask image and the rendering surface mask image to obtain a space contour line of the key part object, and determine space contour coordinate information corresponding to the space part information based on the space contour line.

[0187] The image projection module 15 comprises:

[0188] The patch acquisition unit 151 is configured to perform rasterization on the space part information by using a differentiable renderer to obtain patch information of each space pixel point in the space part information.

[0189] The parameter generation unit 152 is configured to generate projection camera parameters corresponding to the patch information by using spherical harmonics.

[0190] The projection determination unit 153 is configured to perform planar projection processing on the spatial part information based on the projection camera parameters to obtain projected coordinate information.

[0191] The part recognition module 13 comprises:

[0192] The parameter acquisition unit 131 is configured to acquire spatial part parameters of a key part object in an initial spatial part reconstruction model.

[0193] The average acquisition unit 132 is configured to acquire an average part model.

[0194] The spatial adjustment unit 133 is configured to add the spatial part parameters to the average part model to obtain spatial part information of the key part object.

[0195] The spatial part parameters comprise shape parameters.

[0196] The spatial adjustment unit 133 comprises:

[0197] The shape acquisition sub-unit 1331 is configured to acquire identity parameters and part attribute parameters in the shape parameters, and identify identity basis vectors and part attribute basis vectors between the key part object and the average part model.

[0198] The shape offset sub-unit 1332 is configured to generate identity offset information according to the identity parameters and the identity basis vectors, and generate part attribute offset information according to the part attribute parameters and the part attribute basis vectors.

[0199] The shape generation sub-unit 1333 is configured to acquire an average shape part model in the average part model, add the identity offset information and the part attribute offset information to the average shape part model, and generate spatial shape information of the key part object.

[0200] The information determination sub-unit 1334 is configured to determine spatial part information of the key part object according to the spatial shape information.

[0201] The projection camera parameters comprise spatial part parameters and illumination parameters.

[0202] When performing planar projection processing on the spatial contour coordinate information based on the projection camera parameters to obtain the projected coordinate information, the image projection module 15 comprises:

[0203] The scale recognition unit 154 is configured to identify scale change information between the key part object and the spatial shape information, and acquire a projection matrix; the projection matrix is a matrix used for projecting the key part object from space to a plane.

[0204] The direction obtaining unit 155 is configured to identify part direction information of the key part object, and obtain direction offset information between the part direction information and reference direction information; the reference direction information refers to direction information of the key part object indicated by the average part model;

[0205] The plane generating unit 156 is configured to determine an initial plane image of the key part object according to the spatial shape information, the scale change information, the projection matrix, and the direction offset information.

[0206] The position offset unit 157 is configured to identify position offset information between the key part object and a reference position point in the image sample, and add the position offset information in the initial plane image to obtain plane projection information; the reference position point refers to a reference pixel point in the image sample.

[0207] The light rendering unit 158 is configured to perform simulated light rendering on the plane projection information by using light parameters to obtain projection coordinate information.

[0208] The spatial part parameter includes a texture parameter.

[0209] The spatial adjustment unit 133 includes:

[0210] The texture offset sub-unit 1335 is configured to obtain texture base data, and determine a product of the texture base data and the texture parameter as texture offset information.

[0211] The texture generating sub-unit 1336 is configured to obtain average texture information in the average part model, and add the texture offset information to the average texture information to obtain spatial texture information.

[0212] The information determining sub-unit 1334 is further configured to determine spatial part information of the key part object according to the spatial texture information.

[0213] The second loss function includes an L2 loss function.

[0214] The second loss generating module 17 includes:

[0215] The feature point obtaining unit 171 is configured to identify spatial feature points of the key part object in the image sample in the initial spatial part reconstruction model, and obtain true value feature points corresponding to the image sample.

[0216] The first loss generating unit 172 is configured to determine the L2 loss function according to difference information between the spatial feature points and the true value feature points.

[0217] The second loss function includes a pixel loss function.

[0218] The second loss generating module 17 includes:

[0219] a re-rendering unit 173, configured to perform re-rendering processing on the spatial part information by using a differentiable renderer to obtain a re-rendered image;

[0220] a pixel difference unit 174, configured to obtain pixel difference information between a re-rendered pixel point of the re-rendered image and a sample pixel point of the image sample;

[0221] a second loss generation unit 175, configured to identify skin detection data of the image sample, and determine a pixel loss function based on a dot product result between the pixel difference information and the skin detection data.

[0222] The apparatus 1000 further includes:

[0223] a material rendering module 19, configured to obtain a first material image, identify a first key part object in the first material image by using a spatial part reconstruction model, perform re-rendering processing on the first key part object to obtain a planar key part image corresponding to the first key part object;

[0224] a target adjustment module 20, configured to identify a target key part object in a target image, adjust the target key part object in the target image based on the planar key part image to obtain an updated key part object, and generate a second material image based on the updated key part object; a similarity between a planar contour of the updated key part object and a planar contour of the planar key part image is greater than or equal to a contour similarity threshold.

[0225] The image processing apparatus provided in the embodiments of the present application can compare projection coordinate information obtained after re-rendering a key part object in an image sample with planar coordinate information directly identified from the key part object in the image sample, and adjust parameters of a spatial part reconstruction model according to a comparison result. The projection coordinate information can be used to represent contour information of the key part object in the image sample after spatial reconstruction and planar re-rendering, and the planar coordinate information can be used to represent a planar contour of the key part object in the image sample. Therefore, the contour of the model after spatial reconstruction of the key part object can be constrained by the planar contour of the key part object, so that the contour of the model after re-rendering can be as close as possible to the original planar contour, thereby reducing ambiguity between a 2D contour on an image level and a 3D contour of a three-dimensional structure, and effectively improving spatial reconstruction accuracy of the key part object and image processing accuracy.

[0226] Referring to Figure 11 , Figure 11 is a structural schematic diagram of a computer device provided in the embodiments of the present application. As shown in Figure 11As shown, the computer device in the embodiment of the present application can include one or more processors 1101, a memory 1102 and an input output interface 1103. The processor 1101, the memory 1102 and the input output interface 1103 are connected through a bus 1104. The memory 1102 is used to store a computer program, the computer program including program instructions, the input output interface 1103 is used to receive data and output data, such as for data interaction between the computer device and the target device; the processor 1101 is used to execute the program instructions stored in the memory 1102.

[0227] The processor 1101 can perform the following operations:

[0228] Obtain an image sample, identify planar coordinate information of a key part object in the image sample, and identify spatial part information of the key part object.

[0229] In the initial spatial part reconstruction model, identify spatial contour coordinate information of the spatial part information, rasterize the spatial part information, obtain projection camera parameters, perform planar projection processing on the spatial contour coordinate information based on the projection camera parameters, and obtain projection coordinate information.

[0230] Generate a first loss function based on the planar coordinate information and the projection coordinate information, and generate a second loss function based on the spatial part information.

[0231] Adjust the initial spatial part reconstruction model based on the first loss function and the second loss function to obtain a spatial part reconstruction model; the spatial part reconstruction model is used for re-rendering processing of a target key part object in a target image.

[0232] In some possible implementations, the processor 1101 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0233] The memory 1102 may include read-only memory and random access memory, and provides instructions and data to the processor 1101 and the input / output interface 1103. A portion of the memory 1102 may also include non-volatile random access memory. For example, the memory 1102 may also store device type information.

[0234] In practice, the computer device can perform actions such as these through its built-in functional modules. Figure 3 For details on the implementation methods provided for each step, please refer to [the relevant documentation / document / etc.]. Figure 3 The implementation methods provided for each step are not elaborated here.

[0235] This application provides a computer device including a processor, an input / output interface, and a memory. The processor retrieves a computer program from the memory and executes it. Figure 3 Each step of the method shown involves image processing operations. This embodiment of the application acquires image samples, identifies the planar coordinate information of key objects in the image samples, and identifies the spatial part information of the key objects. In the initial spatial part reconstruction model, the spatial contour coordinate information of the spatial part information is identified, and the spatial part information is rasterized to obtain projection camera parameters. Based on the projection camera parameters, the spatial contour coordinate information is subjected to planar projection processing to obtain projection coordinate information. A first loss function is generated based on the planar coordinate information and the projection coordinate information, and a second loss function is generated based on the spatial part information. The parameters of the initial spatial part reconstruction model are adjusted based on the first loss function and the second loss function to obtain the spatial part reconstruction model. The spatial part reconstruction model is used to re-render the target key objects in the target image. Through the above process, when reconstructing the spatial structure of an image sample, the projected coordinate information obtained after re-rendering the key parts of the image sample is compared with the planar coordinate information obtained by directly identifying the key parts of the image sample. Based on the comparison results, the parameters of the spatial reconstruction model are adjusted. The projected coordinate information can be used to represent the contour information of the key parts of the image sample after spatial reconstruction and planar re-rendering. The planar coordinate information is used to represent the planar contour of the key parts of the image sample, so that the contour of the model after spatial reconstruction of the key parts can be constrained with the planar contour of the key parts, so that the contour of the model after re-rendering can be as close as possible to the original planar contour, thereby reducing the ambiguity between the 2D contour of the image layer and the 3D contour of the 3D structure, effectively improving the spatial reconstruction accuracy of the key parts and the accuracy of image processing.

[0236] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is suitable for being loaded and executed by the processor Figure 3 The image processing method provided by each step in the embodiment can refer to the implementation manner provided by each step in the embodiment Figure 3 The implementation manner provided by each step in the embodiment can refer to the implementation manner provided by each step in the embodiment, which will not be described herein. In addition, the beneficial effect of using the same method will not be described herein. For the technical details not disclosed in the computer readable storage medium embodiment of the present application, please refer to the description of the method embodiment of the present application. As an example, the computer program can be deployed to be executed on one computer device, or executed on multiple computer devices located in one place, or executed on multiple computer devices distributed in multiple places and interconnected through a communication network.

[0237] The computer readable storage medium can be the internal storage unit of the image processing apparatus or the computer device, for example, the hard disk or the memory of the computer device. The computer readable storage medium can also be the external storage device of the computer device, for example, the plug-in hard disk, the smart media card (SMC), the secure digital (SD) card, the flash card and the like. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0238] The embodiment of the present application further provides a computer program product or computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 3The method provided in various optional manners in the image sample is reconstructed in space, the projection coordinate information of the key part object in the image sample after being re-rendered is compared with the plane coordinate information directly identified from the key part object of the image sample, the parameter of the spatial part reconstruction model is adjusted according to the comparison result, the projection coordinate information can be used to represent the contour information of the key part object in the image sample after being reconstructed in space and re-rendered in plane, and the plane coordinate information is used to represent the plane contour of the key part object in the image sample, so that the contour of the model after being reconstructed in space can be constrained with the plane contour of the key part object, so that the contour after being re-rendered can be as close as possible to the original plane contour, thereby reducing the ambiguity between the 2D contour in the image and the 3D contour of the three-dimensional structure, and effectively improving the spatial reconstruction accuracy of the key part object and the accuracy of image processing.

[0239] The terms "first", "second", and the like in the specification and claims of the present application and the drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, device, product or equipment.

[0240] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0241] The method and related device provided by the embodiments of the present application are described with reference to the method flowchart and / or structural schematic diagram provided by the embodiments of the present application, and each flow and / or block of the method flowchart and / or structural schematic diagram and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable image processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable image processing devices produce a method for implementing the functions described in the flowchart and / or block diagram. Figure 1flowchart or flowcharts and / or a diagram of a structure Figure 1 apparatus that performs the function specified in a flowchart or flowchart Figure 1 flowchart or flowcharts and / or a diagram of a structure Figure 1 apparatus that performs the function specified in a flowchart or flowchart Figure 1 flowchart or flowcharts and / or a diagram of a structure

[0242] The steps in the method of the embodiments of the present application can be adjusted in sequence, combined and deleted according to actual needs.

[0243] The modules in the device of the embodiments of the present application can be combined, divided and deleted according to actual needs.

[0244] The above only discloses the preferred embodiments of the present application, and of course cannot limit the scope of the right of the present application, so the equivalent changes made according to the claims of the present application still belong to the scope covered by the present application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring an image sample, identifying planar coordinate information of a key part object in the image sample, and identifying spatial part information of the key part object; the key part object refers to a part object having a spatial structure and capable of being displayed in a plane; in an initial spatial part reconstruction model, identifying spatial contour coordinate information of the spatial part information, rasterizing the spatial part information to obtain projection camera parameters, performing planar projection processing on the spatial contour coordinate information based on the projection camera parameters, and obtaining projection coordinate information; generating a first loss function according to the planar coordinate information and the projection coordinate information, and generating a second loss function according to the spatial part information; based on the first loss function and the second loss function, adjusting parameters of the initial spatial part reconstruction model to obtain a spatial part reconstruction model; the spatial part reconstruction model is used for re-rendering processing of a target key part object in a target image.

2. The method of claim 1, wherein, The identification of the planar coordinate information of the key part object in the image sample comprises: identifying a planar part feature of the key part object in the image sample, analyzing the planar part feature to obtain a planar contour point corresponding to the key part object; from the image sample, identifying a planar mask image of the key part object according to the planar contour point; acquiring the planar coordinate information of the key part object from the planar mask image.

3. The method of claim 2, wherein, The identification of the planar mask image of the key part object from the image sample according to the planar contour point comprises: connecting the planar contour points to obtain a planar contour line, and generating an initial mask image based on the image sample; updating a value of a first pixel point located in a first region indicated by the planar contour line in the initial mask image to a first default pixel value, and updating a value of a second pixel point located in a second region indicated by the planar contour line to a second default pixel value; the first region refers to a region surrounded by the planar contour line, and the second region refers to a region other than the first region in the initial mask image; determining the updated initial mask image as the planar mask image of the key part object.

4. The method of claim 2, wherein, The acquisition of the planar coordinate information of the key part object from the planar mask image comprises: performing line-by-line scanning on the planar mask image to obtain pixel values of mask pixel points included in each planar pixel row of the planar mask image; the planar mask image comprises d planar pixel rows, and d is a positive integer; acquiring a target mask pixel point whose pixel value belongs to a contour pixel range in an i-th planar pixel row, determining a representative coordinate corresponding to the i-th planar pixel row according to a planar pixel column where the target mask pixel point is located in the i-th planar pixel row and the i-th planar pixel row; i is a positive integer less than or equal to d; determining the planar coordinate information of the key part object according to the representative coordinates corresponding to the d planar pixel rows, respectively.

5. The method of claim 1, wherein, The space contour coordinate information of the space site information is identified in the initial space site reconstruction model, and the space contour coordinate information includes: A planar site feature of a key site object in the image sample is identified, and the planar site feature is analyzed to obtain a planar contour point corresponding to the key site object; A starting contour point and an ending contour point of the planar contour point are obtained, and a trapezoidal mask image corresponding to the image sample is generated according to the starting contour point, the ending contour point, a first image vertex and a second image vertex of the image sample; In the initial space site reconstruction model, a rendering surface mask image of the space site information is obtained; The trapezoidal mask image and the rendering surface mask image are merged to obtain a space contour line of the key site object, and space contour coordinate information corresponding to the space site information is determined based on the space contour line.

6. The method of claim 1, wherein, The space site information is rasterized to obtain a projection camera parameter, and the space contour coordinate information is planarly projected based on the projection camera parameter to obtain projection coordinate information, and the method includes: A differentiable renderer is used to rasterize the space site information to obtain patch information of each space pixel point in the space site information; A spherical harmonic function is used to generate a projection camera parameter corresponding to the patch information; The space site information is planarly projected based on the projection camera parameter to obtain projection coordinate information after projection.

7. The method of claim 1, wherein, The space site information of the key site object is identified, and the method includes: In the initial space site reconstruction model, a space site parameter of the key site object is obtained; An average site model is obtained, and the space site parameter is added to the average site model to obtain the space site information of the key site object.

8. The method of claim 7, wherein, The space site parameter includes a shape parameter; The space site parameter is added to the average site model to obtain the space site information of the key site object, and the method includes: An identity parameter and a site attribute parameter in the shape parameter are obtained, and an identity basis vector and a site attribute basis vector between the key site object and the average site model are identified; Identity offset information is generated according to the identity parameter and the identity basis vector, and site attribute offset information is generated according to the site attribute parameter and the site attribute basis vector; An average shape site model in the average site model is obtained, and the identity offset information and the site attribute offset information are added to the average shape site model to generate space shape information of the key site object; The space site information of the key site object is determined according to the space shape information.

9. The method of claim 8, wherein, The projection camera parameter includes the space site parameter and an illumination parameter; The space contour coordinate information is planarly projected based on the projection camera parameter to obtain projection coordinate information, and the method includes: Scale change information between the key site object and the space shape information is identified, and a projection matrix is obtained; the projection matrix is a matrix used for projecting the key site object from space to plane. The position direction information of the key part object is identified, and direction offset information between the position direction information and reference direction information is obtained; the reference direction information refers to direction information of the key part object indicated by the average part model; According to the spatial shape information, the scale change information, the projection matrix and the direction offset information, an initial planar image of the key part object is determined; Position offset information between the key part object and a reference position point in the image sample is identified, and the position offset information is added to the initial planar image to obtain planar projection information; the reference position point refers to a reference pixel point in the image sample; The planar projection information is simulated and rendered by using the illumination parameter to obtain projection coordinate information.

10. The method of claim 7, wherein, The spatial part parameter includes a texture parameter; The spatial part parameter is added to the average part model to obtain spatial part information of the key part object, including: Texture base data is obtained, and a product of the texture base data and the texture parameter is determined as texture offset information; Average texture information in the average part model is obtained, and the texture offset information is added to the average texture information to obtain spatial texture information; According to the spatial texture information, spatial part information of the key part object is determined.

11. The method of claim 1, wherein, The second loss function includes an L2 loss function; The second loss function is generated according to the spatial part information, including: In the initial spatial part reconstruction model, spatial feature points of the key part object in the image sample are identified, and true value feature points corresponding to the image sample are obtained; According to difference information between the spatial feature points and the true value feature points, the L2 loss function is determined.

12. The method of claim 1, wherein, The second loss function includes a pixel loss function; The second loss function is generated according to the spatial part information, including: A differentiable renderer is used to perform re-rendering processing on the spatial part information to obtain a re-rendered image; Pixel difference value information between re-rendered pixel points of the re-rendered image and sample pixel points of the image sample is obtained; Skin detection data of the image sample is identified, and a pixel loss function is determined based on a dot product result between the pixel difference value information and the skin detection data.

13. The method of claim 1, wherein, The method further includes: A first material image is obtained, a first key part object in the first material image is identified by the spatial part reconstruction model, and the first key part object is re-rendered to obtain a planar key part image corresponding to the first key part object; A target key part object in the target image is identified, the target key part object in the target image is adjusted based on the planar key part image to obtain an updated key part object, and a second material image is generated based on the updated key part object; a similarity between a planar contour of the updated key part object and a planar contour of the planar key part image is greater than or equal to a contour similarity threshold.

14. An image processing apparatus characterized by comprising: The device includes: A sample acquisition module is configured to acquire an image sample; A plane coordinate identification module is configured to identify plane coordinate information of a key part object in the image sample. The key part object refers to a part object having a spatial structure and capable of being displayed in a plane. A part identification module is configured to identify spatial part information of the key part object. An image projection module is configured to, in an initial spatial part reconstruction model, identify spatial contour coordinate information of the spatial part information, perform rasterization processing on the spatial part information to obtain projection camera parameters, perform plane projection processing on the spatial contour coordinate information based on the projection camera parameters, and obtain projection coordinate information. A first loss generation module is configured to generate a first loss function based on the plane coordinate information and the projection coordinate information. A second loss generation module is configured to generate a second loss function based on the spatial part information. A model adjustment module is configured to perform parameter adjustment on the initial spatial part reconstruction model based on the first loss function and the second loss function to obtain a spatial part reconstruction model. The spatial part reconstruction model is configured to perform re-rendering processing on a target key part object in a target image.

15. A computer device, comprising: A computer device comprises a processor, a memory, and an input / output interface. The processor is connected to the memory and the input / output interface. The input / output interface is configured to receive and output data. The memory is configured to store a computer program. The processor is configured to call the computer program to enable the computer device to perform the method in any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program. The computer program is adapted to be loaded and executed by the processor to enable the computer device having the processor to perform the method in any one of claims 1-13.

17. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method in any one of claims 1-13.

Images