Limb rendering method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a limb rendering method and system, electronic equipment and a storage medium, and belongs to the technical field of three-dimensional reconstruction. The method comprises the following steps: firstly, acquiring limb movement data of a target object and acquisition parameters of the limb movement data, and inputting the limb movement data and the acquisition parameters into a preset rendering model for limb rendering; the rendering model comprises a limb grid sub-model and a neural network. And carrying out code conversion on the limb movement data and the acquisition parameters through the limb grid sub-model to obtain deformable code information, and carrying out feature parameter calculation on the deformable code information through a neural network to obtain spatial feature parameters of each spatial sampling point. The spatial characteristic parameters comprise sampling point color information and sampling point density information. And finally, performing rendering processing on the sampling point color information and the sampling point density information to obtain a target limb rendering image. Through the method, the limb rendering quality and the limb rendering efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction technology, and in particular to a limb rendering method and system, an electronic device and a storage medium. Background Art

[0002] Limb rendering refers to generating limb images from various perspectives under a specified limb action. Realistic limb rendering has important applications in many fields, such as human-computer interaction, virtual reality, games, etc. In related technologies, limb rendering methods usually first perform upsampling on a parameterized limb mesh model to obtain a high-resolution parameterized limb mesh model. Then, based on the high-resolution parameterized limb mesh model, the geometric information and appearance information of the limb are modeled separately. Among them, the geometric information of the modeled limb is realized by the following steps: the vertices of the sampled high-resolution limb mesh model and the spatial point to be queried are input into the neural network together, and then feature processing is performed through the neural network to output the occupancy value of this spatial point; the appearance information of the modeled limb is realized by the following steps: decoupling the surface appearance and illumination information of the limb, predicting a color information from the surface position information of the limb and then multiplying it by the illumination information predicted from the current position to obtain the final color information of this spatial point.

[0003] However, the limb rendering method in the related art needs to calculate the color information of all spatial points of the limbs in multiple times before the rendering can be completed, which makes the rendering process complicated. The above shortcomings lead to low efficiency of limb rendering in the existing limb rendering method. Therefore, how to improve the efficiency of limb rendering has become a technical problem to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a body rendering method and system, an electronic device and a storage medium, aiming to improve the efficiency of body rendering.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a limb rendering method, the method comprising:

[0006] Acquire the limb motion data of the target object and the acquisition parameters of the limb motion data; wherein the acquisition parameters are the operating parameters of the acquisition device for acquiring the limb motion data;

[0007] Inputting the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network;

[0008] The limb motion data and the acquisition parameters are encoded and converted by the limb mesh sub-model to obtain deformable encoding information; wherein the deformable encoding information represents the spatial characteristics of each spatial sampling point on the limb of the target object in three-dimensional space;

[0009] The deformable coding information is subjected to feature parameter calculation by the neural network to obtain the spatial feature parameters of each of the spatial sampling points; wherein the spatial feature parameters include: sampling point color information and sampling point density information;

[0010] The sampling point color information and the sampling point density information are rendered to obtain a target limb rendering image.

[0011] In some embodiments, the acquisition parameters include: acquisition position information and acquisition internal parameters; the limb mesh sub-model includes: a sampling network and a limb mesh construction network;

[0012] The encoding and conversion of the limb motion data and the acquisition parameters by the limb mesh sub-model to obtain deformable encoding information includes:

[0013] A plane is constructed by using the sampling network, the acquisition position information and the acquisition internal parameters to obtain an imaging plane image; wherein the imaging plane image includes a plurality of plane pixels;

[0014] Back-projecting each of the plane pixels into the three-dimensional space through the sampling network and the preset mapping relationship to obtain a virtual light corresponding to each of the plane pixels;

[0015] Performing equidistant sampling on each of the virtual rays through the sampling network to obtain a plurality of spatial sampling points;

[0016] Acquire the position information of each of the spatial sampling points through the sampling network to obtain the spatial sampling point position information;

[0017] Performing limb mesh construction on the limb motion data through the limb mesh construction network to obtain a limb parameterized mesh;

[0018] The deformable coding information is obtained by performing coding conversion according to the limb parameterized grid, the limb motion data and the spatial sampling point position information.

[0019] In some embodiments, the limb parameterized mesh is composed of a plurality of triangular facets, and the encoding conversion is performed according to the limb parameterized mesh, the limb motion data and the spatial sampling point position information to obtain the deformable encoding information, including:

[0020] Determining the motion parameter characteristics according to the limb motion data;

[0021] Acquire vertex space position information and vertex map position information corresponding to each vertex of the triangular facet according to the limb parameterized mesh;

[0022] Determine a target triangular patch according to the vertex spatial position information, the spatial sampling point position information and a preset target triangular patch selection rule; wherein the spatial position information of the target triangular patch is represented by the target vertex spatial position information;

[0023] Determine a plane normal feature according to the target triangular facet;

[0024] According to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature, the target sampling point is projected onto the target triangle facet to obtain the plane projection distance feature and the spatial projection point; wherein the target sampling point is selected based on the spatial sampling point;

[0025] Acquire the position information of the spatial projection point to obtain the spatial position information of the projection point;

[0026] Acquire the mapping features of the triangular face according to the target vertex spatial position information, the vertex mapping position information and the projection point spatial position information to obtain mapping coordinate features;

[0027] Determining a feature value on a preset feature map according to the map coordinate feature;

[0028] Determine an observation direction feature according to the spatial sampling point position information and the virtual light; wherein the observation direction feature is used to describe the direction of the virtual light where the spatial sampling point is located;

[0029] The action parameter feature, the plane normal feature, the plane projection distance feature, the mapping coordinate feature, the feature value and the observation direction feature are spliced ​​to obtain the deformable coding information.

[0030] In some embodiments, before projecting the target sampling point to the target triangle patch according to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature to obtain the plane projection distance feature and the spatial projection point, the method further includes:

[0031] Selecting the target sampling point from the spatial sampling points specifically includes:

[0032] Compare the preset distance threshold with the plane projection distance feature to obtain a comparison result;

[0033] The spatial sampling points are filtered according to the comparison result to obtain target sampling points.

[0034] In some embodiments, rendering the sampling point color information and the sampling point density information to obtain a target limb rendering image includes:

[0035] Determine a weight coefficient according to the sampling point density information and a preset weight mapping relationship table; wherein the weight mapping relationship table includes the weight coefficient corresponding to each sampling point density information;

[0036] According to the weight coefficient, the opacity information of the sampling point and the virtual light, weighted summation is performed on the color information of the sampling point to obtain the color information of the virtual light; wherein the opacity information of the sampling point is determined according to the density information of the sampling point;

[0037] The target limb rendering image is constructed according to the virtual light color information, the virtual light, the mapping relationship and the plane pixels.

[0038] In some embodiments, the collecting internal parameters include color channel parameters, and after the rendering process is performed on the sampling point color information and the sampling point density information to obtain the target limb rendering image, the method further includes:

[0039] Performing deviation correction on the color channel parameters according to preset correction parameters to obtain corrected color channel parameters;

[0040] The target limb rendering image is corrected according to the correction color channel parameters to obtain a limb rendering correction image.

[0041] In some embodiments, after inputting the limb motion data and the acquisition parameters into a preset rendering model, the method further includes:

[0042] Optimizing the rendering model specifically includes:

[0043] Constructing a loss function based on a preset limb verification image and the target limb rendering image;

[0044] Performing back propagation calculation on the loss function to obtain the optimization parameter gradient;

[0045] Performing gradient descent update on preset optimizable parameters according to the optimization parameter gradient to obtain target optimizable parameters; wherein the optimizable parameters include the eigenvalues ​​and the correction parameters;

[0046] The rendering model is optimized according to the target optimizable parameters.

[0047] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a limb rendering system, the system comprising:

[0048] A parameter acquisition module, used to acquire the body movement data of the target object and acquisition parameters of the body movement data; wherein the acquisition parameters are operating parameters of an acquisition device for acquiring the body movement data;

[0049] A parameter input module, used to input the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network;

[0050] A coding conversion module, used for coding and converting the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable coding information; wherein the deformable coding information represents the spatial characteristics of each spatial sampling point on the limb of the target object in three-dimensional space;

[0051] A spatial feature parameter calculation module, used to calculate the feature parameters of the deformable coding information through the neural network to obtain the spatial feature parameters of each of the spatial sampling points; wherein the spatial feature parameters include: sampling point color information and sampling point density information;

[0052] The image rendering module is used to render the sampling point color information and the sampling point density information to obtain a target limb rendering image.

[0053] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0054] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0055] The limb rendering method and system, electronic device and storage medium proposed in the present application obtain the limb motion data of the target object and the acquisition parameters of the limb motion data, and input the limb motion data and the acquisition parameters into a preset rendering model for limb rendering. Among them, the rendering model includes: a limb grid sub-model and a neural network. Therefore, the limb motion data and the acquisition parameters are first encoded and converted by the limb grid sub-model to obtain deformable coding information, and then the deformable coding information is calculated by the neural network to obtain the spatial feature parameters of each spatial sampling point, wherein the spatial feature parameters include: sampling point color information and sampling point density information. Finally, the sampling point color information and the sampling point density information are rendered to obtain the target limb rendering image. Therefore, through the limb rendering method and system, electronic device and storage medium proposed in the present application, it is possible to directly use the parameterized mesh model to convert the limb motion data and acquisition parameters of the limb mesh sub-model into deformable coding information, and then calculate the spatial feature parameters of the spatial sampling points through the deformable coding information. After that, the image rendering is performed through volume rendering, so as to achieve fast rendering of the limb image. There is no need to model the parameterized limb mesh model multiple times to model the geometric information and appearance information of the limb separately, which simplifies the limb rendering steps, shortens the time required for manual modeling and rendering, and improves the limb rendering quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is an optional flow chart of the limb rendering method provided in an embodiment of the present application;

[0057] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0058] Figure 3 is another optional flow chart of the limb rendering method provided in an embodiment of the present application;

[0059] Figure 4 yes Figure 3 Flow chart of step S206 in FIG.

[0060] Figure 5 yes Figure 3 Flow chart of step S105 in FIG.

[0061] Figure 6 is another optional flow chart of the limb rendering method provided in an embodiment of the present application;

[0062] Figure 7 is another optional flow chart of the limb rendering method provided in an embodiment of the present application;

[0063] Figure 8It is an optional overall architecture diagram of the limb rendering method provided in the embodiment of the present application;

[0064] Fig. 9 is an optional structural diagram of a limb rendering system provided in an embodiment of the present application;

[0065] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0069] The embodiments of the present application provide a body rendering method and system, an electronic device, and a storage medium, aiming to improve the efficiency of body rendering.

[0070] The limb rendering method and system, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the limb rendering method in the embodiments of the present application is described.

[0071] The limb rendering method provided in the embodiment of the present application relates to the field of three-dimensional reconstruction technology. The limb rendering method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the limb rendering method, etc., but is not limited to the above forms.

[0072] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0073] Figure 1 is an optional flow chart of the limb rendering method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0074] Step S101, obtaining the body movement data of the target object and the acquisition parameters of the body movement data; wherein the acquisition parameters are the operating parameters of the acquisition device for acquiring the body movement data;

[0075] Step S102, inputting the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network;

[0076] Step S103, encoding and converting the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable coded information; wherein the deformable coded information represents the spatial characteristics of each spatial sampling point on the limb of the target object in the three-dimensional space;

[0077] Step S104, calculating characteristic parameters of the deformable coding information through a neural network to obtain spatial characteristic parameters of each spatial sampling point; wherein the spatial characteristic parameters include: sampling point color information and sampling point density information;

[0078] Step S105 , rendering the sampling point color information and the sampling point density information to obtain a target limb rendering image.

[0079] Steps S101 to S105 shown in the embodiment of the present application obtain the limb motion data of the target object and the acquisition parameters of the limb motion data, and input the limb motion data and the acquisition parameters into a preset rendering model for limb rendering. The rendering model includes: a limb mesh sub-model and a neural network. The rendering model first encodes and converts the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable coding information, and then calculates the feature parameters of the deformable coding information through the neural network to obtain the spatial feature parameters of each spatial sampling point, wherein the spatial feature parameters include: sampling point color information and sampling point density information. Finally, the sampling point color information and the sampling point density information are rendered to obtain a rendered image of the target limb. Therefore, the limb rendering method proposed in the present application can directly utilize the parameterized mesh model to convert the limb motion data and acquisition parameters of the limb mesh sub-model into deformable coding information, and then calculate the spatial feature parameters of the spatial sampling points through the deformable coding information. Finally, the image rendering is performed through volume rendering, so as to achieve fast rendering of limb images. There is no need to model the parameterized limb mesh model multiple times to model the geometric information and appearance information of the limb separately, which simplifies the limb rendering steps, shortens the time required for limb rendering, and improves the limb rendering quality and efficiency.

[0080] In step S101 of some embodiments, the limb motion data of the target object and the acquisition parameters of the limb motion data are obtained. Specifically, the limb motion data is data that can characterize the limb motion of the target object. The acquisition parameters are operating parameters of the acquisition device that acquires the limb motion data, and the acquisition parameters include at least one of the following: acquisition position information and acquisition internal parameters. For example, the limb motion data may be the motion data of a human hand, the acquisition device may be a virtual camera, the acquisition parameters are the operating parameters of the virtual camera, and the operating parameters of the virtual camera include at least one of the following: camera position parameters and camera internal parameters. Among them, the camera position parameters are determined by the camera origin position information, camera displacement information and camera rotation information of the virtual camera in three-dimensional space, and the camera internal parameters include the focal length information and optical center information of the camera during the projection process. The acquisition parameters may also be other operating parameters of the acquisition device, as long as the limb motion data required for the limb rendering model can be acquired. The specific selection may be based on the actual needs of those skilled in the art, and is not limited to the acquisition parameters described in the above examples.

[0081] See also Figure 2 In some embodiments, the limb mesh sub-model includes: a sampling network and a limb mesh construction network, and step S103 may include but is not limited to steps S201 to S206:

[0082] Step S201, constructing a plane by sampling a network, collecting position information, and collecting internal parameters to obtain an imaging plane image; wherein the imaging plane image includes a plurality of plane pixels;

[0083] Step S202, back-projecting each plane pixel into the three-dimensional space through a sampling network and a preset mapping relationship to obtain a virtual light corresponding to each plane pixel;

[0084] Step S203, sampling each virtual light ray at equal distances through a sampling network to obtain a plurality of spatial sampling points;

[0085] Step S204, obtaining the position information of each spatial sampling point through the sampling network to obtain the spatial sampling point position information;

[0086] Step S205, performing limb mesh construction on the limb motion data through a limb mesh construction network to obtain a limb parameterized mesh;

[0087] Step S206 , performing encoding conversion according to the limb parameterized grid, the limb motion data and the spatial sampling point position information to obtain deformable encoding information.

[0088] In step S201 of some embodiments, the acquisition position information is used to describe the position of the acquisition device in the three-dimensional space, and the acquisition internal parameters are used to describe the internal parameters required by the acquisition device for acquiring the position information. On this basis, a plane is constructed through the sampling network, the acquisition position information and the acquisition internal parameters to obtain an imaging plane image; wherein the imaging plane image includes a plurality of plane pixels. Specifically, the position of the acquisition device in the three-dimensional space is determined through the sampling network and the acquisition position information, and then the imaging plane is constructed in the three-dimensional space through the sampling network, the acquisition position information of the acquisition device and the acquisition internal parameters, and the imaging plane image is determined on the imaging plane. Taking a virtual camera as an example, the position coordinate information and projection direction information of the virtual camera in the three-dimensional space are obtained through the camera position parameters, and according to the position coordinate information, focal length information and optical center information of the virtual camera in the three-dimensional space, the plane passing through the projection focus and perpendicular to the projection direction is taken as the image plane, and then the imaging plane image is determined on the image plane according to the sampling network. Among them, the focal length information and the optical center information are the acquisition internal parameters of the virtual camera.

[0089] In step S202 of some embodiments, each plane pixel can be back-projected into three-dimensional space through a sampling network and a preset mapping relationship to obtain a virtual light ray corresponding to each plane pixel. Among them, the preset mapping relationship is used to represent the one-to-one correspondence between the plane pixels on the imaging plane image and the virtual light ray in the three-dimensional space. Taking a virtual camera as an example, taking the camera position information as the origin, and drawing a ray along the origin-plane pixel direction, the obtained ray will coincide with a virtual ray, and the direction of the ray is the direction of the virtual ray. On this basis, the camera position information and the plane pixel position information through which the virtual ray passes are obtained through the sampling network, and the virtual light position information can be determined. The preset mapping relationship may include: camera position information, plane pixel position information and virtual light position information, and the preset mapping relationship may also be a mapping relationship that can be obtained by other technical personnel in this field based on the prior art.

[0090] In step S203 of some embodiments, a plurality of spatial sampling points may be obtained by performing equidistant sampling on each virtual light ray through a sampling network. The spatial sampling points may be obtained by performing equidistant sampling on the virtual light ray, or by performing non-equidistant sampling on the virtual light ray according to specific rules, wherein the specific rules may be set by a person skilled in the art according to actual needs. The number of spatial sampling points may be 128, or may be a specific number set by other persons skilled in the art when actually performing spatial sampling.

[0091] In step S205 of some embodiments, the limb mesh construction network can be used to construct limb meshes for limb motion data to obtain limb parametric meshes. The limb mesh construction network is used to construct limb parametric meshes, and the limb parametric meshes can be head parametric meshes, hand parametric meshes, leg parametric meshes or other human body parametric model meshes, or animal limb parametric meshes. Taking the human hand parameter mesh as an example, the limb motion data of the human hand can be represented by 48 motion parameter features, and these 48 motion parameter features include three-dimensional rotation angle vector features of 16 joints, and the 16 joints include 3 joints of each finger and 1 joint of the palm. The changes in the three-dimensional space can determine the movement of the human hand based on these 48 motion features.

[0092] In step S206 of some embodiments, encoding conversion is performed based on the limb parameterized mesh, limb motion data and spatial sampling point position information to obtain deformable encoding information.

[0093] See also Figure 3 In some embodiments, step S206 may include but is not limited to steps S301 to S310:

[0094] Step S301, determining motion parameter characteristics according to the body motion data;

[0095] Step S302, obtaining vertex space position information and vertex map position information corresponding to each vertex of the triangular facet according to the limb parameterized mesh;

[0096] Step S303, determining the target triangle patch according to the vertex spatial position information, the spatial sampling point position information and the preset target triangle patch selection rule; wherein the spatial position information of the target triangle patch is represented by the target vertex spatial position information;

[0097] Step S304, determining the plane normal feature according to the target triangle facet;

[0098] Step S305, projecting the target sampling point to the target triangle patch according to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature, to obtain the plane projection distance feature and the spatial projection point; wherein the target sampling point is selected based on the spatial sampling point;

[0099] Step S306, acquiring the position information of the spatial projection point to obtain the spatial position information of the projection point;

[0100] Step S307, obtaining the mapping features of the triangle patch according to the target vertex spatial position information, the vertex mapping position information and the projection point spatial position information, and obtaining the mapping coordinate features;

[0101] Step S308, determining a feature value on a preset feature map according to the map coordinate feature;

[0102] Step S309, determining an observation direction feature according to the spatial sampling point position information and the virtual light; wherein the observation direction feature is used to describe the direction of the virtual light where the spatial sampling point is located;

[0103] Step S310, concatenating the action parameter features, the plane normal features, the plane projection distance features, the mapping coordinate features, the feature values ​​and the observation direction features to obtain deformable coding information.

[0104] In step S301 of some embodiments, the body motion data includes motion parameter features and motion parameter feature values, so the motion parameter features can be determined according to the body motion data. Taking the hand motion data of a person as an example, the body motion data of a person's hand can be represented by 48 motion parameter features, and these 48 motion parameter features respectively have corresponding motion parameter feature values.

[0105] In step S302 of some embodiments, the limb parameterized mesh is obtained by stitching together a plurality of triangular facets. When the limb movement is determined, the limb mesh can be constructed according to the limb movement data to obtain the limb parameterized mesh. The limb parameterized mesh can be unfolded into a plane mesh, and the position information of each pixel on the plane mesh corresponds one-to-one to the spatial position information of the limb parameterized mesh in three-dimensional space. In addition, the position information of each pixel on the plane mesh corresponds one-to-one to the position information of each pixel on a preset feature map, and the position information of each pixel on the feature map is referred to as map position information. Therefore, the spatial position information and map position information corresponding to each vertex of the triangular facet can be obtained according to the limb parameterized mesh, wherein the spatial position information corresponding to the vertex is the vertex spatial position information, and the map position information corresponding to the vertex is the vertex map position information.

[0106] In step S303 of some embodiments, the preset target triangle patch selection rule is: first, the distance from the spatial sampling point to each triangle patch is obtained according to the spatial sampling point position information and the vertex spatial position, and a candidate distance is obtained, and then the triangle patch is obtained according to the shortest candidate distance. The obtained triangle patch is the target triangle patch. Therefore, the target triangle patch is determined according to the vertex spatial position, the spatial sampling point position information and the preset target triangle patch selection rule. Among them, the spatial position information of the target triangle patch is represented by the target vertex spatial position information, and the target vertex spatial position information includes the three vertex spatial position information of the target triangle.

[0107] In step S304 of some embodiments, each triangular facet of the limb parameterized mesh has a corresponding normal vector. Therefore, the corresponding normal vector can be determined based on the target triangular facet, and the plane normal feature can be determined based on the normal vector.

[0108] In step S307 of some embodiments, the mapping coordinate feature is obtained according to the target vertex spatial position information, the vertex mapping position information and the projection point spatial position information. Specifically, the spatial distance from the spatial projection point to the three vertices of the target triangular facet is first determined according to the target vertex spatial position information and the projection point spatial position information, and a weighted distance value is obtained. Among them, each weighted distance value corresponds to a vertex of the target triangular facet. Therefore, the projection point spatial position information can be expressed as a weighted sum of the target vertex spatial position information and its corresponding weighted distance value. Then, the vertex mapping position information of the target triangular facet is obtained. Since the plane mesh is obtained by unfolding the limb parameterized mesh, the position information of each pixel on the plane mesh corresponds to the spatial position information of the limb parameterized mesh in three-dimensional space, and the position information of each pixel on the plane mesh corresponds to the position information of each pixel on the preset feature map. Therefore, the mapping position information corresponding to the spatial projection point on the feature map can be determined according to the target vertex mapping position information and the weighted distance value. The mapping coordinate feature can be obtained according to the mapping position information corresponding to the spatial projection point on the feature map.

[0109] In step S308 of some embodiments, each pixel on the feature map corresponds to an eigenvalue, and the eigenvalue is used to represent the color information of each pixel on the feature map. Therefore, the eigenvalue can be determined on the preset feature map according to the map coordinate features. Specifically, the eigenvalue is a high-dimensional vector value, and the number of dimensions of the high-dimensional vector is determined according to the color information. The color information of the limbs includes at least: surface color information, roughness information, etc. of the limbs, and can also be other color information commonly used by those skilled in the art. By using the high-dimensional vector value as the eigenvalue, a variety of color information can be considered simultaneously when the limb rendering model is rendering the limb, so as to obtain a more refined and realistic rendering image.

[0110] In step S309 of some embodiments, the observation direction feature can be determined according to the spatial sampling point position information and the virtual light; wherein the observation direction feature is used to describe the direction of the virtual light where the spatial sampling point is located. Specifically, the virtual light where the spatial sampling point is located is determined according to the spatial sampling point position, and the direction of the virtual light is determined according to the virtual light. The observation direction feature is determined according to the direction of the virtual light.

[0111] In step S310 of some embodiments, the action parameter feature, the plane normal feature, the plane projection distance feature, the mapping coordinate feature, the feature value and the viewing direction feature are spliced ​​to obtain the deformable coding information. The specific splicing order can be selected according to the needs of those skilled in the art.

[0112] In the step S301 to step S310 of the embodiment of the present application, the action parameter feature is first determined according to the limb action data. Then, the vertex spatial position information and vertex mapping position information corresponding to each vertex of the triangular patch are obtained according to the limb parameterized mesh, and the target triangular patch is determined according to the vertex spatial position information, the spatial sampling point position information and the preset target triangular patch selection rule; wherein, the spatial position information of the target triangular patch is represented by the target vertex spatial position information. Then, the plane normal feature is determined according to the target triangular patch, and the spatial sampling point is projected to the target triangular patch according to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature to obtain the plane projection distance feature and the spatial projection point; wherein, the projection point spatial position information of the spatial projection point is determined according to the spatial projection point and the target vertex spatial position information. Thereafter, the projection point spatial position information is obtained according to the spatial projection point, the mapping coordinate feature is obtained according to the target vertex spatial position information, the vertex mapping position information and the projection point spatial position information, and the feature value is determined on the preset feature map according to the mapping coordinate feature. And determine the observation direction characteristics according to the spatial sampling point position information and the virtual light; wherein the observation direction characteristics are used to describe the direction of the virtual light where the spatial sampling point is located. Finally, the action parameter characteristics, plane normal characteristics, plane projection distance characteristics, mapping coordinate characteristics, characteristic values ​​and observation direction characteristics are spliced ​​to obtain deformable coding information. The deformable coding information of the present application simultaneously includes the position information characteristics of the spatial sampling points (including action parameter characteristics, plane normal characteristics, plane projection distance characteristics, mapping coordinate characteristics and observation direction characteristic geometric information) and color information characteristics (including characteristic values). Therefore, when the limbs are rendered through the deformable coding information, the geometric information and appearance information of the spatial sampling points can be modeled at the same time, which simplifies the rendering process and improves the efficiency of limb rendering.

[0113] See also Figure 4 In some embodiments, before step S305, the limb rendering method further includes selecting a target sampling point from the spatial sampling points. The specific steps of selecting the target sampling point from the spatial sampling points include but are not limited to steps S401 to S402:

[0114] Step S401, performing comparison based on a preset distance threshold and a plane projection distance feature to obtain a comparison result;

[0115] Step S402: filtering the spatial sampling points according to the comparison result to obtain target sampling points.

[0116] In step S401 of some embodiments, a comparison result can be obtained by comparing the preset distance threshold and the plane projection distance feature. Specifically, the preset distance threshold can be determined according to the following rules: obtain the two vertices with the farthest distance on the limb parameterized mesh, calculate the distance between the two vertices, obtain the maximum distance of the mesh, and select one tenth of the length of the maximum distance of the mesh as the distance threshold. The distance threshold can also be adjusted according to the actual effect, or determined according to other rules.

[0117] In step S402 of some embodiments, the spatial sampling points are filtered according to the comparison result to obtain the target sampling point. Specifically, if the comparison result indicates that the plane projection distance is less than the distance threshold, the spatial sampling point is retained as the target sampling point. The spatial sampling points other than the target sampling point are deleted.

[0118] In some embodiments of the present application, steps S401 to S402 first compare the preset distance threshold and the plane projection distance feature to obtain a comparison result, and then filter the spatial sampling points according to the comparison result to obtain the target sampling points. Therefore, the present application selects appropriate target sampling points to participate in the encoding conversion according to the distance threshold, which reduces the amount of encoding conversion calculation of the limb rendering method, thereby improving the efficiency of limb rendering.

[0119] In step S104 of some embodiments, the feature parameters of the deformable coding information are calculated by a neural network to obtain the spatial feature parameters of each spatial sampling point, wherein the spatial feature parameters include: sampling point color information and sampling point density information. Specifically, the neural network is an eight-layer multilayer perceptron (MLP), and each layer of the MLP consists of a linear transformation module and a nonlinear transformation module.

[0120] See also Figure 5 In some embodiments, step S105 includes but is not limited to steps S501 to S503:

[0121] Step S501, determining a weight coefficient according to the sampling point density information and a preset weight mapping relationship table; wherein the weight mapping relationship table includes a weight coefficient corresponding to each sampling point density information;

[0122] Step S502, performing weighted summation on the sampling point color information according to the weight coefficient, the sampling point opacity information and the virtual light, to obtain the virtual light color information; wherein the sampling point opacity information is determined according to the sampling point density information;

[0123] Step S503 , constructing a target limb rendering image according to the virtual light color information, the virtual light, the mapping relationship and the plane pixels.

[0124] In step S501 of some embodiments, the weight coefficient can be determined according to the sampling point density information and a preset weight mapping relationship table; wherein the weight mapping relationship table includes the weight coefficient corresponding to each sampling point density information. Specifically, each weight coefficient in the weight mapping relationship table is calculated from each sampling point density information from near to far, and the specific calculation process is as follows:

[0125]

[0126] Among them, T i represents the weight coefficient of the i-th spatial sampling point, σ j represents the density information of the jth spatial sampling point, δ j Indicates the relative size relationship between the jth spatial sampling point and the j+1th spatial sampling point. When the virtual light is sampled equidistantly, δ j The value of is 1.

[0127] In step S502 of some embodiments, the color information of the sampling points is weighted and summed according to the weight coefficient, the opacity information of the sampling points and the virtual light to obtain the color information of the virtual light; wherein the opacity information of the sampling points is determined according to the density information of the sampling points. According to the weight coefficient and the virtual light, the color of each spatial sampling point on a virtual light is weighted and summed from near to far to obtain the color information of the virtual light as follows:

[0128]

[0129] in, represents the color information of virtual light, N represents the total number of spatial sampling points on a virtual light, α i represents the sampling point opacity information of the i-th spatial sampling point, c i Represents the color information of the i-th spatial sampling point.

[0130] As disclosed above, the sampling point opacity information α of each spatial sampling point i It is obtained based on the relative size of the sampling point density information and the distance between the spatial sampling points. The specific process is as follows:

[0131] α i =1-exp(σ i δ i );

[0132] Among them, σ i represents the density information of the i-th spatial sampling point, δ i Indicates the relative size relationship between the i-th spatial sampling point and the i+1-th spatial sampling point. When the virtual light is sampled equidistantly, δ i The value of is 1.

[0133] In step S503 of some embodiments, the target limb rendering image can be constructed according to the virtual light color information, the virtual light, the mapping relationship and the plane pixels. Specifically, since there is a mapping relationship between the virtual light and the plane pixels, the rendering color information of each plane pixel can be determined according to the virtual light color information, the virtual light, the mapping relationship and the plane pixels. Therefore, the target limb rendering image can be constructed according to all the plane pixels and their corresponding rendering color information.

[0134] In steps S501 to S503 of the embodiment of the present application, the weight coefficient is first determined based on the sampling point density information and the preset weight mapping relationship table; wherein the weight mapping relationship table includes the weight coefficient corresponding to each sampling point density information. Then, according to the weight coefficient, the sampling point opacity information and the virtual light, the sampling point color information is weighted and summed to obtain the virtual light color information; wherein the sampling point opacity information is determined according to the sampling point density information. Finally, the target limb rendering image is constructed based on the virtual light color information, the virtual light, the mapping relationship and the plane pixels. Through the above steps, the limb rendering method of the present application can finally render a target limb rendering image with a high degree of color similarity to the real limb image.

[0135] See also Figure 6 In some embodiments, the internal parameters collected include color channel parameters. After step S105, the method further includes but is not limited to steps S601 to S602:

[0136] Step S601, performing deviation correction on the color channel parameters according to preset correction parameters to obtain corrected color channel parameters;

[0137] Step S602: Correct the target limb rendering image according to the correction color channel parameters to obtain a limb rendering correction image.

[0138] In step S601 of some embodiments, the color channel parameters can be corrected for deviation according to preset correction parameters to obtain corrected color channel parameters. Specifically, the color channel parameters may include but are not limited to: red channel parameters, green channel parameters and blue channel parameters; the correction parameters include gain parameters and deviation correction parameters. The color channel parameter process is as follows:

[0139] c′=c⊙g+b;

[0140] Among them, c′ is the correction color channel parameter feature vector, c′ can be converted into the correction color channel parameter, c is the color channel parameter feature vector obtained by vector transformation of the color channel parameter, g is the gain parameter feature vector obtained by vector transformation according to the gain parameter, b is the deviation correction parameter feature vector obtained by vector transformation according to the deviation correction parameter, c′, c, g and b are all high-dimensional vectors with the same dimension, ⊙ represents the multiplication of two features of the same dimension of two high-dimensional feature vectors.

[0141] In step S602 of some embodiments, the target limb rendering image is corrected according to the correction color channel parameters to obtain the limb rendering correction image. Specifically, the rendering color information of each plane pixel is corrected according to the correction color channel parameters to obtain the correction rendering color information. The limb rendering correction image is constructed according to all plane pixels and their corresponding correction rendering color information.

[0142] In step S601 to step S602 of the embodiment of the present application, the color channel parameters are firstly corrected for deviation according to the preset correction parameters to obtain the corrected color channel parameters, and then the target limb rendering image is corrected according to the corrected color channel parameters to obtain the limb rendering corrected image. Therefore, the limb rendering method of the present application can reduce the color information error of the target limb rendering image caused by the difference in the internal acquisition parameters of the acquisition device by correcting the target limb rendering image.

[0143] See also Figure 7 In some embodiments, after step S105, the limb rendering method further includes optimizing the rendering model. The specific steps of optimizing the rendering model include but are not limited to steps S701 to S704:

[0144] Step S701, constructing a loss function based on a preset limb verification image and a target limb rendering image;

[0145] Step S702, performing back propagation calculation on the loss function to obtain the optimization parameter gradient;

[0146] Step S703, performing gradient descent update on the preset optimizable parameters according to the optimization parameter gradient to obtain the target optimizable parameters; wherein the optimizable parameters include eigenvalues ​​and correction parameters;

[0147] Step S704: Optimize the rendering model according to the target optimizable parameters.

[0148] In step S701 of some embodiments, a loss function is constructed based on a preset limb verification image and a target limb rendering image. The specific process of constructing the loss function is as follows:

[0149] First, a photometric loss function is constructed based on the photometric of the target limb rendering image and the photometric of the limb verification image:

[0150]

[0151] in, represents the photometric loss function, I pred Represents the luminosity of the target limb rendered image, I gt represents the luminosity of the limb verification image;

[0152] Then, the learned perceptual image patch similarity (LPIPS) loss function is constructed based on the eigenvalues ​​of the target limb rendering image and the eigenvalues ​​of the limb verification image:

[0153]

[0154] in, represents the learning perceptual image patch similarity loss function, f(I pred ) represents the feature value of the target limb rendering image, f(I gt ) represents the feature value of the limb verification image;

[0155] The total loss function is constructed based on the photometric loss function and the learning-perceptual image patch similarity loss function:

[0156]

[0157] in, Indicates that λ1 is 0.2.

[0158] In step S703 of some embodiments, the target optimizable parameters can be obtained by performing gradient descent update on the preset optimizable parameters according to the optimization parameter gradient; wherein the optimizable parameters include eigenvalues ​​and correction parameters. Specifically, by updating the eigenvalues, after the rendering model training is completed, fine eigenvalues ​​reflecting multiple detail features of different parts of the limb can be obtained, thereby enhancing the expression ability of the model and ultimately making the target limb rendering image more realistic. By updating the correction parameters, the correction parameters finally obtained can better reduce the color information error of the target limb rendering image caused by the difference in the internal acquisition parameters of the acquisition device, and ultimately improve the realism of the target limb rendering image.

[0159] In steps S701 to S704 of the embodiment of the present application, a loss function is first constructed based on a preset limb verification image and a target limb rendering image, and a back propagation calculation is performed on the loss function to obtain an optimization parameter gradient. Then, the preset optimizable parameters are updated by gradient descent according to the optimization parameter gradient to obtain the target optimizable parameters; wherein the optimizable parameters include eigenvalues ​​and correction parameters. Finally, the rendering model is optimized according to the target optimizable parameters. It can be seen that the limb rendering method of the present application optimizes the optimizable parameters to obtain the target optimizable parameters, and continuously optimizes the rendering model through the target optimizable parameters, so that the final rendering model has a higher rendering accuracy and renders a higher quality target limb rendering image.

[0160] See also Figure 8 , Figure 8 It is an optional overall architecture diagram of the limb rendering method provided in the embodiment of the present application.

[0161] like Figure 8As shown, in some embodiments, the limb rendering method provided by the present application is used for rendering a human hand image. The specific human hand image rendering process is as follows: First, obtain the human hand motion data (i.e., the limb motion data of the target object) and the virtual camera parameters (i.e., the acquisition parameters of the limb motion data), and then input the human hand motion data and the virtual camera parameters into the preset human hand rendering model; wherein, the human hand rendering model includes: a human hand grid sub-model (i.e., a limb grid sub-model) and an MLP (i.e., a neural network); the virtual camera parameters include: camera position information and camera internal parameter information; the human hand grid sub-model includes: a sampling network and a human hand grid construction network (i.e., a limb grid construction network). Then, a plane is constructed through the sampling network, the camera position information, and the camera internal parameter information to obtain an imaging plane image; wherein, the imaging plane image includes a plurality of plane pixels; and each plane pixel is back-projected into a three-dimensional space through the sampling network and the preset mapping relationship to obtain a virtual light corresponding to each plane pixel. Thereafter, each virtual light is sampled at equal distances through the sampling network to obtain a plurality of sampling space points (i.e., spatial sampling points), and the position information of each sampling space point is obtained through the sampling network to obtain the sampling space point position information. Next, the hand mesh construction network is used to construct the hand mesh of the hand action data to obtain a parameterized mesh, and the sampling space points are projected onto the parameterized mesh. Then, encoding and conversion are performed based on the hand parameterized mesh, the hand action data and the sampling space point position information to obtain a deformable code (i.e., deformable code information); wherein the deformable code includes action parameters (i.e., action parameter features), triangle patch normals (i.e., plane normal features), projection distances (i.e., plane projection distance features), mapping coordinates (i.e., mapping coordinate features), mapping feature values ​​(i.e., feature values) and viewing directions (i.e., viewing direction features). Finally, the deformable code is calculated by MLP to calculate the feature parameters, obtain the color (i.e., sampling point color information) and density (i.e., sampling point density information) of each sampling space point, and perform volume rendering processing (i.e., rendering processing) on ​​the color and density to obtain a hand rendered image (i.e., target limb rendered image).

[0162] See also Fig. 9 The embodiment of the present application further provides a limb rendering system, which can implement the above limb rendering method, and the system includes:

[0163] The parameter acquisition module 901 is used to acquire the body movement data of the target object and the acquisition parameters of the body movement data; wherein the acquisition parameters are the operating parameters of the acquisition device for acquiring the body movement data;

[0164] The parameter input module 902 is used to input the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network;

[0165] The encoding conversion module 903 is used to encode the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable encoding information; wherein the deformable encoding information represents the spatial characteristics of each spatial sampling point on the limb of the target object in the three-dimensional space;

[0166] The spatial feature parameter calculation module 904 is used to calculate the feature parameters of the deformable coding information through a neural network to obtain the spatial feature parameters of each spatial sampling point; wherein the spatial feature parameters include: sampling point color information and sampling point density information;

[0167] The image rendering module 905 is used to render the sampling point color information and the sampling point density information to obtain a target limb rendering image.

[0168] The specific implementation of the limb rendering system is substantially the same as the specific implementation of the limb rendering method described above, and will not be described in detail herein.

[0169] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned limb rendering method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0170] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0171] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0172] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other applications. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the limb rendering method of the embodiment of the present application;

[0173] Input / output interface 1003, used to implement information input and output;

[0174] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0175] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0176] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0177] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned limb rendering method is implemented.

[0178] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0179] The limb rendering method, limb rendering system, electronic device and storage medium provided in the embodiments of the present application obtain the limb motion data of the target object and the acquisition parameters of the limb motion data, and input the limb motion data and the acquisition parameters into a preset rendering model for limb rendering. The rendering model includes: a limb mesh sub-model and a neural network. The rendering model first encodes and converts the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable coding information, and then calculates the feature parameters of the deformable coding information through the neural network to obtain the spatial feature parameters of each spatial sampling point, wherein the spatial feature parameters include: sampling point color information and sampling point density information. Finally, the sampling point color information and the sampling point density information are rendered to obtain a target limb rendering image. Therefore, through the limb rendering method and system, electronic device and storage medium proposed in the present application, it is possible to directly use the parameterized mesh model to convert the limb motion data and acquisition parameters of the limb mesh sub-model into deformable coding information, and then calculate the spatial feature parameters of the spatial sampling points through the deformable coding information. After that, the image rendering is performed through volume rendering, so as to achieve fast rendering of the limb image. There is no need to model the parameterized limb mesh model multiple times to model the geometric information and appearance information of the limb separately, which simplifies the limb rendering steps, shortens the time required for manual modeling and rendering, and improves the limb rendering quality and efficiency.

[0180] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0181] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0182] The system embodiment described above is merely illustrative, and the modules described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules in the devices disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0184] The terms "including" and "having" and any variations thereof in the specification of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system or device comprising a series of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0185] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. The character " / " generally indicates that the objects associated with each other are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of a single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0186] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0187] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0188] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0189] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0190] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A body rendering method, characterized in that: The method comprises: Acquire the limb motion data of the target object and the acquisition parameters of the limb motion data; wherein the acquisition parameters are the operating parameters of the acquisition device for acquiring the limb motion data; Inputting the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network; The limb motion data and the acquisition parameters are encoded and converted by the limb mesh sub-model to obtain deformable encoding information; wherein the deformable encoding information represents the spatial characteristics of each spatial sampling point on the limb of the target object in three-dimensional space; The deformable coding information is subjected to feature parameter calculation by the neural network to obtain the spatial feature parameters of each of the spatial sampling points; wherein the spatial feature parameters include: sampling point color information and sampling point density information; The sampling point color information and the sampling point density information are rendered to obtain a target limb rendering image.

2. The method according to claim 1, characterized in that The acquisition parameters include: acquisition position information and acquisition internal parameters; the limb mesh sub-model includes: a sampling network and a limb mesh construction network; The encoding and conversion of the limb motion data and the acquisition parameters by the limb mesh sub-model to obtain deformable encoding information includes: A plane is constructed by using the sampling network, the acquisition position information and the acquisition internal parameters to obtain an imaging plane image; wherein the imaging plane image includes a plurality of plane pixels; Back-projecting each of the plane pixels into the three-dimensional space through the sampling network and the preset mapping relationship to obtain a virtual light corresponding to each of the plane pixels; Performing equidistant sampling on each of the virtual rays through the sampling network to obtain a plurality of spatial sampling points; Acquire the position information of each of the spatial sampling points through the sampling network to obtain the spatial sampling point position information; Performing limb mesh construction on the limb motion data through the limb mesh construction network to obtain a limb parameterized mesh; The deformable coding information is obtained by performing coding conversion according to the limb parameterized grid, the limb motion data and the spatial sampling point position information.

3. The method according to claim 2, characterized in that The limb parameterized mesh is composed of a plurality of triangular facets, and the encoding conversion is performed according to the limb parameterized mesh, the limb motion data and the spatial sampling point position information to obtain the deformable encoding information, including: Determining the motion parameter characteristics according to the limb motion data; Acquire vertex space position information and vertex map position information corresponding to each vertex of the triangular facet according to the limb parameterized mesh; Determine a target triangular patch according to the vertex spatial position information, the spatial sampling point position information and a preset target triangular patch selection rule; wherein the spatial position information of the target triangular patch is represented by the target vertex spatial position information; Determine a plane normal feature according to the target triangular facet; According to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature, the target sampling point is projected onto the target triangle facet to obtain the plane projection distance feature and the spatial projection point; wherein the target sampling point is selected based on the spatial sampling point; Acquire the position information of the spatial projection point to obtain the spatial position information of the projection point; Acquire the mapping features of the triangular face according to the target vertex spatial position information, the vertex mapping position information and the projection point spatial position information to obtain mapping coordinate features; Determining a feature value on a preset feature map according to the map coordinate feature; Determine an observation direction feature according to the spatial sampling point position information and the virtual light; wherein the observation direction feature is used to describe the direction of the virtual light where the spatial sampling point is located; The action parameter feature, the plane normal feature, the plane projection distance feature, the mapping coordinate feature, the feature value and the observation direction feature are spliced ​​to obtain the deformable coding information.

4. The method according to claim 3, characterized in that: Before projecting the target sampling point to the target triangle patch according to the spatial sampling point position information, the target vertex spatial position information and the plane normal feature to obtain the plane projection distance feature and the spatial projection point, the method further includes: Selecting the target sampling point from the spatial sampling points specifically includes: Compare the preset distance threshold with the plane projection distance feature to obtain a comparison result; The spatial sampling points are filtered according to the comparison result to obtain target sampling points.

5. The method according to claim 2, characterized in that: The rendering process of the sampling point color information and the sampling point density information to obtain a target limb rendering image includes: Determine a weight coefficient according to the sampling point density information and a preset weight mapping relationship table; wherein the weight mapping relationship table includes the weight coefficient corresponding to each sampling point density information; According to the weight coefficient, the opacity information of the sampling point and the virtual light, weighted summation is performed on the color information of the sampling point to obtain the color information of the virtual light; wherein the opacity information of the sampling point is determined according to the density information of the sampling point; The target limb rendering image is constructed according to the virtual light color information, the virtual light, the mapping relationship and the plane pixels.

6. The method according to claim 5, characterized in that The collecting of internal parameters includes color channel parameters, and after the rendering processing of the sampling point color information and the sampling point density information is performed to obtain a target limb rendering image, the method further includes: Performing deviation correction on the color channel parameters according to preset correction parameters to obtain corrected color channel parameters; The target limb rendering image is corrected according to the correction color channel parameters to obtain a limb rendering correction image.

7. The method according to any one of claims 1 to 6, characterized in that: After inputting the limb motion data and the acquisition parameters into a preset rendering model, the method further includes: Optimizing the rendering model specifically includes: Constructing a loss function based on a preset limb verification image and the target limb rendering image; Performing back propagation calculation on the loss function to obtain the optimization parameter gradient; Performing gradient descent update on preset optimizable parameters according to the optimization parameter gradient to obtain target optimizable parameters; wherein the optimizable parameters include the eigenvalues ​​and the correction parameters; The rendering model is optimized according to the target optimizable parameters.

8. A limb rendering system, characterized in that: The system comprises: A parameter acquisition module, used to acquire the body movement data of the target object and acquisition parameters of the body movement data; wherein the acquisition parameters are operating parameters of an acquisition device for acquiring the body movement data; A parameter input module, used to input the limb motion data and the acquisition parameters into a preset rendering model; wherein the rendering model includes: a limb mesh sub-model and a neural network; A coding conversion module, used for coding and converting the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable coding information; wherein the deformable coding information represents the spatial characteristics of each spatial sampling point on the limb of the target object in three-dimensional space; A spatial feature parameter calculation module, used to calculate the feature parameters of the deformable coding information through the neural network to obtain the spatial feature parameters of each of the spatial sampling points; wherein the spatial feature parameters include: sampling point color information and sampling point density information; The image rendering module is used to render the sampling point color information and the sampling point density information to obtain a target limb rendering image.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the limb rendering method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the limb rendering method according to any one of claims 1 to 7 is implemented.

Images