Limb rendering method and system, electronic device, and storage medium

Abstract

Embodiments of the present application provide a limb rendering method and system, an electronic device and a storage medium, belonging to the technical field of three-dimensional reconstruction. The method first acquires limb motion data of a target object and acquisition parameters of the limb motion data, and inputs the limb motion data and the acquisition parameters to a preset rendering model for limb rendering. The rendering model includes a limb mesh sub-model and a neural network. The limb motion data and the acquisition parameters are encoded and converted by the limb mesh sub-model to obtain deformable encoding information, and the neural network is used to calculate feature parameters of the deformable encoding information to obtain spatial feature parameters of each spatial sampling point. The spatial feature parameters include sampling point color information and sampling point density information. Finally, the sampling point color information and the sampling point density information are rendered to obtain a target limb rendering image. Through the above method, the embodiments of the present application can improve the quality and efficiency of limb rendering.

Description

Technical Field

[0001] This application relates to the field of 3D reconstruction technology, and in particular to a limb rendering method and system, electronic device and storage medium. Background Technology

[0002] Limb rendering refers to generating images of a limb from various perspectives based on a given limb movement. Realistic limb rendering has important applications in many fields, such as human-computer interaction, virtual reality, and games. In related technologies, limb rendering methods typically begin by upsampling a parametric limb mesh model to obtain a high-resolution parametric limb mesh model. Then, based on the high-resolution parametric limb mesh model, the geometric and appearance information of the limb is modeled separately. Modeling the geometric information of the limb is achieved through the following steps: the vertices of the sampled high-resolution limb mesh model and the spatial point to be queried are input into a neural network, and then the neural network performs feature processing to output the occupancy value of this spatial point. Modeling the appearance information of the limb is achieved through the following steps: decoupling the surface appearance and lighting information of the limb, predicting color information from the surface position information of the limb, and then multiplying it by the lighting information predicted from the current position to obtain the final color information of this spatial point.

[0003] However, current limb rendering methods require calculating the color information of all spatial points of the limb multiple times before rendering can be completed, making the rendering process complex. These drawbacks result in low limb rendering efficiency in existing methods. Therefore, improving limb rendering efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0004] The main objective of this application is to provide a limb rendering method and system, electronic device and storage medium, which aims to improve limb rendering efficiency.

[0005] To achieve the above objectives, a first aspect of this application proposes a limb rendering method, the method comprising:

[0006] Acquire the limb movement data of the target object and the acquisition parameters of the limb movement data; wherein, the acquisition parameters are the operating parameters of the acquisition device that acquires the limb movement data;

[0007] The limb movement data and the acquisition parameters are input into a preset rendering model; wherein, the rendering model includes: a limb mesh sub-model and a neural network;

[0008] The limb motion data and the collected parameters are encoded and converted using the limb mesh sub-model to obtain deformable encoded information; wherein, the deformable encoded information characterizes the spatial features of each spatial sampling point on the target object's limb in three-dimensional space;

[0009] The deformable encoded information is processed by the neural network to calculate the feature parameters, thereby obtaining the spatial feature parameters of each spatial sampling point; wherein, the spatial feature parameters include: sampling point color information and sampling point density information;

[0010] The color information and density information of the sampling points are processed by rendering to obtain a rendered image of the target limb.

[0011] In some embodiments, the acquisition parameters include: acquisition location information and acquisition internal parameters; the limb mesh sub-model includes: a sampling network and a limb mesh construction network;

[0012] The process of encoding and converting the limb movement data and the acquired parameters through the limb mesh sub-model to obtain deformable encoded information includes:

[0013] An imaging plane image is obtained by constructing a plane using the sampling network, the acquisition location information, and the acquisition internal parameters; wherein, the imaging plane image includes multiple planar pixels;

[0014] Each planar pixel is back-projected into three-dimensional space through the sampling network and the preset mapping relationship to obtain the virtual ray corresponding to each planar pixel;

[0015] The sampling network is used to sample each virtual ray at equal intervals to obtain multiple spatial sampling points;

[0016] The location information of each spatial sampling point is obtained through the sampling network to obtain the spatial sampling point location information.

[0017] The limb motion data is used to construct a parametric limb mesh through the limb mesh construction network to obtain a limb parametric mesh.

[0018] The deformable encoded information is obtained by encoding and converting the limb parametric mesh, the limb motion data, and the spatial sampling point position information.

[0019] In some embodiments, the limb parametric mesh is composed of multiple triangular patches. The step of encoding and converting the limb parametric mesh, the limb motion data, and the spatial sampling point position information to obtain the deformable encoded information includes:

[0020] The movement parameter characteristics are determined based on the limb movement data;

[0021] Based on the parametric mesh of the limbs, obtain the vertex spatial position information and vertex texture position information corresponding to each vertex of the triangular facet;

[0022] The target triangle facet is determined based on the vertex spatial position information, the spatial sampling point position information, and the preset target triangle facet selection rules; wherein, the spatial position information of the target triangle facet is represented by the target vertex spatial position information;

[0023] Determine the planar normal features based on the target triangular facet;

[0024] Based on the spatial sampling point location information, the target vertex spatial location information, and the plane normal feature, the target sampling point is projected onto the target triangle facet to obtain the plane projection distance feature and the spatial projection point; wherein, the target sampling point is selected based on the spatial sampling point;

[0025] Obtain the position information of the spatial projection point to obtain the spatial position information of the projection point;

[0026] Based on the target vertex spatial position information, the vertex texture position information, and the projection point spatial position information, the texture features of the triangle facet are obtained, and the texture coordinate features are obtained.

[0027] Based on the texture coordinate features, determine the feature values ​​on the preset feature texture;

[0028] The observation direction features are determined based on the spatial sampling point location information and the virtual ray; wherein, the observation direction features are used to describe the direction of the virtual ray where the spatial sampling point is located;

[0029] The deformable coding information is obtained by concatenating the motion parameter features, the plane normal features, the plane projection distance features, the texture coordinate features, the feature values, and the observation direction features.

[0030] In some embodiments, before projecting the target sampling point onto the target triangle facet based on the spatial sampling point location information, the target vertex spatial location information, and the plane normal feature to obtain the plane projection distance feature and the spatial projection point, the method further includes:

[0031] Selecting the target sampling point from the spatial sampling points specifically includes:

[0032] The comparison is performed based on a preset distance threshold and the planar projection distance features to obtain a comparison result;

[0033] The spatial sampling points are filtered based on the comparison results to obtain the target sampling points.

[0034] In some embodiments, the rendering process of the sampled point color information and the sampled point density information to obtain a target limb rendering image includes:

[0035] The weight coefficients are determined based on the sampling point density information and a preset weight mapping table; wherein, the weight mapping table includes the weight coefficients corresponding to each sampling point density information.

[0036] The color information of the sampling points is weighted and summed based on the weighting coefficients, the opacity information of the sampling points, and the virtual ray to obtain the color information of the virtual ray; wherein, the opacity information of the sampling points is determined based on the density information of the sampling points.

[0037] The target limb rendering image is constructed based on the virtual light color information, the virtual light, the mapping relationship, and the planar pixels.

[0038] In some embodiments, the acquisition of internal parameters includes color channel parameters. After rendering the color information and density information of the sampling points to obtain a rendered image of the target limb, the method further includes:

[0039] The color channel parameters are corrected for deviation according to the preset correction parameters to obtain the corrected color channel parameters;

[0040] The target limb rendering image is corrected according to the corrected color channel parameters to obtain a limb rendering corrected image.

[0041] In some embodiments, after inputting the limb movement data and the acquisition parameters into a preset rendering model, the method further includes:

[0042] Optimizing the rendering model specifically includes:

[0043] A loss function is constructed based on the preset limb verification image and the target limb rendering image;

[0044] The loss function is backpropagated to obtain the gradient of the optimization parameters;

[0045] The preset optimizable parameters are updated by gradient descent based on the gradient of the optimization parameters to obtain the target optimizable parameters; wherein, the optimizable parameters include the feature values ​​and the correction parameters;

[0046] The rendering model is optimized based on the target optimizable parameters.

[0047] To achieve the above objectives, a second aspect of this application provides a limb rendering system, the system comprising:

[0048] The parameter acquisition module is used to acquire the limb movement data of the target object and the acquisition parameters of the limb movement data; wherein, the acquisition parameters are the operating parameters of the acquisition device that acquires the limb movement data;

[0049] The parameter input module is used to input the limb movement data and the collected parameters into a preset rendering model; wherein, the rendering model includes: a limb mesh sub-model and a neural network;

[0050] The encoding conversion module is used to encode and convert the limb motion data and the acquisition parameters through the limb mesh sub-model to obtain deformable encoded information; wherein, the deformable encoded information represents the spatial characteristics of each spatial sampling point on the target object's limb in three-dimensional space;

[0051] The spatial feature parameter calculation module is used to calculate the feature parameters of the deformable encoded information through the neural network to obtain the spatial feature parameters of each spatial sampling point; wherein, the spatial feature parameters include: sampling point color information and sampling point density information;

[0052] The image rendering module is used to render the color information and density information of the sampling points to obtain a rendered image of the target limb.

[0053] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0054] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0055] The limb rendering method, system, electronic device, and storage medium proposed in this application acquire limb motion data and acquisition parameters of the target object, and input the limb motion data and acquisition parameters into a preset rendering model for limb rendering. The rendering model includes a limb mesh sub-model and a neural network. Therefore, the limb motion data and acquisition parameters are first encoded and converted using the limb mesh sub-model to obtain deformable encoded information. Then, the deformable encoded information is used to calculate feature parameters, obtaining spatial feature parameters for each spatial sampling point. These spatial feature parameters include sampling point color information and sampling point density information. Finally, the sampling point color information and sampling point density information are rendered to obtain the target limb rendered image. Therefore, the limb rendering method, system, electronic device, and storage medium proposed in this application can directly utilize a parametric mesh model to convert limb motion data and acquisition parameters into deformable coded information from the limb mesh sub-model. Then, the spatial feature parameters of the spatial sampling points can be calculated using the deformable coded information, and finally, image rendering can be performed through volume rendering. This enables rapid rendering of limb images without the need for multiple modeling steps of the parametric limb mesh model, which separately model the geometric and appearance information of the limb. This simplifies the limb rendering process, shortens the time required for human hand modeling and rendering, and improves the quality and efficiency of limb rendering. Attached Figure Description

[0056] Figure 1 This is an optional flowchart of the limb rendering method provided in the embodiments of this application;

[0057] Figure 2 yes Figure 1 The flowchart of step S103 in the process;

[0058] Figure 3 This is another optional flowchart of the limb rendering method provided in the embodiments of this application;

[0059] Figure 4 yes Figure 3 The flowchart of step S206 in the text;

[0060] Figure 5 yes Figure 3 The flowchart of step S105 in the process;

[0061] Figure 6 This is another optional flowchart of the limb rendering method provided in the embodiments of this application;

[0062] Figure 7 This is another optional flowchart of the limb rendering method provided in the embodiments of this application;

[0063] Figure 8This is a schematic diagram of an optional overall architecture of the limb rendering method provided in the embodiments of this application;

[0064] Figure 9 This is an optional structural diagram of the limb rendering system provided in the embodiments of this application;

[0065] Figure 10 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that although functional modules are divided in the system diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0069] This application provides a limb rendering method and system, electronic device and storage medium, which aim to improve the efficiency of limb rendering.

[0070] The limb rendering method, system, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the limb rendering method in this application embodiment is described.

[0071] The limb rendering method provided in this application relates to the field of 3D reconstruction technology. The limb rendering method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the limb rendering method, but is not limited to the above forms.

[0072] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0073] Figure 1 This is an optional flowchart of the limb rendering method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S105.

[0074] Step S101: Obtain the limb movement data of the target object and the acquisition parameters of the limb movement data; wherein, the acquisition parameters are the operating parameters of the acquisition device for acquiring limb movement data;

[0075] Step S102: Input the limb motion data and acquisition parameters into the preset rendering model; wherein, the rendering model includes: limb mesh sub-model and neural network;

[0076] Step S103: Encode and convert limb motion data and acquisition parameters through limb mesh sub-model to obtain deformable coded information; wherein, deformable coded information characterizes the spatial features of each spatial sampling point on the target object's limb in three-dimensional space.

[0077] Step S104: The feature parameters of the deformable coded information are calculated by the neural network to obtain the spatial feature parameters of each spatial sampling point; wherein, the spatial feature parameters include: sampling point color information and sampling point density information;

[0078] Step S105: Render the color information and density information of the sampling points to obtain the rendered image of the target limb.

[0079] Steps S101 to S105, as illustrated in this embodiment, involve acquiring limb movement data and acquisition parameters of the target object, and then inputting the limb movement data and acquisition parameters into a preset rendering model for limb rendering. The rendering model includes a limb mesh sub-model and a neural network. The rendering model first encodes and converts the limb movement data and acquisition parameters using the limb mesh sub-model to obtain deformable encoded information. Then, it calculates feature parameters for the deformable encoded information using the neural network to obtain spatial feature parameters for each spatial sampling point. These spatial feature parameters include sampling point color information and sampling point density information. Finally, the sampling point color information and sampling point density information are rendered to obtain a rendered image of the target limb. Therefore, the limb rendering method proposed in this application can directly utilize the parametric mesh model to convert limb motion data and acquisition parameters into deformable coded information from the limb mesh sub-model. Then, the spatial feature parameters of the spatial sampling points are calculated using the deformable coded information, and the image is rendered using volume rendering. This enables rapid rendering of limb images without the need for multiple modeling steps of the parametric limb mesh model, which separately model the geometric and appearance information of the limb. This simplifies the limb rendering process, shortens the time required for limb rendering, and improves the quality and efficiency of limb rendering.

[0080] In step S101 of some embodiments, limb movement data of the target object and acquisition parameters for the limb movement data are acquired. Specifically, the limb movement data is data that can characterize the limb movements of the target object. The acquisition parameters are the operating parameters of the acquisition device that acquires the limb movement data, and the acquisition parameters include at least one of the following: acquisition position information and acquisition internal parameters. For example, the limb movement data can be the movement data of a human hand, the acquisition device can be a virtual camera, and the acquisition parameters are the operating parameters of the virtual camera, and the operating parameters of the virtual camera include at least one of the following: camera position parameters and camera internal parameters. Among them, the camera position parameters are determined by the camera origin position information, camera displacement information, and camera rotation information of the virtual camera in three-dimensional space, and the camera internal parameters include the focal length information and optical center information of the camera during the projection process. The acquisition parameters can also be other operating parameters of the acquisition device, as long as they can acquire the limb movement data required for the limb rendering model. The specific parameters can be selected according to the actual needs of those skilled in the art, and are not limited to the acquisition parameters described in the above examples.

[0081] Please see Figure 2 In some embodiments, the limb mesh sub-model includes: a sampling network and a limb mesh construction network, and step S103 may include, but is not limited to, steps S201 to S206:

[0082] Step S201: A plane is constructed by sampling network, acquiring location information and acquiring internal parameters to obtain an imaging plane image; wherein, the imaging plane image includes multiple planar pixels;

[0083] Step S202: Each plane pixel is back-projected into three-dimensional space through a sampling network and a preset mapping relationship to obtain the virtual ray corresponding to each plane pixel;

[0084] Step S203: Sample each virtual ray at equal intervals using a sampling network to obtain multiple spatial sampling points;

[0085] Step S204: Obtain the location information of each spatial sampling point through the sampling network to obtain the spatial sampling point location information;

[0086] Step S205: The limb motion data is constructed by constructing a limb mesh to obtain a parametric limb mesh.

[0087] Step S206: Based on the limb parametric mesh, limb motion data and spatial sampling point location information, the encoding conversion is performed to obtain deformable encoding information.

[0088] In step S201 of some embodiments, the acquired position information is used to describe the position of the acquisition device in three-dimensional space, and the acquired internal parameters are used to describe the internal parameters required by the acquisition device to acquire the position information. Based on this, a plane is constructed using a sampling network, the acquired position information, and the acquired internal parameters to obtain an imaging plane image; wherein, the imaging plane image includes multiple planar pixels. Specifically, the position of the acquisition device in three-dimensional space is determined using a sampling network and the acquired position information. Then, an imaging plane is constructed in three-dimensional space using the sampling network, the acquired position information of the acquisition device, and the acquired internal parameters, and an imaging plane image is determined on the imaging plane. Taking a virtual camera as an example, the position coordinates and projection direction information of the virtual camera in three-dimensional space are obtained through the camera position parameters. Based on the position coordinates, focal length, and optical center information of the virtual camera in three-dimensional space, a plane passing through the projection focal point and perpendicular to the projection direction is taken as the image plane. Then, the imaging plane image is determined on the image plane using the sampling network. The focal length and optical center information are the acquired internal parameters of the virtual camera.

[0089] In step S202 of some embodiments, each planar pixel can be back-projected into three-dimensional space through a sampling network and a preset mapping relationship to obtain a virtual ray corresponding to each planar pixel. The preset mapping relationship represents a one-to-one correspondence between planar pixels on the imaging plane and virtual rays in three-dimensional space. Taking a virtual camera as an example, with the camera position information as the origin, a ray is drawn along the origin-planar pixel direction; the resulting ray will coincide with a virtual ray, and the direction of the ray is the direction of the virtual ray. Based on this, the camera position information and planar pixel position information through which the virtual ray passes can be obtained through the sampling network to determine the virtual ray position information. The preset mapping relationship may include: camera position information, planar pixel position information, and virtual ray position information; the preset mapping relationship may also be other mapping relationships that can be obtained by those skilled in the art based on existing technology.

[0090] In step S203 of some embodiments, multiple spatial sampling points can be obtained by sampling each virtual ray at equal intervals through a sampling network. These spatial sampling points can be obtained by sampling the virtual ray at equal intervals, or by sampling the virtual ray at non-equal intervals according to specific rules. The specific rules can be set by those skilled in the art according to actual needs. The number of spatial sampling points can be 128, or it can be a specific number set by those skilled in the art when actually performing spatial sampling.

[0091] In step S205 of some embodiments, a limb mesh construction network can be used to construct a parametric limb mesh from the limb motion data. This limb mesh construction network is used to construct the parametric limb mesh, which can be a head parametric mesh, a hand parametric mesh, a leg parametric mesh, or other human body parametric model mesh, or even an animal limb parametric mesh. Taking a human hand parametric mesh as an example, the limb motion data of a human hand can be represented by 48 motion parameter features. These 48 motion parameter features include the three-dimensional rotation angle vector features of 16 joints, including three joints for each finger and one joint for the palm. The changes in three-dimensional space, based on these 48 motion features, can determine the human hand's movement.

[0092] In step S206 of some embodiments, encoding conversion is performed based on the limb parameterized mesh, limb motion data and spatial sampling point location information to obtain deformable encoded information.

[0093] Please see Figure 3 In some embodiments, step S206 may include, but is not limited to, steps S301 to S310:

[0094] Step S301: Determine the motion parameter characteristics based on limb motion data;

[0095] Step S302: Obtain the vertex spatial position information and vertex texture position information corresponding to each vertex of the triangular facet based on the limb parametric mesh;

[0096] Step S303: Determine the target triangle facet based on the vertex spatial position information, the spatial sampling point position information, and the preset target triangle facet selection rules; wherein, the spatial position information of the target triangle facet is represented by the target vertex spatial position information;

[0097] Step S304: Determine the planar normal features based on the target triangular facet;

[0098] Step S305: Based on the spatial sampling point location information, the target vertex spatial location information, and the plane normal features, the target sampling points are projected onto the target triangle facet to obtain the plane projection distance features and the spatial projection points; wherein, the target sampling points are selected based on the spatial sampling points.

[0099] Step S306: Obtain the location information of the spatial projection point to obtain the spatial location information of the projection point;

[0100] Step S307: Obtain the texture features of the triangle facet based on the target vertex spatial position information, vertex texture position information and projection point spatial position information, and obtain the texture coordinate features;

[0101] Step S308: Determine feature values ​​on a preset feature map based on the texture coordinate features;

[0102] Step S309: Determine the observation direction features based on the spatial sampling point location information and the virtual ray; wherein, the observation direction features are used to describe the direction of the virtual ray where the spatial sampling point is located;

[0103] Step S310: The motion parameter features, plane normal features, plane projection distance features, texture coordinate features, feature values ​​and viewing direction features are spliced ​​together to obtain deformable coding information.

[0104] In step S301 of some embodiments, the limb movement data includes movement parameter features and movement parameter feature values. Therefore, the movement parameter features can be determined based on the limb movement data. Taking human hand movement data as an example, human hand limb movement data can be represented by 48 movement parameter features, each of which has a corresponding movement parameter feature value.

[0105] In step S302 of some embodiments, the limb parameterized mesh is obtained by piecing together multiple triangular facets. Once the limb movement is determined, a limb mesh can be constructed based on the limb movement data to obtain the limb parameterized mesh. The limb parameterized mesh can be unfolded into a planar mesh, where the position information of each pixel on the planar mesh corresponds one-to-one with the spatial position information of the limb parameterized mesh in three-dimensional space. Furthermore, the position information of each pixel on the planar mesh corresponds one-to-one with the position information of each pixel on a preset feature map; the position information of each pixel on the feature map is called the map position information. Therefore, the spatial position information and map position information corresponding to each vertex of the triangular facets can be obtained from the limb parameterized mesh, where the spatial position information corresponding to the vertex is called vertex spatial position information, and the map position information corresponding to the vertex is called vertex map position information.

[0106] In step S303 of some embodiments, the preset target triangle facet selection rule is as follows: First, the distance from the spatial sampling point to each triangle facet is obtained based on the spatial sampling point position information and the vertex spatial position, resulting in candidate distances. Then, the triangle facet is selected based on the shortest candidate distance. This selected triangle facet is the target triangle facet. Therefore, the target triangle facet is determined based on the vertex spatial position, the spatial sampling point position information, and the preset target triangle facet selection rule. The spatial position information of the target triangle facet is represented by the target vertex spatial position information, which includes the spatial position information of the three vertices of the target triangle.

[0107] In step S304 of some embodiments, each triangular facet of the limb parameterized mesh has a corresponding normal vector. Therefore, the normal vector corresponding to the target triangular facet can be determined, and the planar normal feature can be determined based on the normal vector.

[0108] In step S307 of some embodiments, texture coordinate features are obtained based on the target vertex spatial position information, vertex texture position information, and projection point spatial position information. Specifically, firstly, the spatial distance from the spatial projection point to the three vertices of the target triangle facet is determined based on the target vertex spatial position information and the projection point spatial position information, resulting in a weighted distance value. Each weighted distance value corresponds to a vertex of the target triangle facet. Therefore, the projection point spatial position information can be represented as a weighted sum of the target vertex spatial position information and its corresponding weighted distance value. Then, the vertex texture position information of the target triangle facet is obtained. Since the planar mesh is obtained by unfolding the limb parameterized mesh, the position information of each pixel on the planar mesh corresponds one-to-one with the spatial position information of the limb parameterized mesh in three-dimensional space, and the position information of each pixel on the planar mesh corresponds one-to-one with the position information of each pixel on the preset feature map. Therefore, the texture position information corresponding to the spatial projection point on the feature map can be determined based on the target vertex texture position information and the weighted distance value. Based on the texture position information corresponding to the spatial projection point on the feature map, texture coordinate features can be obtained.

[0109] In step S308 of some embodiments, each pixel on the feature map corresponds to a feature value, which represents the color information of each pixel on the feature map. Therefore, feature values ​​can be determined on a preset feature map based on the map coordinate features. Specifically, the feature value is a high-dimensional vector value, and the number of dimensions of the high-dimensional vector is determined according to the color information. The color information of the limb includes at least: the surface color information and roughness information of the limb, and can also be other color information commonly used by those skilled in the art. By using high-dimensional vector values ​​as feature values, the limb rendering model can consider multiple color information simultaneously during limb rendering, resulting in a more refined and realistic rendered image.

[0110] In step S309 of some embodiments, an observation direction feature can be determined based on the spatial sampling point location information and the virtual ray; wherein, the observation direction feature is used to describe the direction of the virtual ray where the spatial sampling point is located. Specifically, the virtual ray where the spatial sampling point is located is determined based on the spatial sampling point location, and the direction of the virtual ray is determined based on the virtual ray. The observation direction feature is determined based on the direction of the virtual ray.

[0111] In step S310 of some embodiments, motion parameter features, plane normal features, plane projection distance features, texture coordinate features, feature values, and viewing direction features are concatenated to obtain deformable coding information. The specific concatenation order can be selected according to the needs of those skilled in the art.

[0112] Steps S301 to S310 of this embodiment first determine motion parameter features based on limb motion data. Then, based on the limb parameterized mesh, obtain the vertex spatial position information and vertex texture position information corresponding to each vertex of the triangular facet, and determine the target triangular facet based on the vertex spatial position information, spatial sampling point position information, and a preset target triangular facet selection rule; wherein, the spatial position information of the target triangular facet is represented by the target vertex spatial position information. Next, determine the planar normal feature based on the target triangular facet, and project the spatial sampling point onto the target triangular facet based on the spatial sampling point position information, the target vertex spatial position information, and the planar normal feature to obtain the planar projection distance feature and the spatial projection point; wherein, the spatial position information of the projection point is determined based on the spatial projection point and the target vertex spatial position information. Subsequently, obtain the projection point spatial position information based on the spatial projection point, obtain the texture coordinate feature based on the target vertex spatial position information, vertex texture position information, and projection point spatial position information, and determine the feature value on the preset feature map based on the texture coordinate feature. The observation direction features are determined based on the spatial sampling point location information and virtual rays; the observation direction features describe the direction of the virtual ray where the spatial sampling point is located. Finally, the motion parameter features, plane normal features, plane projection distance features, texture coordinate features, feature values, and observation direction features are concatenated to obtain deformable coding information. The deformable coding information of this application simultaneously includes the location information features of the spatial sampling points (including the geometric information of motion parameter features, plane normal features, plane projection distance features, texture coordinate features, and observation direction features) and color information features (including feature values). Therefore, when rendering limbs using deformable coding information, the geometric and appearance information of the spatial sampling points can be modeled simultaneously, simplifying the rendering process and improving the efficiency of limb rendering.

[0113] Please see Figure 4 In some embodiments, prior to step S305, the limb rendering method further includes selecting a target sampling point from spatial sampling points. Specific steps for selecting the target sampling point from spatial sampling points include, but are not limited to, steps S401 to S402:

[0114] Step S401: Compare the distance with the preset distance threshold and the planar projection distance features to obtain the comparison result;

[0115] Step S402: Filter the spatial sampling points according to the comparison results to obtain the target sampling points.

[0116] In step S401 of some embodiments, a comparison is made between a preset distance threshold and a planar projection distance feature to obtain a comparison result. Specifically, the preset distance threshold can be determined according to the following rules: obtain the two farthest vertices on the limb parameterized mesh, calculate the distance between these two vertices to obtain the maximum mesh distance, and select one-tenth of the maximum mesh distance as the distance threshold. The distance threshold can also be adjusted according to the actual effect or determined according to other rules.

[0117] In step S402 of some embodiments, the spatial sampling points are filtered according to the comparison result to obtain the target sampling point. Specifically, if the comparison result indicates that the projected distance to the plane is less than a distance threshold, the spatial sampling point is retained as the target sampling point. All other spatial sampling points are deleted.

[0118] In steps S401 to S402 of some embodiments of this application, a comparison is first made between a preset distance threshold and a planar projection distance feature to obtain a comparison result. Then, the spatial sampling points are filtered based on the comparison result to obtain target sampling points. Therefore, this application selects appropriate target sampling points for encoding conversion based on a distance threshold, reducing the computational load of encoding conversion in the limb rendering method and thus improving the efficiency of limb rendering.

[0119] In step S104 of some embodiments, feature parameters are calculated on the deformable encoded information using a neural network to obtain spatial feature parameters for each spatial sampling point; wherein, the spatial feature parameters include: sampling point color information and sampling point density information. Specifically, the neural network is an eight-layer multilayer perceptron (MLP), and each layer of the MLP consists of a linear transformation module and a nonlinear transformation module.

[0120] Please see Figure 5 In some embodiments, step S105 includes, but is not limited to, steps S501 to S503:

[0121] Step S501: Determine the weight coefficients based on the sampling point density information and the preset weight mapping relationship table; wherein, the weight mapping relationship table includes the weight coefficients corresponding to the density information of each sampling point.

[0122] Step S502: Based on the weighting coefficients, sampling point opacity information, and virtual ray, the color information of the sampling points is weighted and summed to obtain the color information of the virtual ray; wherein, the sampling point opacity information is determined based on the sampling point density information.

[0123] Step S503: Construct a target limb rendering image based on virtual ray color information, virtual ray, mapping relationship, and planar pixels.

[0124] In step S501 of some embodiments, weight coefficients can be determined based on the sampling point density information and a preset weight mapping table; wherein, the weight mapping table includes the weight coefficient corresponding to each sampling point density information. Specifically, each weight coefficient in the weight mapping table is calculated from the nearest to the farthest sampling point density information, and the specific calculation process is as follows:

[0125]

[0126] Among them, T i σ represents the weight coefficient of the i-th spatial sampling point. j δ represents the density information of the j-th spatial sampling point. j δ represents the relative distance between the j-th spatial sampling point and the (j+1)-th spatial sampling point. When performing equidistant sampling on virtual rays, δ j The value of is 1.

[0127] In step S502 of some embodiments, the color information of the sampling points is weighted and summed according to the weighting coefficients, the opacity information of the sampling points, and the virtual ray to obtain the color information of the virtual ray; wherein, the opacity information of the sampling points is determined based on the sampling point density information. The specific process of weighting and summing the colors of each spatial sampling point on a virtual ray from near to far according to the weighting coefficients and the virtual ray to obtain the color information of the virtual ray is as follows:

[0128]

[0129] in, This represents the color information of a virtual ray, where N represents the total number of spatial sampling points on a virtual ray, and α... i c represents the sampling point opacity information of the i-th spatial sampling point. i This represents the color information of the i-th spatial sampling point.

[0130] As disclosed above, the sampling point opacity information α of each spatial sampling point... i It is obtained based on the sampling point density information and the relative magnitude of the distance between spatial sampling points, and the specific process is as follows:

[0131] α i =1-exp(σ i δ i );

[0132] Where, σ i δ represents the density information of the i-th spatial sampling point. i δ represents the relative distance between the i-th spatial sampling point and the (i+1)-th spatial sampling point. When performing equidistant sampling on virtual rays, δ i The value of is 1.

[0133] In step S503 of some embodiments, a target limb rendering image can be constructed based on the virtual ray color information, the virtual ray, the mapping relationship, and the planar pixels. Specifically, since there is a mapping relationship between the virtual ray and the planar pixels, the rendering color information of each planar pixel can be determined based on the virtual ray color information, the virtual ray, the mapping relationship, and the planar pixels. Therefore, a target limb rendering image can be constructed based on all planar pixels and their corresponding rendering color information.

[0134] Steps S501 to S503 of this embodiment first determine weight coefficients based on the sampling point density information and a preset weight mapping table; wherein the weight mapping table includes the weight coefficients corresponding to each sampling point density information. Then, based on the weight coefficients, sampling point opacity information, and virtual rays, the sampling point color information is weighted and summed to obtain the virtual ray color information; wherein the sampling point opacity information is determined based on the sampling point density information. Finally, a target limb rendering image is constructed based on the virtual ray color information, the virtual ray, the mapping relationship, and planar pixels. Through the above steps, the limb rendering method of this application can ultimately render a target limb rendering image with a high degree of color similarity to the real limb image.

[0135] Please see Figure 6 In some embodiments, the acquisition of internal parameters includes color channel parameters. After step S105, the method further includes, but is not limited to, steps S601 to S602:

[0136] Step S601: Correct the deviation of the color channel parameters according to the preset correction parameters to obtain the corrected color channel parameters;

[0137] Step S602: Correct the target limb rendering image according to the correction color channel parameters to obtain the limb rendering correction image.

[0138] In step S601 of some embodiments, the color channel parameters are corrected for deviation according to preset correction parameters to obtain corrected color channel parameters. Specifically, the color channel parameters may include, but are not limited to, red channel parameters, green channel parameters, and blue channel parameters; the correction parameters include gain parameters and deviation correction parameters. The color channel parameter process is as follows:

[0139] c′=c⊙g+b;

[0140] Where c′ is the feature vector of the corrected color channel parameter, c′ can be converted into the corrected color channel parameter, c is the feature vector of the color channel parameter obtained by vector conversion of the color channel parameter, g is the feature vector of the gain parameter obtained by vector conversion of the gain parameter, and b is the feature vector of the deviation correction parameter obtained by vector conversion of the deviation correction parameter. c′, c, g and b are all high-dimensional vectors with the same dimension, and ⊙ represents the multiplication of two features of the same dimension of two high-dimensional feature vectors.

[0141] In step S602 of some embodiments, the target limb rendering image is corrected according to the corrected color channel parameters to obtain a limb rendering corrected image. Specifically, the rendering color information of each planar pixel is corrected according to the corrected color channel parameters to obtain corrected rendering color information. The limb rendering corrected image is constructed based on all planar pixels and their corresponding corrected rendering color information.

[0142] In steps S601 to S602 of this embodiment, the color channel parameters are first corrected according to preset correction parameters to obtain corrected color channel parameters. Then, the target limb rendering image is corrected according to the corrected color channel parameters to obtain a corrected limb rendering image. Therefore, the limb rendering method of this application can reduce the color information error of the target limb rendering image caused by the difference in the internal parameters of the acquisition device by correcting the target limb rendering image.

[0143] Please see Figure 7 In some embodiments, after step S105, the limb rendering method further includes optimizing the rendering model. Specific steps for optimizing the rendering model include, but are not limited to, steps S701 to S704:

[0144] Step S701: Construct a loss function based on the preset limb verification image and the target limb rendering image;

[0145] Step S702: Perform backpropagation calculation on the loss function to obtain the gradient of the optimized parameters;

[0146] Step S703: Update the preset optimizable parameters by gradient descent according to the gradient of the optimization parameters to obtain the target optimizable parameters; wherein, the optimizable parameters include eigenvalues ​​and correction parameters;

[0147] Step S704: Optimize the rendering model based on the target optimizable parameters.

[0148] In step S701 of some embodiments, a loss function is constructed based on a preset limb verification image and a target limb rendering image. The specific process of constructing the loss function is as follows:

[0149] First, a photometric loss function is constructed based on the photometric values ​​of the target limb rendered image and the limb verification image:

[0150]

[0151] in, I represents the photometric loss function. pred I represents the luminosity of the rendered image of the target limb. gt Indicates the luminosity of the limb verification image;

[0152] Then, based on the feature values ​​of the target limb rendered image and the limb verification image, a Learned Perceptual Image Patch Simulation (LPIPS) loss function is constructed:

[0153]

[0154] in, Let f(I) represent the loss function for learning perceptual image patch similarity. pred f(I) represents the feature value of the rendered image of the target limb. gt ) represents the feature value of the limb verification image;

[0155] Construct a total loss function based on the photometric loss function and the learned perceptual image patch similarity loss function:

[0156]

[0157] in, This indicates that λ1 is 0.2.

[0158] In step S703 of some embodiments, updating the preset optimizable parameters using gradient descent based on the optimization parameter gradient yields the target optimizable parameters; these optimizable parameters include feature values ​​and correction parameters. Specifically, updating the feature values ​​allows the rendering model to obtain refined feature values ​​reflecting multiple detailed features of different parts of the limb after training, thereby enhancing the model's expressive power and ultimately making the target limb rendered image more realistic. Updating the correction parameters allows the final corrected parameters to better reduce color information errors in the target limb rendered image caused by differences in the internal parameters of the acquisition device, ultimately improving the realism of the target limb rendered image.

[0159] Steps S701 to S704 of this embodiment first construct a loss function based on a preset limb verification image and a target limb rendering image, and then perform backpropagation calculation on the loss function to obtain the gradient of the optimized parameters. Next, gradient descent is performed on the preset optimizable parameters according to the gradient of the optimized parameters to obtain the target optimizable parameters; wherein, the optimizable parameters include eigenvalues ​​and correction parameters. Finally, the rendering model is optimized based on the target optimizable parameters. It can be seen that the limb rendering method of this application obtains the target optimizable parameters by optimizing the optimizable parameters, and continuously optimizes the rendering model through the target optimizable parameters, so that the final rendering model has higher rendering accuracy and renders a higher quality target limb rendering image.

[0160] Please see Figure 8 , Figure 8 This is a schematic diagram of an optional overall architecture of the limb rendering method provided in the embodiments of this application.

[0161] like Figure 8As shown, in some embodiments, the limb rendering method provided in this application is used for rendering human hand images. The specific human hand image rendering process is as follows: First, human hand motion data (i.e., limb motion data of the target object) and virtual camera parameters (i.e., acquisition parameters of limb motion data) are acquired. Then, the human hand motion data and virtual camera parameters are input into a preset human hand rendering model. The human hand rendering model includes a human hand mesh sub-model (i.e., limb mesh sub-model) and an MLP (i.e., neural network). The virtual camera parameters include camera position information and camera internal parameter information. The human hand mesh sub-model includes a sampling network and a human hand mesh construction network (i.e., limb mesh construction network). Then, a plane is constructed through the sampling network, camera position information, and camera internal parameter information to obtain an imaging plane image. The imaging plane image includes multiple planar pixels. Each planar pixel is back-projected into three-dimensional space through the sampling network and a preset mapping relationship to obtain virtual rays corresponding to each planar pixel. Subsequently, each virtual ray is sampled at equal intervals through the sampling network to obtain multiple sampling space points (i.e., spatial sampling points). The position information of each sampling space point is obtained through the sampling network to obtain the sampling space point position information. Next, a parametric mesh is constructed from the hand motion data using a hand mesh construction network, and the sampling spatial points are projected onto the parametric mesh. Then, based on the parametric mesh, hand motion data, and sampling spatial point position information, encoding transformation is performed to obtain deformable codes (i.e., deformable code information). These deformable codes include motion parameters (i.e., motion parameter features), triangular patch normals (i.e., planar normal features), projection distances (i.e., planar projection distance features), texture coordinates (i.e., texture coordinate features), texture feature values ​​(i.e., feature values), and viewing direction (i.e., viewing direction features). Finally, the deformable codes are processed using an MLP to calculate feature parameters, obtaining the color (i.e., sampling point color information) and density (i.e., sampling point density information) of each sampling spatial point. Volume rendering processing (i.e., rendering processing) is then performed on the color and density to obtain a rendered image of the hand (i.e., the target limb rendered image).

[0162] Please see Figure 9 This application also provides a limb rendering system that can implement the above-described limb rendering method. The system includes:

[0163] The parameter acquisition module 901 is used to acquire the limb movement data of the target object and the acquisition parameters of the limb movement data; wherein, the acquisition parameters are the operating parameters of the acquisition device for acquiring limb movement data;

[0164] The parameter input module 902 is used to input limb motion data and collected parameters into a preset rendering model; wherein, the rendering model includes: a limb mesh sub-model and a neural network;

[0165] The encoding conversion module 903 is used to encode and convert limb motion data and acquisition parameters through the limb mesh sub-model to obtain deformable encoded information; wherein, the deformable encoded information represents the spatial characteristics of each spatial sampling point on the target object's limb in three-dimensional space;

[0166] The spatial feature parameter calculation module 904 is used to calculate the feature parameters of deformable encoded information through a neural network to obtain the spatial feature parameters of each spatial sampling point; wherein, the spatial feature parameters include: sampling point color information and sampling point density information;

[0167] The image rendering module 905 is used to render the color information and density information of the sampling points to obtain the rendered image of the target limb.

[0168] The specific implementation of this limb rendering system is basically the same as the specific implementation of the limb rendering method described above, and will not be repeated here.

[0169] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described limb rendering method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0170] Please see Figure 10 , Figure 10 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0171] The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0172] The memory 1002 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 using the limb rendering method of the embodiments of this application.

[0173] Input / output interface 1003 is used to implement information input and output;

[0174] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0175] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0176] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described limb rendering method.

[0178] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0179] The limb rendering method, limb rendering system, electronic device, and storage medium provided in this application embodiment acquire limb movement data and acquisition parameters of the target object, and input the limb movement data and acquisition parameters into a preset rendering model for limb rendering. The rendering model includes a limb mesh sub-model and a neural network. The rendering model first encodes and converts the limb movement data and acquisition parameters through the limb mesh sub-model to obtain deformable encoded information. Then, it calculates feature parameters of the deformable encoded information through the neural network to obtain spatial feature parameters of each spatial sampling point. The spatial feature parameters include sampling point color information and sampling point density information. Finally, it performs rendering processing on the sampling point color information and sampling point density information to obtain a rendered image of the target limb. Therefore, the limb rendering method, system, electronic device, and storage medium proposed in this application can directly utilize a parametric mesh model to convert limb motion data and acquisition parameters into deformable coded information from the limb mesh sub-model. Then, the spatial feature parameters of the spatial sampling points can be calculated using the deformable coded information, and finally, image rendering can be performed through volume rendering. This enables rapid rendering of limb images without the need for multiple modeling steps of the parametric limb mesh model, which separately model the geometric and appearance information of the limb. This simplifies the limb rendering process, shortens the time required for human hand modeling and rendering, and improves the quality and efficiency of limb rendering.

[0180] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0181] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0182] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0184] The terms “comprising” and “having”, and any variations thereof, in the specification and accompanying drawings of this application are intended to cover non-exclusive inclusion, such that a process, method, system, or apparatus that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0185] It should be understood that in this application, "at least one (item)" means one or more, and "more than one" means two or more. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0186] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0187] The modules described above as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0188] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0189] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0190] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method of rendering a limb, characterized by, The method comprises: acquiring limb action data of a target object and acquisition parameters of the limb action data; wherein the acquisition parameters are operating parameters of an acquisition device for acquiring the limb action data; inputting the limb action data and the acquisition parameters into a preset rendering model; wherein the rendering model comprises a limb mesh submodel and a neural network; performing encoding conversion on the limb action data and the acquisition parameters through the limb mesh submodel to obtain deformable encoding information; wherein the deformable encoding information represents spatial features of each spatial sampling point on the limbs of the target object in three-dimensional space; performing feature parameter calculation on the deformable encoding information through the neural network to obtain spatial feature parameters of each spatial sampling point; wherein the spatial feature parameters comprise sampling point color information and sampling point density information; performing rendering processing on the sampling point color information and the sampling point density information to obtain a target limb rendering image; the acquisition parameters comprise acquisition position information and acquisition internal parameters; and the limb mesh submodel comprises a sampling network and a limb mesh construction network; the encoding conversion on the limb action data and the acquisition parameters through the limb mesh submodel to obtain deformable encoding information comprises: performing plane construction through the sampling network, the acquisition position information and the acquisition internal parameters to obtain an imaging plane image; wherein the imaging plane image comprises a plurality of plane pixels; projecting each plane pixel into three-dimensional space through the sampling network and a preset mapping relationship to obtain a virtual light ray corresponding to each plane pixel; performing equidistance sampling on each virtual light ray through the sampling network to obtain a plurality of spatial sampling points; obtaining position information of each spatial sampling point through the sampling network to obtain spatial sampling point position information; performing limb mesh construction on the limb action data through the limb mesh construction network to obtain a limb parameterized mesh; performing encoding conversion on the limb parameterized mesh, the limb action data and the spatial sampling point position information to obtain the deformable encoding information.

2. The method of claim 1, wherein, the limb parameterized mesh is composed of a plurality of triangular patches, and the encoding conversion on the limb parameterized mesh, the limb action data and the spatial sampling point position information to obtain the deformable encoding information comprises: determining action parameter features according to the limb action data; obtaining vertex spatial position information and vertex map position information corresponding to each vertex of the triangular patches according to the limb parameterized mesh; determining a target triangular patch according to the vertex spatial position information, the spatial sampling point position information and a preset target triangular patch selection rule; wherein the spatial position information of the target triangular patch is represented by target vertex spatial position information; determining plane normal features according to the target triangular patch; projecting a target sampling point to the target triangular patch according to the spatial sampling point position information, the target vertex spatial position information, and the plane normal feature, to obtain a plane projection distance feature and a spatial projection point; wherein the target sampling point is selected based on the spatial sampling point; acquiring position information of the spatial projection point to obtain projection point spatial position information; acquiring a texture feature of the triangular patch according to the target vertex spatial position information, the vertex texture position information, and the projection point spatial position information to obtain a texture coordinate feature; determining a feature value on a preset feature texture according to the texture coordinate feature; determining an observation direction feature according to the spatial sampling point position information and the virtual light ray; wherein the observation direction feature is used to describe a direction of the virtual light ray where the spatial sampling point is located; concatenating the action parameter feature, the plane normal feature, the plane projection distance feature, the texture coordinate feature, the feature value, and the observation direction feature to obtain the deformable encoding information.

3. The method of claim 2, wherein, Before the step of projecting the target sampling point to the target triangular patch according to the spatial sampling point position information, the target vertex spatial position information, and the plane normal feature, to obtain a plane projection distance feature and a spatial projection point, the method further comprises: selecting the target sampling point from the spatial sampling point, specifically including: comparing a preset distance threshold and the plane projection distance feature to obtain a comparison result; performing filtering processing on the spatial sampling point according to the comparison result to obtain a target sampling point.

4. The method of claim 3, wherein, The rendering processing on the sampling point color information and the sampling point density information to obtain a target limb rendering image comprises: determining a weight coefficient according to the sampling point density information and a preset weight mapping relationship table; wherein the weight mapping relationship table includes the weight coefficient corresponding to each sampling point density information; performing weighted summation on the sampling point color information according to the weight coefficient, sampling point opacity information, and the virtual light ray to obtain virtual light ray color information; wherein the sampling point opacity information is determined according to the sampling point density information; constructing the target limb rendering image according to the virtual light ray color information, the virtual light ray, the mapping relationship, and the plane pixel.

5. The method of claim 4, wherein, The collection internal parameter includes a color channel parameter, and after the rendering processing on the sampling point color information and the sampling point density information to obtain a target limb rendering image, the method further comprises: performing deviation correction on the color channel parameter according to a preset correction parameter to obtain a corrected color channel parameter; performing correction on the target limb rendering image according to the corrected color channel parameter to obtain a limb rendering correction image.

6. The method of claim 5, wherein, After the step of inputting the limb action data and the collection parameter into a preset rendering model, the method further comprises: optimizing the rendering model, specifically including: constructing a loss function based on a preset limb verification image and the target limb rendering image; performing back propagation calculation on the loss function to obtain an optimization parameter gradient; According to the optimization parameter gradient, a preset optimization parameter is updated by gradient descent to obtain a target optimization parameter; wherein the optimization parameter includes the eigenvalue and the correction parameter; According to the target optimization parameter, the rendering model is optimized.

7. A system for rendering a limb, characterized by The system comprises: A parameter acquisition module is configured to acquire limb action data of a target object and acquisition parameters of the limb action data; wherein the acquisition parameters are operation parameters of an acquisition device used to acquire the limb action data; A parameter input module is configured to input the limb action data and the acquisition parameters to a preset rendering model; wherein the rendering model comprises a limb mesh sub-model and a neural network; An encoding conversion module is configured to perform encoding conversion on the limb action data and the acquisition parameters by using the limb mesh sub-model to obtain deformable encoding information; wherein the deformable encoding information represents spatial features of each spatial sampling point on a limb of the target object in a three-dimensional space; the acquisition parameters include acquisition position information and acquisition internal parameters; the limb mesh sub-model comprises a sampling network and a limb mesh construction network; the encoding conversion on the limb action data and the acquisition parameters by using the limb mesh sub-model to obtain the deformable encoding information comprises: performing plane construction by using the sampling network, the acquisition position information, and the acquisition internal parameters to obtain an imaging plane image; wherein the imaging plane image comprises a plurality of plane pixels; projecting each plane pixel to a three-dimensional space by using the sampling network and a preset mapping relationship to obtain a virtual light ray corresponding to each plane pixel; performing equidistance sampling on each virtual light ray by using the sampling network to obtain a plurality of spatial sampling points; obtaining position information of each spatial sampling point by using the sampling network to obtain spatial sampling point position information; performing limb mesh construction on the limb action data by using the limb mesh construction network to obtain a limb parameterized mesh; and performing encoding conversion on the limb parameterized mesh, the limb action data, and the spatial sampling point position information to obtain the deformable encoding information; A spatial feature parameter calculation module is configured to perform feature parameter calculation on the deformable encoding information by using the neural network to obtain spatial feature parameters of each spatial sampling point; wherein the spatial feature parameters include sampling point color information and sampling point density information; An image rendering module is configured to perform rendering processing on the sampling point color information and the sampling point density information to obtain a target limb rendering image.

8. An electronic device, comprising: The electronic device comprises a memory and a processor; the memory stores a computer program; and the processor implements the limb rendering method of any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program, when executed by the processor, implements the limb rendering method of any one of claims 1 to 6.

Images