Human body three-dimensional point cloud adjustment method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a human body three-dimensional point cloud adjustment method and device, electronic equipment and a storage medium. According to the method, the structural information of the SMPLH model is fully utilized, the vertex displacement and the skin weight of the SMPLH model are assigned to the scanned human body model, efficient deformation of the human body scanning point cloud is achieved, bone binding and posture adjustment of the scanned model can be achieved, the human body shape can be directly changed, and the human body shape can be adjusted. Therefore, free adjustment of the scanning model is realized, professional skill requirements of users are reduced, and working efficiency is improved. In addition, the method can also be applied to the fields of computer graphics, animation production, virtual reality and the like, and more possibilities are provided for applications of personalized role generation, reality enhancement and the like.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic equipment and storage medium for adjusting a three-dimensional point cloud of a human body. Background Art

[0002] Human 3D scanning and modeling technology has made significant progress. The main 3D human body scanning methods include using a depth camera to obtain point cloud data and using a multi-view camera to take static photos and then perform 3D reconstruction. The depth camera method requires expensive equipment, while the multi-view camera method uses the MVS (Multi View Stereo) algorithm or Metashape software to reconstruct the 3D model of the human body. These methods can generate point cloud data of the human body and convert the point cloud into a mesh model through triangulation.

[0003] However, the current methods for automatically adjusting the human body 3D model mainly focus on the skeleton binding and weight brushing of the human body 3D model. In these technologies, professionals are usually required to manually bind the skeleton and brush the weights of the scanned human body 3D model in order to adjust the model pose. However, these methods are often complicated and time-consuming, and require high professional skills from the user. And there are relatively few methods for directly adjusting the human body shape. Although some methods based on blend shapes can achieve local adjustment of the human body shape, these methods often rely on pre-defined shape changes and cannot achieve free adjustment of any scanned model. Summary of the invention

[0004] The embodiments of the present invention aim to provide a method, an adjustment device, an electronic device and a storage medium for adjusting a three-dimensional point cloud of a human body, so as to solve the technical problem in the prior art that free adjustment of any scanned model cannot be achieved.

[0005] The embodiment of the present invention solves the technical problem by adopting the following technical solution:

[0006] A method for adjusting a three-dimensional point cloud of a human body is provided, comprising the following steps:

[0007] Get the initial point cloud;

[0008] Matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created three-dimensional human body data set according to a PCA algorithm;

[0009] The matched initial point cloud is deformed to obtain the target point cloud.

[0010] In some embodiments, matching the initial point cloud with a pre-created SMPLH model includes:

[0011] Preprocessing the initial point cloud to obtain a preprocessed point cloud;

[0012] Extract key points from the preprocessed point cloud to determine the posture of the human body;

[0013] Based on the human body posture, setting an implicit function between the preprocessed point cloud and the SMPLH model, wherein the implicit function is obtained by determining a corresponding relationship between the initial point cloud and the SMPLH model based on the human body posture;

[0014] Based on the implicit function, the preprocessed point cloud is matched with the SMPLH model to obtain a matched initial point cloud.

[0015] In some embodiments, before the step of matching the initial point cloud with the pre-created SMPLH model, the step further includes:

[0016] Obtain human body point cloud scanning data;

[0017] Creating the human body three-dimensional data set based on the human body point cloud scanning data;

[0018] Based on the human body three-dimensional data set, the parameters of the SMPLH model are calculated according to the PCA algorithm.

[0019] In some embodiments, the step of calculating the parameters of the SMPLH model based on the human body three-dimensional data set according to the PCA algorithm includes:

[0020] Preprocessing the human body three-dimensional data set to obtain a target data set;

[0021] Aligning the target data set with the initial model through the ICP algorithm to determine the model shape and posture parameters;

[0022] Based on the preset principal component analysis strategy, the PCA algorithm is used to perform dimension reduction processing on the model shape and posture parameters to obtain the parameters of the SMPLH model.

[0023] In some embodiments, the deforming the matched initial point cloud includes:

[0024] Performing a neighborhood search in the matched initial point cloud to determine the nearest neighbor point, wherein the nearest neighbor point includes the nearest neighbor point of each point in the matched initial point cloud within a preset range;

[0025] Calculate the weight value of the nearest neighbor point;

[0026] Based on the weight value, each point in the matched initial point cloud is interpolated and deformed to obtain the target point cloud.

[0027] In some embodiments, calculating the weight value of the nearest neighbor point specifically includes:

[0028] w_i=1 / d_i

[0029] Wherein, w_i is the weight value of the i-th neighboring point, and d_i is the distance between the point in the point cloud and the i-th neighboring point;

[0030] After that, each calculated weight value is normalized:

[0031] w_i_normalized = w_i / sum(w_1, w_2, ..., w_k), where w_i_normalized is the result of normalizing the weight values ​​of i adjacent points.

[0032] In some embodiments, performing interpolation deformation processing on each point in the matched initial point cloud based on the weight value includes:

[0033] Suppose a point P in the point cloud has k nearest neighbor points in the previous SMPLH mesh as {P1_before, P2_before, ..., Pk_before}, and k nearest neighbor points in the next SMPLH mesh as {P1_after, P2_after, ..., Pk_after}, then the position of point P after interpolation deformation is P_after:

[0034] P_after=P+sum(w_i_normalized*(P_i_after-P_i_before)for iin range(1,k+1))

[0035] Among them, w_i_normalized is the normalized weight of the i-th neighboring point, and (P_i_after-P_i_before) represents the displacement of the i-th neighboring point in the two SMPLH meshes before and after.

[0036] The present invention further provides a device for adjusting a three-dimensional point cloud of a human body, comprising:

[0037] An initial point cloud acquisition module is used to acquire the initial point cloud;

[0038] A model matching module, used for matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created three-dimensional human body data set according to a PCA algorithm;

[0039] The deformation adjustment module is used to deform the initial point cloud after matching to obtain the target point cloud.

[0040] Another aspect of the present invention further provides an electronic device, the electronic device comprising:

[0041] a memory having computer instructions stored thereon; and

[0042] A processor is connected to the memory and is configured to execute computer instructions stored in the memory to implement the method for adjusting the three-dimensional point cloud of a human body as described above.

[0043] Another aspect of the present invention further provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the method for adjusting the three-dimensional point cloud of a human body as described above is implemented.

[0044] Compared with the prior art, in the adjustment method, adjustment device, electronic device and storage medium of a human body three-dimensional point cloud provided by the embodiment of the present invention, the present invention makes full use of the structural information of the SMPLH model, and by assigning the vertex displacement and skin weight of the SMPLH model to the scanned human body model, the efficient deformation of the human body scan point cloud is achieved. The present invention can not only realize the skeleton binding and posture adjustment of the scanned model, but also directly change the human body shape, thereby realizing the free adjustment of the scanned model, reducing the professional skill requirements of the user and improving the work efficiency. In addition, the present invention can also be applied to the fields of computer graphics, animation production, virtual reality, etc., providing more possibilities for applications such as personalized character generation and enhanced sense of reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0046] Figure 1 Schematic diagram of the structure of a method for adjusting a three-dimensional point cloud of a human body provided by an embodiment of the present invention;

[0047] Figure 2 It is a structural block diagram of interactive information in a method for adjusting a three-dimensional point cloud of a human body provided by another embodiment of the present invention;

[0048] Figure 3is a detailed flow chart of a method for adjusting a three-dimensional point cloud of a human body provided by another embodiment of the present invention;

[0049] Figure 4 This is another detailed flow chart of a method for adjusting a three-dimensional point cloud of a human body provided by yet another embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0051] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0052] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0053] The method for deforming a three-dimensional point cloud of a human body scanned by the present invention can be used for three-dimensional scanning and modeling of a human body.

[0054] See also Figures 1 to 4 In one embodiment, the present invention provides a method for adjusting a three-dimensional point cloud of a human body, comprising the following steps:

[0055] S1: Get the initial point cloud;

[0056] S2: matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created human body three-dimensional data set according to a PCA algorithm;

[0057] S3: Deform the matched initial point cloud to obtain the target point cloud.

[0058] Among them, a common method for obtaining the initial point cloud is to use a depth camera to perform a three-dimensional scan of the human body to obtain point cloud data. The depth camera uses infrared light or structured light technology to obtain the depth information of objects in the scene, and then generates point cloud data to obtain the initial point cloud.

[0059] More accurate human body modeling is achieved by matching the initial point cloud with the pre-created SMPLH model. The SMPLH model is obtained by processing the data set of human point cloud, which means that the model has learned a lot of information about human body shape and posture. During the matching process, an optimization algorithm can be used to adjust the parameters of the SMPLH model so that the human body shape generated by the model is more consistent with the initial point cloud data.

[0060] Reference Figure 2 In some embodiments, matching the initial point cloud with a pre-created SMPLH model includes:

[0061] S21: preprocessing the initial point cloud to obtain a preprocessed point cloud;

[0062] S22: extracting key points from the preprocessed point cloud to determine the human body posture;

[0063] S23: Based on the human body posture, an implicit function between the preprocessed point cloud and the SMPLH model is set, wherein the implicit function is obtained by determining a corresponding relationship between the initial point cloud and the SMPLH model based on the human body posture;

[0064] S24: Based on the implicit function, the preprocessed point cloud is matched with the SMPLH model to obtain the matched initial point cloud.

[0065] When preprocessing the initial point cloud, you can use open source libraries such as PCL (PointCloud Library) to implement it:

[0066] These include operations such as Noise Removal, Hole Filling, Smoothing, and Resampling.

[0067] Noise Removal: Noise points may be introduced during the scanning process, so these noise points need to be removed to purify the point cloud data. PCL provides a variety of denoising algorithms, such as Statistical Outlier Removal and Radius Outlier Removal. These algorithms can detect and remove noise points based on the neighborhood information around the point.

[0068] Hole Filling: During the scanning process, some missing points may appear, resulting in holes in the point cloud. Using PCL's hole filling algorithms, such as MovingLeastSquares and Poisson hole filling, the point positions of the missing areas can be estimated based on the information of the surrounding points and filled.

[0069] Smoothing: Point cloud data may have some rough surfaces during scanning, and smoothing operations are required to obtain a more continuous and smoother point cloud surface. Smoothing algorithms in PCL, such as MovingLeastSquares and Bilateral filter, can smooth point clouds by weighted averaging or Gaussian filtering the neighborhood around the point.

[0070] Resampling: Sometimes the density of point cloud data may be uneven. The density and uniformity of point cloud data can be adjusted through resampling. The Voxel Grid filter in PCL can split the point cloud data into voxels and downsample the points in each voxel to obtain uniform point cloud data.

[0071] You can use the API interface of PCL to call the corresponding functional module in the program to perform preprocessing operations on the initial point cloud. The specific implementation steps may include: reading, visualizing and loading point cloud data, selecting appropriate preprocessing algorithms, setting parameters (such as filter window size, sampling accuracy, etc.), applying the corresponding algorithms, and finally saving or displaying the processed point cloud data. In practical applications, appropriate preprocessing operations can be selected according to the characteristics of point cloud data and actual needs to improve the quality and applicability of point cloud data. In addition, PCL also provides other functions such as feature extraction, registration, segmentation, etc., which can be used in combination as needed.

[0072] In order to match the initial point cloud with the SMPLH model, the human body posture needs to be estimated first. In this process, some key point-based posture estimation methods can be used, such as OpenPose and AlphaPose. These methods can extract the key point information of the human body from the input image or video. Key points are important markers of various parts of the human body, including the head, torso, limbs, etc. By identifying and tracking these key points, we can obtain the posture information of the human body. Here, we pay special attention to the joint angles because they are very important for defining the posture of the human body. Once we have the key point information of the human body, we can use the Inverse Kinematics (IK) method to solve the joint angles. IK is a mathematical algorithm that can calculate the joint angles that make the end effector reach the specified position based on the given end effector position (such as the position of the hand or foot) and the joint constraints. Here, we hope to obtain the posture parameters of the SMPLH model through IK. The SMPLH model is a parameterized model used to describe the human body posture, which can generate the corresponding human body posture based on the given posture parameters. Therefore, by solving the IK problem, we can obtain the pose parameters of the SMPLH model and thus match the point cloud with the model.

[0073] It should be noted that some details and scenarios need to be taken into account in this process. For example, we need to effectively identify and track key points to ensure accurate key point information. At the same time, in order to obtain the correct joint angle, we need to consider the range of motion and restrictions of the joints, as well as possible uncertainties and noise.

[0074] Implicit function is a method that can learn the distance relationship between points and the surface of an object in three-dimensional space. In the specific implementation, in order to realize the mapping relationship between the point cloud and the SMPLH model, the implicit function can be used to represent the distance relationship between each point in the point cloud and the surface of the model. By learning this distance function, a continuous signed distance function can be obtained to describe the topological and geometric relationship between the point cloud and the model.

[0075] The matching of the preprocessed point cloud and the SMPLH model is realized through implicit functions. The specific implementation steps of obtaining the matched initial point cloud are as follows:

[0076] Data preprocessing: First, the collected raw point cloud data needs to be preprocessed. This may include point cloud noise reduction, filtering, sampling and other steps to improve the quality and accuracy of the data. The preprocessed point cloud data will be used as input for the subsequent matching process.

[0077] Implicit function learning: Determine an implicit function model that can take the input point cloud coordinates as input and output the signed distance between the point and the SMPLH model surface.

[0078] Matching process: For the preprocessed point cloud data, the coordinates of each point are passed as input to the determined implicit function model. By calculating the signed distance output by the implicit function model, the position of each point on the surface of the SMPLH model can be determined.

[0079] Optimization and iteration: The matching process usually does not completely obtain accurate results, so optimization and iteration may be required to further improve the accuracy of the matching. This can be achieved by minimizing the distance between the point cloud points and the SMPLH model surface, which can be solved by an optimization algorithm such as the least squares method or an optimizer.

[0080] Initial point cloud acquisition: Through the above matching process and optimization iteration, the initial point cloud after matching can be obtained. The topological and geometric relationship between the initial point cloud and the surface of the SMPLH model can be mapped to a certain extent.

[0081] In some embodiments, before the step of matching the initial point cloud with the pre-created SMPLH model, the step also includes: acquiring human body point cloud scanning data; creating the human body three-dimensional data set based on the human body point cloud scanning data; and calculating the parameters of the SMPLH model according to the PCA algorithm based on the human body three-dimensional data set.

[0082] First, it is necessary to collect a data set containing diverse human postures and shapes. This data set can include diverse human data of different heights, body shapes, ages, etc., as well as corresponding joint angles and posture parameters. These data can be obtained through scanners, cameras or other data acquisition technologies. The SMPLH model is a parametric model used to describe human posture and shape. It consists of a posture parameter and a shape parameter, which can generate the corresponding human posture and shape. After that, parameter optimization is required to enable the SMPLH model to better fit the real human posture and shape. Through the above process, the SMPLH model can be obtained, which can generate the corresponding human posture and shape according to the given posture parameters and shape parameters. In the subsequent matching process, the initial point cloud can be matched with the obtained SMPLH model to obtain the matched initial point cloud

[0083] Reference Figure 3 In some embodiments, based on the three-dimensional human body data set, the parameters of the SMPLH model are calculated according to the PCA algorithm, including:

[0084] A21: preprocessing the human body three-dimensional data set to obtain a target data set;

[0085] A22: Align the target data set with the initial model through the ICP algorithm to determine the model shape and posture parameters;

[0086] A23: Based on the preset principal component analysis strategy, the PCA algorithm is used to reduce the dimension of the model shape and posture parameters to obtain the parameters of the SMPLH model.

[0087] Dataset preprocessing: When calculating the parameters of the SMPLH model, the human point cloud dataset needs to be preprocessed to obtain the target dataset. The preprocessing steps may include point cloud denoising, filtering, sampling, and outlier removal to improve the quality and accuracy of the data.

[0088] ICP Alignment: A common method is to use the Iterative Closest Point (ICP) algorithm to align the target dataset with the initial model. The ICP algorithm can determine the shape and pose parameters of the model by minimizing the distance between the points in the target dataset and the initial model. Through iterative optimization, the aligned model can be obtained.

[0089] Dimensionality reduction using principal component analysis (PCA): To reduce the parameter dimensionality and complexity of the model, the shape and posture parameters of the model can be reduced using a preset principal component analysis strategy. Principal component analysis is a statistical technique that can extract the main variation patterns from high-dimensional data and reduce their dimensionality. By performing principal component analysis on the shape and posture parameters of the model, a set of lower dimensional parameters can be obtained to represent the SMPLH model.

[0090] Reference Figure 4 In some embodiments, the deformation processing of the matched initial point cloud includes:

[0091] S31: performing a neighborhood search in the matched initial point cloud to determine the nearest neighbor point, wherein the nearest neighbor point includes the nearest neighbor point of each point in the matched initial point cloud within a preset range;

[0092] S32: Calculate the weight value of the nearest neighbor point;

[0093] S33: Based on the weight value, each point in the matched initial point cloud is interpolated and deformed to obtain the target point cloud.

[0094] Neighborhood search: In the initial point cloud after matching, the nearest neighbor of each point can be determined using the neighborhood search method. The nearest neighbor is the point closest to each point within a preset range, and the distance between points is usually measured using Euclidean distance or other similarity metrics.

[0095] Specifically, first, we need to find the k nearest neighbor points on the SMPLH mesh for each point in the initial point cloud corresponding to the matched SMPLH model. Neighborhood search usually involves two data structures: KD-Tree and Ball-Tree.

[0096] KD-Tree is a data structure that divides the data point space into several subspaces, and the points in each subspace are close to each other. The construction process of KD-Tree is as follows:

[0097] Choose a dividing axis, usually the axis with the largest variance of the data points.

[0098] Divide the data points into two parts according to the median on the dividing axis.

[0099] Recursively repeat steps 1 and 2 for the left and right subspaces until the number of points in the subspace is less than a given threshold.

[0100] Ball-Tree is another data structure used for nearest neighbor search. Unlike KD-Tree, Ball-Tree encloses the data points in a hypersphere and then divides the hypersphere into two sub-spheres. The construction process of Ball-Tree is as follows:

[0101] Compute the centers of the data points and the smallest hypersphere that encloses all the points.

[0102] Choose a dividing axis, usually the axis with the largest variance of the data points.

[0103] Divide the data points into two parts according to the median on the dividing axis.

[0104] Recursively repeat steps 1 to 3 for the left and right subspaces until the number of points in the subspace is less than a given threshold.

[0105] KD-Tree or Ball-Tree can be used to speed up the neighborhood search process. During the search process, we can use the characteristics of these data structures to quickly find the k nearest neighbors of each point.

[0106] Weight calculation: In order to perform interpolation deformation processing, it is necessary to calculate the weight values ​​of the nearest neighbor points. These weight values ​​can be determined based on the distance between the nearest neighbor points and other similarity measures.

[0107] The present invention adopts the reciprocal of the distance as the weight, that is, the weight is inversely proportional to the distance.

[0108] Interpolation deformation processing: Based on the calculated weight value, each point in the matched initial point cloud can be interpolated and deformed. Interpolation deformation is a method of adjusting the position and shape of a point based on the weight of the nearest neighbor point. By performing weighted averaging or other interpolation methods on each point, the target point cloud, that is, the point cloud after deformation processing, can be obtained.

[0109] The neighborhood search method can select an appropriate algorithm based on the density and distribution of the point cloud to ensure that a valid nearest neighbor point is found. The weight calculation method can also be adjusted according to demand to obtain a suitable weight distribution. For interpolation deformation processing, different interpolation functions and parameters can be used to obtain different deformation effects.

[0110] In some embodiments, calculating the weight value of the nearest neighbor point specifically includes:

[0111] w_i=1 / d_i

[0112] Wherein, w_i is the weight value of the i-th neighboring point, and d_i is the distance between the point in the point cloud and the i-th neighboring point;

[0113] After that, each calculated weight value is normalized:

[0114] w_i_normalized = w_i / sum(w_1, w_2, ..., w_k), where w_i_normalized is the result of normalizing the weight values ​​of i adjacent points.

[0115] In some embodiments, performing interpolation deformation processing on each point in the matched initial point cloud based on the weight value includes:

[0116] Suppose a point P in the point cloud has k nearest neighbor points in the previous SMPLH mesh as {P1_before, P2_before, ..., Pk_before}, and k nearest neighbor points in the next SMPLH mesh as {P1_after, P2_after, ..., Pk_after}, then the position of point P after interpolation deformation is P_after:

[0117] P_after=P+sum(w_i_normalized*(P_i_after-P_i_before)for iin range(1,k+1))

[0118] Among them, w_i_normalized is the normalized weight of the i-th neighboring point, and (P_i_after-P_i_before) represents the displacement of the i-th neighboring point in the two SMPLH meshes before and after.

[0119] After calculating the weights, we can perform weighted interpolation deformation on each point in the point cloud according to the displacement of the neighboring points of each point in the two SMPLH meshes before and after, and the calculated weights. Assume that for a point P in the point cloud, the k nearest neighboring points in the previous SMPLH mesh are {P1_before, P2_before, ..., Pk_before}, and the k nearest neighboring points in the next SMPLH mesh are {P1_after, P2_after, ..., Pk_after}. Then, the position P_after of point P after interpolation deformation can be calculated by the following formula:

[0120] P_after=P+sum(w_i_normalized*(P_i_after-P_i_before)for iin range(1,k+1))

[0121] Among them, w_i_normalized is the normalized weight of the i-th neighboring point, and (P_i_after-P_i_before) represents the displacement of the i-th neighboring point in the two SMPLH meshes before and after. Not only can it be applied to the displacement of point clouds, interpolation deformation can also be used for the interpolation of point cloud attributes such as normal vectors and colors.

[0122] After interpolation deformation, the point cloud may have some unnatural shapes, such as overlapping facial features. To solve these problems, we can post-process the deformed point cloud. Smoothing methods such as Laplacian smoothing and bilateral filtering can be used to improve the naturalness of the point cloud deformation.

[0123] After interpolating and deforming each frame of point cloud and post-processing, we can arrange all the deformed point clouds in order to generate human animation. If needed, we can also add textures, shadows and other effects to the point cloud to obtain a more realistic visual effect.

[0124] In addition, there is a method of human body posture transformation through skin weight transfer, including completing skin weight transfer and performing bone binding and animation production.

[0125] In the SMPLH model, the skin weight of each vertex is pre-calculated to describe the degree of association between the vertex and each bone. These weights are combined with the bone transformation through the hybrid linear skinning method to achieve the deformation of the human body model. Among them, the skin weight is the same as the vertex displacement, and the skin weight of SMPLH can be transferred to the real scanned human body through nearest neighbor interpolation.

[0126] After completing the skin weight transfer, the scanned human body model is rigged. By associating the vertices of the scanned model with the bones of the SMPLH model, the posture adjustment and animation of the scanned model can be achieved. The specific steps are as follows:

[0127] The bone structure of the SMPLH model was copied to the scanned model so that the two models had the same bone hierarchy.

[0128] The transferred skin weights are assigned to the vertices of the scanned model so that the vertices are associated with the corresponding bones.

[0129] Use the hybrid linear skinning method to achieve the deformation of the model. According to the skin weight and bone transformation, calculate the final position of each vertex in the scanned model. The specific formula is as follows:

[0130] final_position=sum(skin_weight_i*(bone_transform_i*initial_position)for iin range(1,num_bones+1))

[0131] Among them, final_position represents the final position of the vertex, skin_weight_i represents the skin weight of the vertex and the i-th bone, bone_transform_i represents the transformation matrix of the i-th bone, initial_position represents the initial position of the vertex, and num_bones represents the number of bones.

[0132] By adjusting the posture of the skeleton, the posture adjustment and animation of the scanned model can be realized. For example, the posture of the scanned model can be set to a specific posture of the SMPLH model, or the animation of the scanned model can be generated according to the animation sequence.

[0133] The present invention makes full use of the structural information of the SMPLH model, and by assigning the vertex displacement and skin weight of the SMPLH model to the scanned human body model, realizes efficient deformation of the human body scan point cloud. The present invention can not only realize the skeleton binding and posture adjustment of the scanned model, but also directly change the human body shape, thereby realizing free adjustment of the scanned model, reducing the professional skill requirements of the user and improving work efficiency. In addition, the present invention can also be applied to the fields of computer graphics, animation production, virtual reality, etc., providing more possibilities for applications such as personalized character generation and enhanced sense of reality.

[0134] The present invention also provides a device for adjusting a three-dimensional point cloud of a human body, comprising:

[0135] An initial point cloud acquisition module is used to acquire the initial point cloud;

[0136] A model matching module, used for matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created three-dimensional human body data set according to a PCA algorithm;

[0137] The deformation adjustment module is used to deform the initial point cloud after matching to obtain the target point cloud.

[0138] The present invention also provides an electronic device, comprising the above-mentioned adjustment device.

[0139] The present invention also provides a computer-readable storage medium on which computer instructions are stored. When the program is executed by a processor, any of the above-mentioned methods for adjusting the three-dimensional point cloud of a human body is implemented.

[0140] It should be noted that, in this article, the terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. The serial numbers of the above-mentioned embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for adjusting a three-dimensional point cloud of a human body, characterized in that: include: Get the initial point cloud; Matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created three-dimensional human body data set according to a PCA algorithm; The matched initial point cloud is deformed to obtain the target point cloud.

2. The method according to claim 1, characterized in that The matching process of the initial point cloud with the pre-created SMPLH model includes: Preprocessing the initial point cloud to obtain a preprocessed point cloud; Extract key points from the preprocessed point cloud to determine the posture of the human body; Based on the human body posture, setting an implicit function between the preprocessed point cloud and the SMPLH model, wherein the implicit function is obtained by determining a corresponding relationship between the initial point cloud and the SMPLH model based on the human body posture; Based on the implicit function, the preprocessed point cloud is matched with the SMPLH model to obtain the matched initial point cloud.

3. The method according to claim 1, characterized in that Before matching the initial point cloud with the preset SMPLH model, the method further includes: Obtain human body point cloud scanning data; Creating the human body three-dimensional data set based on the human body point cloud scanning data; Based on the human body three-dimensional data set, the parameters of the SMPLH model are calculated according to the PCA algorithm.

4. The method according to claim 3, characterized in that The step of calculating the parameters of the SMPLH model based on the human body three-dimensional data set and using the PCA algorithm includes: Preprocessing the human body three-dimensional data set to obtain a target data set; Aligning the target data set with the initial model through the ICP algorithm to determine the model shape and posture parameters; Based on the preset principal component analysis strategy, the PCA algorithm is used to perform dimension reduction processing on the model shape and posture parameters to obtain the parameters of the SMPLH model.

5. The method according to claim 1, characterized in that The deforming process of the matched initial point cloud includes: Performing a neighborhood search in the matched initial point cloud to determine the nearest neighbor point, wherein the nearest neighbor point includes the nearest neighbor point of each point in the matched initial point cloud within a preset range; Calculate the weight value of the nearest neighbor point; Based on the weight value, each point in the matched initial point cloud is interpolated and deformed to obtain the target point cloud.

6. The method according to claim 5, characterized in that The calculating the weight value of the nearest neighbor point specifically includes: w_i=1 / d_i Wherein, w_i is the weight value of the i-th neighboring point, and d_i is the distance between the point in the point cloud and the i-th neighboring point; After that, each calculated weight value is normalized: w_i_normalized = w_i / sum(w_1, w_2, ..., w_k), where w_i_normalized is the result of normalizing the weight values ​​of i adjacent points.

7. The method according to any one of claims 5 to 6, characterized in that: The interpolation deformation processing is performed on each point in the matched initial point cloud based on the weight value, including: Suppose a point P in the point cloud has k nearest neighbor points in the previous SMPLH mesh as {P1_before, P2_before, ..., Pk_before}, and k nearest neighbor points in the next SMPLH mesh as {P1_after, P2_after, ..., Pk_after}, then the position of point P after interpolation deformation is P_after: P_after=P+sum(w_i_normalized*(P_i_after-P_i_before)for iin range(1,k+1)) Among them, w_i_normalized is the normalized weight of the i-th neighboring point, and (P_i_after-P_i_before) represents the displacement of the i-th neighboring point in the two SMPLH meshes before and after.

8. A device for adjusting a three-dimensional point cloud of a human body, characterized in that: include: An initial point cloud acquisition module is used to acquire the initial point cloud; A model matching module, used for matching the initial point cloud with a preset SMPLH model, wherein the parameters of the SMPLH model are calculated based on a pre-created three-dimensional human body data set according to a PCA algorithm; The deformation adjustment module is used to deform the initial point cloud after matching to obtain the target point cloud.

9. An electronic device, characterized in that: The electronic device comprises: a memory having computer instructions stored thereon; and A processor is connected to the memory and is configured to execute computer instructions stored in the memory to implement the method for adjusting the three-dimensional point cloud of a human body according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the method for adjusting the three-dimensional point cloud of a human body as described in any one of claims 1 to 7 is implemented.

Images