Data registration method and apparatus therefor

Abstract

The application discloses a data registration method and device, and belongs to the field of image processing. The method comprises the following steps: detecting human body key points of three-dimensional human body mesh data to obtain each three-dimensional human body key point in the three-dimensional human body mesh data; performing first registration on a first human body linear skin model based on each three-dimensional human body key point to obtain a second human body linear skin model; and performing second registration on the second human body linear skin model based on the three-dimensional human body mesh data to obtain a target human body linear skin model.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to a data registration method and device thereof. BACKGROUND

[0002] Three-dimensional human body scanning restores a three-dimensional mesh of a human body through multi-angle camera simultaneous imaging, can obtain an accurate three-dimensional representation, is a key step for obtaining real human body data, and is therefore widely applied in a human body reconstruction process.

[0003] However, although the human body three-dimensional scanning method can reconstruct a relatively accurate three-dimensional human body mesh, an accurate human body representation is often composed of 200,000-1,000,000 three-dimensional mesh points and triangular faces, the data amount of the three-dimensional human body mesh is too large, and the three-dimensional human body mesh is discrete data, which is difficult to directly apply. SUMMARY

[0004] The embodiment of the present application aims to provide a data registration method and device thereof, which can solve the problem that the data amount of the three-dimensional human body mesh is too large, and the three-dimensional human body mesh is discrete data, which is difficult to directly apply.

[0005] In a first aspect, the embodiment of the present application provides a data registration method, which comprises:

[0006] detecting human body key points in three-dimensional human body mesh data to obtain each three-dimensional human body key point in the three-dimensional human body mesh data;

[0007] first registering a first human body linear skin model based on each three-dimensional human body key point to obtain a second human body linear skin model;

[0008] second registering the second human body linear skin model based on the three-dimensional human body mesh data to obtain a target human body linear skin model.

[0009] In a second aspect, the embodiment of the present application provides a data registration device, which comprises:

[0010] a detection module configured to detect human body key points in three-dimensional human body mesh data to obtain each three-dimensional human body key point in the three-dimensional human body mesh data;

[0011] a first registration module configured to first register a first human body linear skin model based on each three-dimensional human body key point to obtain a second human body linear skin model;

[0012] a second registration module configured to second register the second human body linear skin model based on the three-dimensional human body mesh data to obtain a target human body linear skin model.

[0013] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable by the processor. When the programs or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.

[0014] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method according to the first aspect are implemented.

[0015] In a fifth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute programs or instructions to implement the method according to the first aspect.

[0016] In a sixth aspect, a computer program product is provided, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the method according to the first aspect.

[0017] In the embodiments of the present application, the three-dimensional human key points in the three-dimensional human mesh data can be accurately obtained by detecting the three-dimensional human key points in the three-dimensional human mesh data. The number of three-dimensional human key points is not large, but can accurately reflect the characteristics of the three-dimensional human. The first human linear skinning model can be preliminarily registered by the three-dimensional human key points. The second human linear skinning model after the first registration can better conform to the characteristics of the three-dimensional human key points. The second human linear skinning model is further registered by the three-dimensional human mesh data, so that the target human linear skinning model after the second registration is close to the three-dimensional human mesh data. The data of the target human linear skinning model obtained finally is smooth and continuous, and the data amount of the target human linear skinning model is small. The virtual character construction and action generation can be performed on the basis. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A data registration method flowchart is provided for the embodiments of the present application.

[0019] Figure 2 One of the image processing flowcharts is provided for the embodiments of the present application.

[0020] Figure 3 The second of the image processing flowcharts is provided for the embodiments of the present application.

[0021] Figure 4 A data registration device structure diagram is provided for the embodiments of the present application.

[0022] Figure 5An electronic device structure schematic diagram provided by an embodiment of the present application is shown in FIG. 1.

[0023] Figure 6 A hardware structure schematic diagram of an electronic device for implementing an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0025] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than that illustrated or described herein, and the objects distinguished by "first", "second", and the like are generally of a kind and are not limited in number, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in a "or" relationship.

[0026] The data registration method and device provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and their application scenarios.

[0027] Figure 1 A data registration method flowchart provided by an embodiment of the present application is shown in FIG. 3, which includes the following steps. Figure 1

[0028] In step 110, human key point detection is performed on the three-dimensional human mesh data to obtain each three-dimensional human key point in the three-dimensional human mesh data.

[0029] The three-dimensional human mesh data described in the embodiments of the present application can be specifically obtained by three-dimensional human scanning through a multi-angle camera, by reconstructing human three-dimensional information from multiple human images scanned by the multi-angle camera, and can be specifically a mathematical process and computer technology for recovering three-dimensional information of an object using two-dimensional projection information. Three-dimensional human scanning obtains three-dimensional human mesh data representing a human body by estimating possible three-dimensional points in a specified space range and surfaces connecting these three-dimensional points.

[0030] ​The three-dimensional human body mesh data described in the embodiments of the present application can specifically include three-dimensional mesh points and their connected surfaces representing a human body, and each three-dimensional mesh vertex also includes a human body key point. A texture map corresponding to the three-dimensional human body mesh data reflects the color, skin texture, and clothing texture of the connected surfaces, etc.

[0031] In the embodiments of the present application, the three-dimensional human body mesh data is subjected to human body key point detection. Specifically, after rendering the three-dimensional human body mesh data into a plurality of two-dimensional images corresponding to different perspectives, the two-dimensional images are subjected to human body key point detection to obtain human body key points in the two-dimensional images under different perspectives. Then, after projecting a plurality of sets of initial three-dimensional human body key point coordinates to the two-dimensional images under different perspectives, the projection errors between the projected key points and the human body key points in the two-dimensional images are used to optimize and determine the three-dimensional human body key points in the three-dimensional human body mesh data, thereby obtaining three-dimensional human body key points with high accuracy.

[0032] The human body key points in the embodiments of the present application can be a series of feature points representing human body actions and body shapes. The human body key points can be defined as 25 specific joint feature points of the head, shoulder, hand, elbow, leg, and foot.

[0033] In step 120, a first human body linear skinning model is subjected to first registration based on the three-dimensional human body key points, thereby obtaining a second human body linear skinning model.

[0034] The human body linear skinning model described in the embodiments of the present application is a method for calculating a three-dimensional representation of a human body according to a series of parameters. The human body linear skinning model represents a human body as vertices and joints. The input includes body shape parameters and joint rotation parameters, and the output includes the three-dimensional positions of the vertices and joints of the human body, which are used to fit the human body.

[0035] The first human body linear skinning model in the embodiments of the present application can be a human body linear skinning model generated according to preset parameters. The preset parameters can be a preset default value, or can be generated according to historical data of a user.

[0036] In an optional embodiment, since the first human body linear skinning model is generated according to preset parameters, there can be a large gap between the first human body linear skinning model and the actual three-dimensional human body mesh data. Therefore, the first human body linear skinning model can be subjected to first registration by using the three-dimensional human body key points in the three-dimensional human body mesh data. The body shape parameters, global translation parameters, and human body key point rotation parameters of the first human body linear skinning model are adjusted, so that the second human body linear skinning model after the first registration can well match the three-dimensional human body key points in the three-dimensional human body mesh data.

[0037] In step 130, the second human body linear skinning model is subjected to second registration based on the three-dimensional human body mesh data, thereby obtaining a target human body linear skinning model.

[0038] In an optional embodiment, the second human linear skinning model after the first registration only better matches the respective three-dimensional human key points in the three-dimensional human mesh data, but there can still be some errors between the second human linear skinning model and the three-dimensional human mesh data, so the second human linear skinning model can be registered by the three-dimensional human mesh data.

[0039] The second registration in the embodiments of the present application can specifically be a registration process of adjusting the body size parameter of the second human linear skinning model and the rotation parameter of all human key points, so that the second human linear skinning model can fit the three-dimensional human mesh data as much as possible.

[0040] The target human linear skinning model after the second registration can better fit the three-dimensional human mesh data, and the target human linear skinning model can realize the generation of different actions of the same human body by modifying the rotation parameter of the human key points, and then be applied in a virtual digital human.

[0041] In the embodiments of the present application, the three-dimensional human key points in the three-dimensional human mesh data can be accurately obtained by human key point detection on the three-dimensional human mesh data. The number of three-dimensional human key points is not large, but can accurately reflect the characteristics of the three-dimensional human body. The first human linear skinning model can be preliminarily registered by the three-dimensional human key points, so that the second human linear skinning model after the first registration can better conform to the characteristics of the three-dimensional human key points. Further, the second human linear skinning model is registered by the three-dimensional human mesh data for the second time, so that the target human linear skinning model after the second registration is close to the three-dimensional human mesh data. The data of the target human linear skinning model obtained finally is smooth and continuous, and the data amount of the target human linear skinning model is small. The virtual character construction and action generation can be carried out on this basis subsequently, which has great product significance.

[0042] Optionally, the first human linear skinning model is registered for the first time based on the respective three-dimensional human key points, and a second human linear skinning model is obtained, including:

[0043] A first distance between each three-dimensional human key point and a target model point corresponding to the three-dimensional human key point is obtained; the target model point is a model point closest to the three-dimensional human key point among respective model points of the first human linear skinning model;

[0044] According to N groups of second human linear skinning model parameters, an average value of the first distance corresponding to each group of second human linear skinning model parameters is determined;

[0045] determine the second human linear skinning model parameters after the first registration according to the second human linear skinning model parameters corresponding to the first target average value, wherein the second human linear skinning model parameters include body shape parameters, global translation parameters and human key point rotation parameters, N is a positive integer, and the first target average value is the minimum value in the average values of the N groups of first distances;

[0046] adjust the first human linear skinning model according to the second human linear skinning model parameters to obtain a second human linear skinning model.

[0047] In an optional embodiment, based on the three-dimensional coordinates of each three-dimensional human key point and the three-dimensional coordinates of each model point of the first human linear skinning model, a model point closest to each three-dimensional human key point is determined, and the closest model point is taken as the target model point corresponding to the three-dimensional human key point.

[0048] In an optional embodiment, each three-dimensional human key point has a corresponding target model point, and the first distance between the three-dimensional key point and the target model point can be specifically calculated according to the three-dimensional coordinates of the three-dimensional human key point and the three-dimensional coordinates of the target model point.

[0049] In an optional embodiment, since each three-dimensional human key point can have a corresponding target model point, a plurality of first distances can be determined based on a plurality of three-dimensional key points, and after obtaining the plurality of first distances, the average value of the plurality of first distances can be obtained by summing the plurality of first distances.

[0050] In an optional embodiment, the variable to be optimized of the first human linear skinning model is the second human linear skinning model parameters, which can specifically include body shape parameters, global translation parameters and human key point rotation parameters.

[0051] In the embodiments of the present application, the body shape parameters can be parameters representing the body shape characteristics such as height, weight, slimness, etc., the global translation parameters can be parameters for controlling the spatial translation of the first human linear skinning model, and the human key point rotation parameters can be the angles for controlling the rotation of the key points of the key joints. Different postures of the human body can be controlled by the key point rotation corresponding to the human key points.

[0052] In the embodiments of the present application, the N groups of second human linear skinning model parameters can be analyzed by the Adam optimizer or the gradient descent method, and the human linear skinning model is driven by the second human linear skinning model parameters. Different parameter inputs will respond to different models.

[0053] In an optional embodiment, the first registration process can specifically be that N sets of different second human body linear skinning model parameters are first randomly generated, where the value of N can be artificially preset or adjusted according to actual requirements. The larger the value of N is, the more second human body linear skinning model parameters are randomly generated, and the more accurate the result of the first registration is. Each corresponding human body linear skinning model is determined according to each set of second human body linear skinning model parameters, and the average value of the first distance between each three-dimensional human body key point and the target model point in each set of human body linear skinning model is obtained. The minimum value of the average value of the N sets of first distance is taken as the first target average value, and the set of parameters corresponding to the first target average value is taken as the second human body linear skinning model parameter after the first registration.

[0054] In an optional embodiment, the first registration process can specifically be that N sets of different second human body linear skinning model parameters are first randomly generated, where the value of N can be artificially preset or adjusted according to actual requirements. The larger the value of N is, the more second human body linear skinning model parameters are randomly generated, and the more accurate the result of the first registration is. Each corresponding human body linear skinning model is determined according to each set of second human body linear skinning model parameters, and the average value of the first distance between each three-dimensional human body key point and the target model point in each set of human body linear skinning model is obtained. The minimum value of the average value of the N sets of first distance is taken as the first target average value, and the set of parameters corresponding to the first target average value is taken as the second human body linear skinning model parameter after the first registration.

[0055] In an optional embodiment, the first registration process can specifically be that N sets of different second human body linear skinning model parameters are first randomly generated, where the value of N can be artificially preset or adjusted according to actual requirements. The larger the value of N is, the more second human body linear skinning model parameters are randomly generated, and the more accurate the result of the first registration is. Each corresponding human body linear skinning model is determined according to each set of second human body linear skinning model parameters, and the average value of the first distance between each three-dimensional human body key point and the target model point in each set of human body linear skinning model is obtained. The minimum value of the average value of the N sets of first distance is taken as the first target average value, and the set of parameters corresponding to the first target average value is taken as the second human body linear skinning model parameter after the first registration.

[0056] In the embodiment of the present application, after the second human body linear skinning model parameter after the first registration is determined, the first human body linear skinning model can be adjusted according to the body type parameter, the global translation parameter and the human body key point rotation parameter in the second human body linear skinning model parameter to obtain the second human body linear skinning model.

[0057] In the embodiment of the present application, after the first distance between the three-dimensional human body key points and the closest target model points is obtained, the average value of the first distance is taken as the optimization target in the first registration process, and the second human body linear skinning model parameters corresponding to the minimum average value of the first distance are taken as the second human body linear skinning model parameters through continuous adjustment of N sets of second human body linear skinning model parameters, so as to effectively ensure that the second human body linear skinning model can fit each three-dimensional human body key point in the three-dimensional human body grid data as much as possible, and make the appearance of the second human body linear skinning model closer to the three-dimensional human body grid data.

[0058] Optionally, the second registration of the second human body linear skinning model based on the three-dimensional human body grid data to obtain a target human body linear skinning model comprises:

[0059] obtaining a second distance between each model point of the second human body linear skinning model and the three-dimensional human body grid data, and a third distance between each grid point of the three-dimensional human body grid data and the second human body linear skinning model;

[0060] determining a target average distance based on the average value of the second distance and the average value of the third distance;

[0061] determining the target average distance corresponding to each set of target human body linear skinning model parameters according to M sets of target human body linear skinning model parameters;

[0062] determining the target human body linear skinning model parameters after the second registration according to the target human body linear skinning model parameters corresponding to the second target average value, wherein the target human body linear skinning model parameters comprise body shape parameters and human body key point rotation parameters, and M is a positive integer;

[0063] adjusting the second human body linear skinning model according to the target human body linear skinning model parameters after the second registration to obtain a target human body linear skinning model.

[0064] In the embodiment of the present application, after the second human body linear skinning model is obtained, there may still be certain errors between the second human body linear skinning model and the three-dimensional human body grid data, and further calibration and adjustment are needed, so in the embodiment of the present application, the second human body linear skinning model is further registered through the most accurate three-dimensional human body grid data.

[0065] In an optional embodiment, the second distance between each model point and the three-dimensional human body grid data can specifically refer to the minimum distance between each model point and the three-dimensional human body grid data, which can be determined according to the three-dimensional coordinates of each model point and the three-dimensional coordinates of each grid point in the three-dimensional human body grid data.

[0066] In an optional embodiment, the third distance of each mesh point of the three-dimensional human body mesh data to the second human body linear skinning model can be specifically the minimum distance between each mesh point of the three-dimensional human body mesh data and the second human body linear skinning model, and can also be determined according to the three-dimensional coordinates of each model point and the three-dimensional coordinates of each mesh point in the three-dimensional human body mesh data.

[0067] In the embodiments of the present application, the second human body linear skinning model can have multiple model points, so multiple second distances can be obtained, and after obtaining the multiple second distances, the sum of the multiple second distances can be taken to obtain the average value of the second distance. Correspondingly, the three-dimensional human body mesh data can also have multiple mesh points, so multiple third distances can be obtained, and after obtaining the multiple third distances, the sum of the multiple third distances can be taken to obtain the average value of the third distance.

[0068] In an optional embodiment, after obtaining the average value of the second distance and the average value of the third distance, the average value of the second distance and the average value of the third distance can be further weighted and summed to obtain a target average distance. The average value of the second distance and the average value of the third distance can be a preset value, or can be a value set by a user according to actual conditions.

[0069] In the embodiments of the present application, the purpose of the second registration is to further adjust the purpose of the first registration, so that the human body linear skinning model is closer to the three-dimensional human body mesh data obtained by scanning. The Adam optimizer or the gradient descent method described above can be used for the second registration.

[0070] In an optional embodiment, the second registration process can specifically be that M groups of different target human body linear skinning model parameters are randomly generated, where the value of M can be artificially preset or adjusted according to actual needs. The larger the value of M, the more target human body linear skinning model parameters are randomly generated, and the more accurate the result of the second registration is. Each group of corresponding human body linear skinning models is determined according to each group of target human body linear skinning model parameters, and a target average distance between the three-dimensional human body mesh data and each group of human body linear skinning models is obtained. The minimum value of the M groups of target average distances is taken as a second target average value, and the parameter of the group corresponding to the second target average value is taken as the target human body linear skinning model parameter after the second registration.

[0071] In an optional embodiment, the second registration process can further specifically be that a set of target human body linear skinning model parameters is first generated randomly, and then the target human body linear skinning model parameters are adjusted M times, each adjustment making the loss value of the target average distance loss function smaller, to obtain M sets of target human body linear skinning model parameters. The set of parameters that makes the target average distance minimum is finally determined and taken as the target human body linear skinning model parameters after the second registration.

[0072] In an optional embodiment, the second registration process can further specifically be that a set of target human body linear skinning model parameters is first generated randomly, and then the target human body linear skinning model parameters are adjusted M times, each adjustment making the loss value of the target average distance loss function smaller, to obtain M sets of target human body linear skinning model parameters. The set of parameters that makes the target average distance minimum is finally determined and taken as the target human body linear skinning model parameters after the second registration.

[0073] In the embodiments of the present application, after the target human body linear skinning model parameters after the second registration are determined, the second human body linear skinning model can be adjusted according to the body shape parameters and human body key point rotation parameters in the target human body linear skinning model parameters after the second registration, and finally the target human body linear skinning model is obtained.

[0074] In the embodiments of the present application, the target average distance capable of representing the distance between the three-dimensional human body mesh data and the second human body linear skinning model is taken as the optimization target, and through the continuous adjustment of multiple sets of target human body linear skinning model parameters, the target human body linear skinning model parameters after the second registration are effectively obtained in the case of the minimum target average distance, so that the target human body linear skinning model generated according to the target human body linear skinning model parameters after the second registration is closer to the scanned three-dimensional human body mesh data and can more vividly present the three-dimensional human body.

[0075] Optionally, the three-dimensional human body mesh data is subjected to human body key point detection to obtain each three-dimensional human body key point in the three-dimensional human body mesh data, including:

[0076] Based on the three-dimensional human body mesh data, three-dimensional human body images under P perspectives are rendered, and P is a positive integer greater than 2.

[0077] projecting the three-dimensional human body image under each perspective into a two-dimensional image to obtain a two-dimensional image corresponding to each perspective, the two-dimensional image including at least one two-dimensional human body key point information;

[0078] projecting an initial three-dimensional human body key point coordinate set under a preset initial perspective into the two-dimensional image corresponding to each perspective to obtain a two-dimensional human body key point projection coordinate set corresponding to each perspective;

[0079] determining each three-dimensional human body key point in the three-dimensional human body grid data based on a difference between each two-dimensional human body key point projection coordinate and corresponding two-dimensional human body key point information, the two-dimensional human body key point projection coordinate being a coordinate in the two-dimensional human body key point projection coordinate set.

[0080] The three-dimensional human body grid data in the embodiments of the present application records three-dimensional grid points identifying a human body and connecting surfaces of the three-dimensional grid points, and further records a texture map, and the texture map can reflect colors, skin textures, and clothing textures of each connecting surface.

[0081] In an optional embodiment, since the three-dimensional human body grid data contains complete data under each perspective, the three-dimensional human body image under multiple different perspectives can be simulated in the embodiments of the present application, for example, by simulating a virtual camera to simulate the three-dimensional human body image under multiple different perspectives. Due to the limitation of perspective, the three-dimensional human body grid data under a single perspective is not complete, and the content seen under different perspectives is different, and the generated three-dimensional human body image is also different.

[0082] In an optional embodiment, since at least three perspectives are required to present a complete three-dimensional image, the three-dimensional human body image under at least three perspectives can be rendered based on the three-dimensional human body grid data in the embodiments of the present application.

[0083] In an optional embodiment, the three-dimensional human body image can be projected into a two-dimensional image by a projection model of a virtual camera corresponding to each perspective to obtain a two-dimensional image corresponding to each perspective.

[0084] In another optional embodiment, an initial three-dimensional human body key point coordinate set under a preset initial perspective can also be defined, the three-dimensional coordinate transformation of a grid point under each perspective relative to the grid point under the preset initial perspective being rotation R_i and translation t_i, and if the grid point under the initial preset perspective is V, then the grid point under each perspective is represented by formula 1:

[0085] V_i=R_i*V+t_i, i=1,...,P (1)

[0086] wherein P is a positive integer greater than 2, and P can be preset by a human. The grid points of each view are projected into a two-dimensional image, the connecting surface and color between each grid point are consistent with the three-dimensional human body grid data, and the projected two-dimensional image M_i is obtained through formula 2.

[0087] M_i = π(V_i) (2)

[0088] wherein π(·) represents a projection model.

[0089] In an optional embodiment, after the three-dimensional human body image under each view is projected into a two-dimensional image, key point detection can be performed on each two-dimensional image through a key point detection algorithm, the human body key points in each two-dimensional image are determined, and the coordinate values of the human body key points in the two-dimensional image are determined to obtain two-dimensional human body key point information.

[0090] The preset initial view described in the embodiments of the present application can be a preset view, which can be used as a reference value under different views. The initial three-dimensional human body key point coordinate set under the initial view can include the three-dimensional coordinates of each human body key point in the three-dimensional human body grid data under the initial view. The three-dimensional coordinates can be a pre-estimated value, and the coordinate values can be continuously adjusted in subsequent algorithms.

[0091] In the embodiments of the present application, after obtaining the initial three-dimensional human body key point coordinate set under the preset initial view, the initial three-dimensional human body key point coordinate set can be adjusted as a whole through rotation and translation to different views to obtain the initial three-dimensional human body key point coordinate set under different views. Then, the initial three-dimensional human body key point coordinate set under different views is projected into a two-dimensional image through a projection model to obtain a two-dimensional human body key point projection coordinate set corresponding to each view.

[0092] In the embodiments of the present application, since the two-dimensional human body key point projection coordinate set and the two-dimensional human body key point information both describe the human body key points of the same object, there is a corresponding relationship between the two-dimensional human body key point projection coordinates and the two-dimensional human body key point information of the same human body key point.

[0093] In the embodiments of the present application, the difference in distance can be determined based on the two-dimensional human body key point projection coordinates and the corresponding two-dimensional human body key point information. After obtaining a plurality of difference values based on a plurality of two-dimensional human body key point projection coordinates, the sum of the plurality of difference values can be used as a minimization target to determine each three-dimensional human body key point in the three-dimensional human body grid data.

[0094] In an optional embodiment, the difference value can be specifically represented by formula 3:

[0095] E_reproj = π(R_i * P_3d + t_i) - P_2d (3)

[0096] wherein the rotation value is R_i, the translation value is t_i, P_3d is the initial three-dimensional human body key point coordinate, and P_2d is the two-dimensional human body key point information.

[0097] Figure 2 One of the image processing flowcharts provided by the embodiments of the present application, as shown in Figure 2 includes: rendering a three-dimensional human body image under multiple perspectives according to three-dimensional human body mesh data, and then projecting the image into a two-dimensional image to obtain a two-dimensional image under each perspective; then detecting two-dimensional human body key point information in the two-dimensional image under each perspective, and projecting an initial three-dimensional human body key point coordinate set into the two-dimensional image corresponding to each perspective to obtain a two-dimensional human body key point projection coordinate set; and performing triangulation difference processing on the two-dimensional human body key point information and the two-dimensional human body key point projection coordinate set to finally determine each three-dimensional human body key point in the three-dimensional human body mesh data.

[0098] In the embodiments of the present application, the initial three-dimensional human body key point under the initial perspective and the human body key point corresponding to the three-dimensional human body image under each perspective are projected into the same plane, and then the three-dimensional human body key point with the minimum projection error is determined as the target, thereby effectively ensuring the accuracy of the three-dimensional human body key point.

[0099] Optionally, each three-dimensional human body key point in the three-dimensional human body mesh data is determined based on the difference between each two-dimensional human body key point projection coordinate and the corresponding two-dimensional human body key point information, including:

[0100] an average value of each of the difference values is obtained;

[0101] a plurality of sets of the initial three-dimensional human body key point coordinate sets are analyzed to obtain an average value of the difference corresponding to each set of the initial three-dimensional human body key point coordinate set;

[0102] a target three-dimensional human body key point coordinate set is determined according to the initial three-dimensional human body key point coordinate set corresponding to the third target average value, wherein the third target average value is the minimum value among the average values of the plurality of sets of the difference, and the target three-dimensional human body key point coordinate set includes the coordinates of each three-dimensional human body key point in the three-dimensional human body mesh data. In the embodiments of the present application, there can be multiple two-dimensional human body key point projection coordinates in the two-dimensional human body key point projection coordinate set, and therefore the difference between the corresponding two-dimensional human body key point projection coordinates and the corresponding two-dimensional human body key point information can be obtained. After obtaining a plurality of difference values, the difference values are summed and averaged to obtain the average value of the difference.

[0103] In the embodiments of the present application, the multiple sets of initial three-dimensional human key point coordinate sets can be analyzed and optimized by using an Adam optimizer or a gradient descent method. The optimization analysis process can be as follows: first, a plurality of different sets of initial three-dimensional human key point coordinate sets are randomly generated, and the above steps are processed to obtain the average value of the difference corresponding to each set of initial three-dimensional human key point coordinate sets. The minimum value of the average value of the multiple sets of differences is taken as a third target average value, and the set of initial three-dimensional human key point coordinate sets corresponding to the third target average value is taken as the target three-dimensional human key point coordinate set.

[0104] In an optional embodiment, the optimization analysis process can further be as follows: first, any set of initial three-dimensional human key point coordinate sets can be selected for processing according to the above steps, and then the initial three-dimensional human key point coordinate sets are adjusted slightly each time to make the loss value of the loss function of the average value of the difference smaller. A third preset threshold is set, and when the loss value of the loss function of the average value of the difference is smaller than or equal to the third preset threshold, it is determined that the loss function of the average value of the difference converges, at which time the adjustment is stopped, and the multiple sets of initial three-dimensional human key point coordinate sets are obtained. Finally, the set of initial three-dimensional human key point coordinate sets that makes the average value of the difference minimum is determined, and is taken as the target three-dimensional human key point coordinate set.

[0105] In an optional embodiment, the optimization analysis process can further be as follows: first, any set of initial three-dimensional human key point coordinate sets can be selected for processing according to the above steps, and then the initial three-dimensional human key point coordinate sets are adjusted slightly each time to make the loss value of the loss function of the average value of the difference smaller. A third preset threshold is set, and when the loss value of the loss function of the average value of the difference is smaller than or equal to the third preset threshold, it is determined that the loss function of the average value of the difference converges, at which time the adjustment is stopped, and the multiple sets of initial three-dimensional human key point coordinate sets are obtained. Finally, the set of initial three-dimensional human key point coordinate sets that makes the average value of the difference minimum is determined, and is taken as the target three-dimensional human key point coordinate set.

[0106] The average value of the difference can be used to determine each three-dimensional human key point in the three-dimensional human mesh data according to the target three-dimensional human key point coordinate set.

[0107] In the embodiments of the present application, the multiple sets of initial three-dimensional human key point coordinate sets are analyzed and optimized by taking the minimization of the average value of each projection difference as the optimization target, the target three-dimensional human key point coordinate set that makes the average value of the projection difference minimum is effectively obtained, the three-dimensional human key points can be determined more accurately according to the target three-dimensional human key point coordinate set, and the accuracy of the three-dimensional key points is effectively ensured.

[0108] Figure 3 The second image processing flowchart provided in the embodiments of the present application is as follows: Figure 3As shown, the method comprises: after performing a three-dimensional human body scan, three-dimensional human body mesh data and a corresponding texture map can be obtained; then, three-dimensional human body key point analysis is performed according to the three-dimensional human body mesh data and the corresponding texture map, to obtain each three-dimensional human body key point in the three-dimensional human body mesh data; then, the first human body linear skinning model is registered for the first time according to the three-dimensional human body key points, and the result of the first registration is registered for the second time in combination with the three-dimensional human body mesh data, to finally obtain the registered target human body linear skinning model.

[0109] The data registration method provided in the embodiments of the present application can be executed by the data registration device. The data registration device provided in the embodiments of the present application is described by taking the data registration device as an example.

[0110] Figure 4 The structural schematic diagram of the data registration device provided in the embodiments of the present application is shown in Figure 4 The data registration device comprises:

[0111] The detection module 410 is configured to perform human body key point detection on the three-dimensional human body mesh data, to obtain each three-dimensional human body key point in the three-dimensional human body mesh data.

[0112] The first registration module 420 is configured to perform first registration on the first human body linear skinning model based on each three-dimensional human body key point, to obtain a second human body linear skinning model.

[0113] The second registration module 430 is configured to perform second registration on the second human body linear skinning model based on the three-dimensional human body mesh data, to obtain a target human body linear skinning model.

[0114] Optionally, the first registration module is specifically configured to:

[0115] obtain a first distance between each three-dimensional human body key point and a target model point corresponding to the three-dimensional human body key point, the target model point being a model point closest to the three-dimensional human body key point among each model point of the first human body linear skinning model;

[0116] determine an average value of the first distance corresponding to each group of second human body linear skinning model parameters according to N groups of second human body linear skinning model parameters;

[0117] determine the second human body linear skinning model parameters after the first registration according to the second human body linear skinning model parameters corresponding to the first target average value, wherein the second human body linear skinning model parameters comprise body shape parameters, global translation parameters and human body key point rotation parameters, N is a positive integer, and the first target average value is the minimum value among the average values of the N groups of first distances;

[0118] According to the second human body linear skinning model parameter after the first registration, the first human body linear skinning model is adjusted to obtain a second human body linear skinning model.

[0119] Optionally, the second registration module is specifically used for:

[0120] The second distance between each model point of the second human body linear skinning model and the three-dimensional human body mesh data, and the third distance between each mesh point of the three-dimensional human body mesh data and the second human body linear skinning model are obtained.

[0121] Based on the average value of the second distance and the average value of the third distance, a target average distance is determined.

[0122] According to the target human body linear skinning model parameter corresponding to each group of target human body linear skinning model parameters, the target average distance corresponding to each group of target human body linear skinning model parameters is determined.

[0123] According to the target human body linear skinning model parameter corresponding to the second target average value, a target human body linear skinning model parameter after the second registration is determined, wherein the target human body linear skinning model parameter includes a body shape parameter and a human body key point rotation parameter, M is a positive integer, and the second target average value is the minimum value in the M groups of target average distances.

[0124] According to the target human body linear skinning model parameter after the second registration, the second human body linear skinning model is adjusted to obtain a target human body linear skinning model.

[0125] Optionally, the detection module is specifically used for:

[0126] Based on the three-dimensional human body mesh data, three-dimensional human body images under P perspectives are rendered, and P is a positive integer greater than 2.

[0127] The three-dimensional human body images under each perspective are projected into two-dimensional images to obtain two-dimensional images corresponding to each perspective, and the two-dimensional images include at least one two-dimensional human body key point information.

[0128] The initial three-dimensional human body key point coordinate set under a preset initial perspective is projected into the two-dimensional images corresponding to each perspective to obtain a two-dimensional human body key point projection coordinate set corresponding to each perspective.

[0129] Based on the difference between each two-dimensional human body key point projection coordinate and the corresponding two-dimensional human body key point information, each three-dimensional human body key point in the three-dimensional human body mesh data is determined, and the two-dimensional human body key point projection coordinate is a coordinate in the two-dimensional human body key point projection coordinate set.

[0130] Optionally, the detection module is specifically used for:

[0131] obtaining an average value of each of the difference values;

[0132] performing optimization analysis on the plurality of sets of initial three-dimensional human key point coordinate sets to obtain an average value of the difference values corresponding to each set of initial three-dimensional human key point coordinate sets;

[0133] determining a target three-dimensional human key point coordinate set according to the initial three-dimensional human key point coordinate set corresponding to the third target average value, wherein the third target average value is the minimum value among the average values of the plurality of difference values, and the target three-dimensional human key point coordinate set includes the coordinates of each three-dimensional human key point in the three-dimensional human mesh data.

[0134] In the embodiments of the present application, the three-dimensional human key points in the three-dimensional human mesh data can be accurately obtained by performing human key point detection on the three-dimensional human mesh data. The number of three-dimensional human key points is not large, but can accurately reflect the characteristics of the three-dimensional human body. The first human linear skinning model can be preliminarily registered by the three-dimensional human key points, so that the second human linear skinning model after the first registration can better conform to the characteristics of the three-dimensional human key points. Further, the second human linear skinning model is registered for the second time by the three-dimensional human mesh data, so that the target human linear skinning model after the second registration is close to the three-dimensional human mesh data. The data of the target human linear skinning model obtained finally is smooth and continuous, and the data amount of the target human linear skinning model is small. The subsequent construction of virtual characters and motion generation can be performed on this basis, which has great product significance.

[0135] The data registration apparatus in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other device than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like, and the embodiments of the present application are not limited in this regard.

[0136] The data registration apparatus in the embodiments of the present application can be an apparatus with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating system, and the embodiments of the present application are not limited in this regard.

[0137] The data registration apparatus provided in the embodiments of the present application can implement the method embodiments, and each process of the method embodiments is not repeated here. Figures 1 to 4

[0138] Optionally, Figure 5 An electronic device structure schematic diagram provided in the embodiments of the present application is shown in FIG. 1, and the embodiments of the present application further provide an electronic device 500, which includes a processor 501 and a memory 502. The memory 502 stores a program or instruction that can run on the processor 501. When the program or instruction is executed by the processor 501, each step of the above-mentioned data registration method embodiments is implemented, and the same technical effects are achieved. Each process of the method embodiments is not repeated here. Figure 5

[0139] It should be noted that the electronic device in the embodiments of the present application includes the above-mentioned mobile electronic device and non-mobile electronic device.

[0140] Figure 6 A hardware structure schematic diagram of an electronic device for implementing the embodiments of the present application.

[0141] ​​The electronic device 600 includes, but is not limited to, a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610, etc.

[0142] Those skilled in the art can understand that the electronic device 600 can also include a power supply (such as a battery) for powering various components, which can be logically connected to the processor 610 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure is not a limitation on the electronic device, and the electronic device can include more or fewer components than shown, or combine certain components, or different component arrangements, which are not described here.

[0143] The processor 610 is configured to perform human key point detection on the three-dimensional human mesh data to obtain each three-dimensional human key point in the three-dimensional human mesh data.

[0144] The first human linear skinning model is registered for the first time based on each three-dimensional human key point to obtain a second human linear skinning model.

[0145] The second human linear skinning model is registered for the second time based on the three-dimensional human mesh data to obtain a target human linear skinning model.

[0146] The processor 610 is configured to obtain a first distance between each three-dimensional human key point and a target model point corresponding to the three-dimensional human key point; the target model point is a model point closest to the three-dimensional human key point among each model point of the first human linear skinning model.

[0147] The average value of each first distance is minimized as the target, and according to N sets of second human linear skinning model parameters, the average value of the first distance corresponding to each set of second human linear skinning model parameters is determined.

[0148] According to the second human linear skinning model parameter corresponding to the first target average value, the second human linear skinning model parameter after the first registration is determined, wherein the second human linear skinning model parameter includes a body shape parameter, a global translation parameter, and a human key point rotation parameter, N is a positive integer, and the first target average value is the minimum value among N sets of first distance average values.

[0149] The first human linear skinning model is adjusted according to the second human linear skinning model parameter after the first registration to obtain a second human linear skinning model.

[0150] The processor 610 is configured to obtain a second distance between each model point of the second human body linear skin model and the three-dimensional human body mesh data, and a third distance between each mesh point of the three-dimensional human body mesh data and the second human body linear skin model.

[0151] Determine a target average distance based on the average of the second distance and the average of the third distance.

[0152] According to the M group of target human body linear skin model parameters, determine the target average distance corresponding to each group of target human body linear skin model parameters.

[0153] According to the target human body linear skin model parameter corresponding to the second target average value, determine the target human body linear skin model parameter after the second registration, wherein the target human body linear skin model parameter includes a body shape parameter and a human body key point rotation parameter, M is a positive integer, and the second target average value is the minimum value in the M groups of target average distances.

[0154] According to the second registration target human body linear skin model parameter, adjust the second human body linear skin model to obtain a target human body linear skin model.

[0155] The processor 610 is configured to render a three-dimensional human body image under P perspectives based on the three-dimensional human body mesh data, wherein P is a positive integer greater than 2.

[0156] Project the three-dimensional human body image under each perspective into a two-dimensional image to obtain a two-dimensional image corresponding to each perspective, wherein the two-dimensional image includes at least one two-dimensional human body key point information.

[0157] Project an initial three-dimensional human body key point coordinate set under a preset initial perspective into the two-dimensional image corresponding to each perspective to obtain a two-dimensional human body key point projection coordinate set corresponding to each perspective.

[0158] Determine each three-dimensional human body key point in the three-dimensional human body mesh data based on the difference between each two-dimensional human body key point projection coordinate and the corresponding two-dimensional human body key point information, wherein the two-dimensional human body key point projection coordinate is a coordinate in the two-dimensional human body key point projection coordinate set.

[0159] The processor 610 is configured to obtain an average value of each of the differences.

[0160] Optimize and analyze a plurality of groups of the initial three-dimensional human body key point coordinate sets to obtain an average value of the difference corresponding to each group of initial three-dimensional human body key point coordinate sets.

[0161] According to the initial three-dimensional human body key point coordinate set corresponding to the third target average value, a target three-dimensional human body key point coordinate set is determined, wherein the third target average value is the minimum value in the average values of the plurality of groups of difference values, and the target three-dimensional human body key point coordinate set includes the coordinates of each three-dimensional human body key point in the three-dimensional human body mesh data.

[0162] In the embodiments of the present application, the three-dimensional human body key points in the three-dimensional human body mesh data can be accurately obtained by detecting the three-dimensional human body key points in the three-dimensional human body mesh data. The number of three-dimensional human body key points is not large, but can accurately reflect the characteristics of the three-dimensional human body. The first human body linear skinning model can be preliminarily registered by the three-dimensional human body key points, so that the second human body linear skinning model after the first registration can better conform to the characteristics of the three-dimensional human body key points. Further, the second human body linear skinning model is registered by the three-dimensional human body mesh data for the second time, so that the target human body linear skinning model after the second registration is close to the three-dimensional human body mesh data. The data of the target human body linear skinning model ultimately obtained is smooth and continuous, and the data amount of the target human body linear skinning model is small. Subsequently, the construction and motion generation of a virtual character can be performed on the basis, which has great product significance.

[0163] It should be understood that in the embodiments of the present application, the input unit 604 can include a graphics processing unit (GPU) 6041 and a microphone 6042. The graphics processing unit 6041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 can include a display panel 6061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 can include a touch detection device and a touch controller. The other input devices 6072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0164] The memory 609 can be used to store software programs and various data. The memory 609 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 609 can include a volatile memory or a non-volatile memory, or the memory 609 can include both a volatile memory and a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 609 in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.

[0165] The processor 610 can include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 610.

[0166] The embodiments of the present application also provide a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize various processes of the above-mentioned data registration method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0167] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0168] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the above data registration method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0169] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system level chip, a system chip, a chip system or a system on chip, etc.

[0170] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and is executed by at least one processor to realize the processes of the above data registration method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0171] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in the opposite order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0173] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A data registration method, characterized by, The method comprises the following steps: human key point detection is performed on three-dimensional human mesh data to obtain each three-dimensional human key point in the three-dimensional human mesh data; a first human linear skinning model is registered based on each three-dimensional human key point to obtain a second human linear skinning model; the second human linear skinning model is registered based on the three-dimensional human mesh data to obtain a target human linear skinning model; the first human linear skinning model is registered based on each three-dimensional human key point to obtain a second human linear skinning model, comprising: a first distance between each three-dimensional human key point and a target model point corresponding to the three-dimensional human key point is obtained; the target model point is a model point closest to the three-dimensional human key point among each model point of the first human linear skinning model; an average value of the first distance corresponding to each group of second human linear skinning model parameters is determined according to N groups of second human linear skinning model parameters; second human linear skinning model parameters after the first registration are determined according to the second human linear skinning model parameters corresponding to the first target average value, wherein the second human linear skinning model parameters comprise body shape parameters, global translation parameters and human key point rotation parameters, N is a positive integer, and the first target average value is the minimum value among average values of N groups of the first distance; the first human linear skinning model is adjusted according to the second human linear skinning model parameters after the first registration to obtain a second human linear skinning model.

2. The data registration method of claim 1, wherein, The second human linear skinning model is registered based on the three-dimensional human mesh data to obtain a target human linear skinning model, comprising: a second distance between each model point of the second human linear skinning model and the three-dimensional human mesh data and a third distance between each mesh point of the three-dimensional human mesh data and the second human linear skinning model are obtained; a target average distance is determined based on an average value of the second distance and an average value of the third distance; the target average distance corresponding to each group of target human linear skinning model parameters is determined according to M groups of target human linear skinning model parameters; target human linear skinning model parameters after the second registration are determined according to the target human linear skinning model parameters corresponding to the second target average value, wherein the target human linear skinning model parameters comprise body shape parameters and human key point rotation parameters, M is a positive integer, and the second target average value is the minimum value among M groups of the target average distance; the second human linear skinning model is adjusted according to the target human linear skinning model parameters after the second registration to obtain a target human linear skinning model.

3. The data registration method of claim 1, wherein, Human key point detection is performed on three-dimensional human mesh data to obtain each three-dimensional human key point in the three-dimensional human mesh data, comprising: three-dimensional human images under P perspectives are rendered based on the three-dimensional human mesh data, and P is a positive integer greater than 2. Projecting the three-dimensional human body image under each perspective into a two-dimensional image to obtain a two-dimensional image corresponding to each perspective, wherein the two-dimensional image includes at least one two-dimensional human body key point information; Projecting an initial three-dimensional human body key point coordinate set under a preset initial perspective into the two-dimensional image corresponding to each perspective to obtain a two-dimensional human body key point projection coordinate set corresponding to each perspective; Determining each three-dimensional human body key point in the three-dimensional human body grid data based on a difference between each two-dimensional human body key point projection coordinate and corresponding two-dimensional human body key point information, wherein the two-dimensional human body key point projection coordinate is a coordinate in the two-dimensional human body key point projection coordinate set.

4. The data registration method of claim 3, wherein, Determining each three-dimensional human body key point in the three-dimensional human body grid data based on a difference between each two-dimensional human body key point projection coordinate and corresponding two-dimensional human body key point information, comprising: Obtaining an average value of each difference value; Optimizing and analyzing a plurality of initial three-dimensional human body key point coordinate sets to obtain an average value of the difference value corresponding to each initial three-dimensional human body key point coordinate set; Determining a target three-dimensional human body key point coordinate set based on the initial three-dimensional human body key point coordinate set corresponding to the third target average value, wherein the third target average value is the minimum value in the average values of the plurality of differences, and the target three-dimensional human body key point coordinate set includes the coordinates of each three-dimensional human body key point in the three-dimensional human body grid data.

5. A data registration apparatus, characterized by Comprising: A detection module configured to detect human body key points in three-dimensional human body grid data to obtain each three-dimensional human body key point in the three-dimensional human body grid data; A first registration module configured to register a first human body linear skin model based on each three-dimensional human body key point to obtain a second human body linear skin model; A second registration module configured to register the second human body linear skin model based on three-dimensional human body grid data to obtain a target human body linear skin model; The first registration module is specifically configured to: Obtain a first distance between each three-dimensional human body key point and a target model point corresponding to the three-dimensional human body key point, wherein the target model point is the closest model point to the three-dimensional human body key point among each model point of the first human body linear skin model; Determine an average value of the first distance corresponding to each second human body linear skin model parameter based on N groups of second human body linear skin model parameters; Determine a second human body linear skin model parameter after the first registration based on the second human body linear skin model parameter corresponding to the first target average value, wherein the second human body linear skin model parameter includes a body shape parameter, a global translation parameter, and a human body key point rotation parameter, N is a positive integer, and the first target average value is the minimum value in the average values of N groups of first distances; Adjust the first human body linear skin model based on the second human body linear skin model parameter after the first registration to obtain a second human body linear skin model.

6. The data registration apparatus of claim 5, wherein, The second registration module is specifically configured to: acquire a second distance between each model point of the second human linear skin model and the three-dimensional human mesh data, and a third distance between each mesh point of the three-dimensional human mesh data and the second human linear skin model; determine a target average distance based on an average value of the second distance and an average value of the third distance; determine the target average distance corresponding to each group of target human linear skin model parameters according to M groups of target human linear skin model parameters; determine the target human linear skin model parameters after the second registration according to the target human linear skin model parameters corresponding to the second target average value, wherein the target human linear skin model parameters include body shape parameters and human key point rotation parameters, M is a positive integer, and the second target average value is the minimum value in the M groups of target average distances; adjust the second human linear skin model according to the target human linear skin model parameters after the second registration to obtain a target human linear skin model.

7. The data registration apparatus of claim 5, wherein, The detection module is specifically configured to: render a three-dimensional human image under P perspectives based on the three-dimensional human mesh data, P being a positive integer greater than 2; project the three-dimensional human image under each perspective into a two-dimensional image to obtain a two-dimensional image corresponding to each perspective, the two-dimensional image including at least one two-dimensional human key point information; project an initial three-dimensional human key point coordinate set under a preset initial perspective into the two-dimensional image corresponding to each perspective to obtain a two-dimensional human key point projection coordinate set corresponding to each perspective; determine each three-dimensional human key point in the three-dimensional human mesh data based on a difference between each two-dimensional human key point projection coordinate and corresponding two-dimensional human key point information, the two-dimensional human key point projection coordinate being a coordinate in the two-dimensional human key point projection coordinate set.

8. The data registration apparatus of claim 7, wherein, The detection module is specifically configured to: acquire an average value of each difference value; perform optimization analysis on multiple groups of initial three-dimensional human key point coordinate sets to acquire an average value of the difference value corresponding to each group of initial three-dimensional human key point coordinate sets; determine a target three-dimensional human key point coordinate set according to the initial three-dimensional human key point coordinate set corresponding to the third target average value, wherein the third target average value is the minimum value in the average values of the multiple groups of difference values, and the target three-dimensional human key point coordinate set includes coordinates of each three-dimensional human key point in the three-dimensional human mesh data.

Images