Skeleton mapping method, device, equipment, and storage medium

Abstract

The present application discloses a bone mapping method, device, equipment, and storage medium. The bone mapping method includes: obtaining a source bone and a target bone; determining a first mapping cost between a node of the source bone and a node of the target bone; and determining a first node mapping relationship between the source bone and the target bone using the first mapping cost. The above scheme can improve the mapping efficiency of bone nodes.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a bone mapping method and apparatus, device, and storage medium. Background Art

[0002] At present, animation migration is gradually coming into people's view. The so-called animation migration refers to using the animation data of one bone to drive the movement of another bone. However, before animation migration, it is necessary to determine the mapping relationship between the bone nodes in the two bones in order to realize animation migration. However, the current method of determining the mapping relationship between the bone nodes in the two bones is still manual, that is, manually establishing the mapping relationship between the two bones. If the number of bones is large and the structure is relatively complex, it is very time-consuming if it is still determined manually. Summary of the invention

[0003] The present application at least provides a skeleton mapping method and apparatus, device, and storage medium.

[0004] The present application provides a bone mapping method, including: obtaining a source bone and a target bone; determining a first mapping cost between a node of the source bone and a node of the target bone; and determining a first node mapping relationship between the source bone and the target bone using the first mapping cost.

[0005] Therefore, by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, the first node mapping relationship between the source bone and the target bone can be determined without manual determination, which improves the mapping efficiency of the bone nodes, saves labor costs and makes the whole process convenient and fast.

[0006] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0007] Among them, the source bone includes at least one source branch, the target bone includes at least one target branch, and the source branch and the target branch each include at least one node; using the first mapping cost, determining the first node mapping relationship between the source bone and the target bone, including: based on the first mapping cost, determining the second mapping cost between the source branch and the target branch; based on the second mapping cost, determining the first node mapping relationship between the source bone and the target bone.

[0008] Therefore, the mapping costs between the branches are determined by the mapping relationship between the nodes. The mapping costs between the branches are determined first, and a more accurate first node mapping relationship can be obtained according to the mapping costs between the branches.

[0009] Among them, based on the first mapping cost, determining the second mapping cost between the source branch and the target branch includes: determining several candidate mapping relationships between the source branch and the target branch; based on the first mapping cost, determining the candidate mapping cost required for each candidate mapping relationship; selecting the candidate mapping cost that meets the first preset condition as the second mapping cost between the source branch and the target branch.

[0010] Therefore, by selecting a candidate mapping cost that meets the first preset condition as the second mapping cost between the source branch and the target branch, the obtained second mapping cost is made more accurate.

[0011] The candidate mapping relationships include second node mapping relationships between source branches and target branches; and there is no intersection between all second node mapping relationships in the candidate mapping relationships.

[0012] Therefore, by limiting the second node mapping relationships to have no intersection, the determined second node mapping relationships are made more accurate.

[0013] Wherein, determining the candidate mapping cost required for each candidate mapping relationship based on the first mapping cost includes: determining the candidate mapping cost required for the candidate branch mapping relationship based on the first mapping cost required for each second node mapping relationship in the candidate mapping relationship.

[0014] Therefore, the candidate mapping costs required for the candidate branch mapping relationships can be determined through the first mapping costs required for each second node mapping relationship.

[0015] Among them, based on the second mapping cost, the first node mapping relationship between the source bone and the target bone is determined, including: taking one of the source branch and the target branch as the first branch, and the other as the second branch; for each first branch, determining the second branch whose second mapping cost with the first branch satisfies the second preset condition, and obtaining the second node mapping relationship between the first branch and the determined second branch, wherein the second mapping cost is determined based on the first mapping cost required for the second node mapping relationship; using the second node mapping relationships corresponding to different first branches, the first node mapping relationship between the source bone and the target bone is determined.

[0016] Therefore, by selecting the second branch whose second mapping cost with the first branch meets the second preset condition and determining the second node mapping relationship between the first branch and the second branch, the acquired first node mapping relationship is made more accurate.

[0017] Among them, determining the first mapping cost between the nodes of the source bone and the nodes of the target bone includes: obtaining the difference information between the nodes of the source bone and the nodes of the target bone regarding at least one parameter, wherein the at least one parameter includes a position parameter representing the position of the node and / or an association parameter representing the degree of association between the node and other nodes of the bone; using the difference information between the at least one parameter, determining the first mapping cost between the nodes of the source bone and the nodes of the target bone.

[0018] Therefore, by determining the first mapping cost through the difference information between the above one or more parameters, it is possible to integrate information from multiple dimensions, so that the final first mapping cost is more accurate.

[0019] Among them, the position parameters include at least one of the following: the position of the node in the world coordinate system, and the ratio of the distance between the node and the root node to the length of the branch where the node is located. The association parameters include centrality and node degree, among which the centrality is the number of branches passed by the node, and the node degree is the total number of parent nodes and child nodes of the node.

[0020] Therefore, by obtaining at least one of the above position parameters and correlation parameters, the first mapping cost obtained is made more accurate.

[0021] Among them, the first mapping cost between the nodes of the source bone and the nodes of the target bone is determined by using the difference information about at least one parameter, including: weighting the difference information about at least one parameter to obtain the first mapping cost between the nodes of the source bone and the nodes of the target bone.

[0022] Therefore, by setting corresponding weights for various parameters, the emphasis of each parameter can be increased, making the first mapping cost more accurate.

[0023] Among them, after using the first mapping cost to determine the first node mapping relationship between the source bone and the target bone, the method also includes: taking at least one of the source bone and the target bone as the bone to be tested; taking the node without the first node mapping relationship in the bone to be tested as the target node, and determining the first node mapping relationship of the target node based on the first node mapping relationship of the nodes adjacent to the target node.

[0024] Therefore, by constructing a first node mapping relationship for nodes that have not established a first node mapping relationship, the first node mapping relationship between two bones is made more complete for subsequent operations.

[0025] Wherein, based on the first node mapping relationship of the nodes adjacent to the target node, the first node mapping relationship of the target node is determined, including: based on the first node mapping relationship of the nodes adjacent to the target node, the pending node mapping relationship of the target node is determined; in response to the fact that there is no intersection between the pending node mapping relationship and the first node mapping relationship of the skeleton to be tested, the pending node mapping relationship of the target node is used as the first node mapping relationship of the target node.

[0026] Therefore, by limiting the first node mapping relationships to have no intersection, the determined first node mapping relationships are made more accurate.

[0027] The step of obtaining the target skeleton includes: extracting skeletons from the target image to obtain an initial skeleton; and clustering nodes of the initial skeleton to obtain a target skeleton.

[0028] Therefore, by clustering the extracted initial skeleton, the final target skeleton is made more concise.

[0029] The initial skeleton includes multiple nodes and at least one edge, wherein the edge is formed by connecting two adjacent nodes; the initial skeleton is clustered to obtain the target skeleton, including: in response to the edge in the initial skeleton satisfying the third preset condition, merging the two nodes corresponding to the edge into one node; wherein the third preset condition includes any one or any combination of the following: the center degree of one or two nodes corresponding to the edge is greater than a first threshold, the number of edges between the two nodes corresponding to the edge is greater than a second threshold, the joint surface area corresponding to one or two nodes corresponding to the edge is less than a third threshold, and the length of the edge is less than a fourth threshold.

[0030] Therefore, by using any one or more of the above conditions, the initial skeleton can be simplified.

[0031] Before determining the first mapping cost between the nodes of the source bone and the nodes of the target bone, the method further includes: normalizing the size parameters of the source bone and the target bone, wherein the size parameters include the distance between two adjacent nodes.

[0032] Therefore, by normalizing the sizes of the two bones first, the size difference between the two can be reduced, thereby improving the accuracy of constructing the first mapping relationship.

[0033] After determining the first node mapping relationship between the source bone and the target bone using the first mapping cost, the method further includes: migrating first animation driving data related to the source bone to the target bone to obtain second animation driving data of the target bone.

[0034] Therefore, by migrating the first animation driving data related to the source bone to the target bone after constructing the first mapping relationship, the animation driving stability of the target bone can be ensured.

[0035] The present application provides a bone mapping device, including: an acquisition module for acquiring a source bone and a target bone; a mapping cost determination module for determining a first mapping cost between a node of the source bone and a node of the target bone; and a mapping module for determining a first node mapping relationship between the source bone and the target bone using the first mapping cost.

[0036] The present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned skeleton mapping method.

[0037] The present application provides a computer-readable storage medium on which program instructions are stored. When the program instructions are executed by a processor, the above-mentioned bone mapping method is implemented.

[0038] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0039] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0042] Figure 1 It is a flowchart of an embodiment of the skeleton mapping method of the present application;

[0043] Figure 2 is a schematic diagram showing an initial skeleton in an embodiment of a skeleton mapping method of the present application;

[0044] Figure 3 is a schematic diagram showing a target skeleton according to an embodiment of the skeleton mapping method of the present application;

[0045] Figure 4 is a schematic diagram showing the joint surface of each node in an embodiment of the skeleton mapping method of the present application;

[0046] Figure 5 is a schematic diagram showing a low-resolution target skeleton in an embodiment of the skeleton mapping method of the present application;

[0047] Figure 6 is a schematic diagram showing a sub-process of step S13 in an embodiment of the skeleton mapping method of the present application;

[0048] Figure 7 is a schematic diagram showing a first mapping cost according to an embodiment of the skeleton mapping method of the present application;

[0049] Figure 8 is a schematic diagram showing some candidate mapping relationships in an embodiment of the skeleton mapping method of the present application;

[0050] Fig. 9 It is a structural schematic diagram of an embodiment of the skeleton mapping device of the present application;

[0051] Fig.10 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0052] Fig.11 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0053] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0054] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0055] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0056] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of the skeleton mapping method of the present application.

[0057] Specifically, the following steps may be included:

[0058] Step S11: Get the source skeleton and the target skeleton.

[0059] The method of obtaining the source skeleton and the target skeleton can be to directly receive the skeleton input by the user, or to receive the picture input by the user, and obtain the corresponding source skeleton and / or target skeleton by extracting the skeleton from the image, or to obtain the multidimensional grid input by the user, and obtain the corresponding skeleton by processing the multidimensional grid.

[0060] Step S12: Determine a first mapping cost between the nodes of the source skeleton and the nodes of the target skeleton.

[0061] The first mapping cost is used to represent the similarity between two nodes. Generally speaking, the higher the similarity between two nodes, the lower the cost of matching the two nodes. Therefore, by obtaining the first mapping cost between each node, the similarity between each node can be determined.

[0062] Step S13: Determine a first node mapping relationship between the source bone and the target bone using the first mapping cost.

[0063] As mentioned above, the first mapping cost between two nodes can be used to represent the similarity between the two. By referring to the first mapping cost, the first node mapping relationship between the source bone and the target bone is determined, which can be considered as indirectly determining the first node mapping relationship between the source bone and the target bone based on the similarity between the nodes in the source bone and the target bone.

[0064] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0065] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0066] In some disclosed embodiments, the method for obtaining the target skeleton may be to perform skeleton extraction on the target image to obtain the initial skeleton. Then the initial skeleton is simplified to obtain the target skeleton. Wherein, the target image refers to an image including the target object, and the target skeleton is the skeleton of the target object. The target image may be in the form of photographing the target object, or a cartoon image, or a hand-drawn image. The method for extracting the skeleton of the target image may be to perform contour extraction on the target image, and triangulate the extracted contour to obtain a three-dimensional mesh, wherein the three-dimensional mesh includes multiple vertices. Skeleton extraction is performed on the three-dimensional mesh to obtain the initial skeleton. There are many ways to simplify the initial skeleton, such as simplifying the initial skeleton according to the skeleton structure of the real body, or performing node clustering on the initial skeleton to obtain the target skeleton.

[0067] In some other disclosed embodiments, the target skeleton can also be obtained by: obtaining an input target mesh, creating a new joint for the input mesh, and executing a region growing algorithm until each polygonal mesh is assigned to a joint. The joint surface is then sampled as a point cloud. The oriented bounding box of each joint is calculated, and it is determined whether the oriented bounding boxes corresponding to each joint collide. If so, it is determined whether the distance between any two point clouds in the joint is less than a preset distance. If so, a contact is generated to obtain a contact map. The contact map is clustered to determine which edges in the skeleton can be folded, and the contact map is simplified in turn to obtain the final target skeleton. Among them, the method of generating the contact map can refer to the general prior art, which will not be repeated here. The method of clustering can also be referred to as follows, which will not be described in detail here.

[0068] To better understand the difference between the initial and target bones, see Figure 2 and Figure 3 , Figure 2 is a schematic diagram showing an initial skeleton in an embodiment of the skeleton mapping method of the present application, Figure 3 This is a schematic diagram showing a target skeleton according to an embodiment of the skeleton mapping method of the present application.

[0069] like Figure 2 and Figure 3 As shown, Figure 2 This is an initial skeleton of a target object, a dinosaur. Obviously, the initial skeleton has a large number of bone branches, and the connection relationship between each node is disordered. After simplification, the number of branches of the target skeleton is relatively small, and the connection relationship between each node is clear. By using the simplified target skeleton for skeleton mapping, the accuracy of skeleton mapping can be improved.

[0070] Wherein, the initial skeleton includes multiple nodes and at least one edge. Multiple can be two or more. Wherein, the edge is formed by connecting two adjacent nodes. The initial skeleton is clustered, and the method for obtaining the target skeleton can be in response to the edge in the initial skeleton satisfying the third preset condition, merging the two nodes corresponding to the edge into one node. Wherein, the method of merging can be to retain the parent node, delete the child node, and merge the child node into its parent node. In other public embodiments, the parent node can be merged into its child node. Of course, in other public embodiments, the two nodes can also be generated as a new node, and the new node can be the midpoint of the two nodes. Therefore, no specific provisions are made here about the method of merging. By clustering the extracted initial skeleton, the final target skeleton is more streamlined.

[0071] The third preset condition includes any one or more of the following:

[0072] 1. The node degree of one or two nodes corresponding to the edge is greater than the first threshold. The node degree is the sum of the number of the parent node of the node and its child nodes. The first threshold can be 3 or more. That is, any node corresponding to the edge includes two or more child nodes.

[0073] 2. The number of edges between the two nodes corresponding to the edge is greater than a second threshold. The second threshold can be two or more, that is, the number of edges between two adjacent nodes is greater than or equal to 2.

[0074] 3. The joint surface area corresponding to one or two nodes corresponding to the edge is less than the third threshold. The joint surface area can be calculated by skinning the skeleton first, obtaining the mesh area corresponding to each vertex, and using the mesh area as the joint surface area. Alternatively, the mesh area can be the area of ​​the mesh in the oriented bounding box corresponding to the node. The third threshold can be set by yourself and is not limited here. For a better understanding of the joint surface area, please refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram showing the joint surface of each node in an embodiment of the skeleton mapping method of the present application. Figure 4 As shown, the colors of different depths on the dinosaur surface represent that it belongs to different nodes.

[0075] 4. The length of the edge is less than a fourth threshold value. The fourth threshold value can also be set by oneself.

[0076] If only the first two conditions are used to simplify the initial skeleton, a high-resolution target skeleton is obtained. If the above four conditions are used to simplify the initial skeleton at the same time, a low-resolution target skeleton is obtained. Among them, the low-resolution target skeleton can be used for bone mapping to reduce the bone mapping time. Of course, it is also possible to first use the high-resolution target skeleton for bone mapping, and then use the low-resolution skeleton to simplify the first node mapping relationship to obtain a more streamlined first node mapping relationship. By using any one or more of the above conditions, the initial skeleton can be simplified. To better understand the high-resolution target skeleton and the low-resolution target skeleton, please also refer to Figure 3 and Figure 5 , Figure 5 is a schematic diagram showing a low-resolution target skeleton according to an embodiment of the skeleton mapping method of the present application, Figure 3 It can be considered as the high-resolution target skeleton simplified by the first and second conditions mentioned above. Figure 3 and Figure 5 As shown, the number of nodes and branches of the low-resolution target skeleton is greatly reduced compared to the high-resolution target skeleton. The high-resolution target skeleton can provide more detailed information, while the low-resolution target skeleton can better represent the rough structure of the target object.

[0077] The method for merging nodes that meet the third preset condition may be to collapse edges that meet the fourth preset condition. The fourth preset condition is the minimum edge cost. The method for calculating the minimum edge cost may be:

[0078] C(E ij )=w1C Shape (E ij )+w2C Samp (E ij );

[0079] Among them, C Shape is the shape cost, C Samp is the sampling cost. The corresponding weights are w1 and w2 respectively. The shape cost helps to maintain the overall shape of the skeleton, while the sampling cost helps to prevent the generation of long edges. The long edges here refer to the edges whose length exceeds the preset normal length.

[0080] The way to get the shape cost can be:

[0081]

[0082] in, and are the sum of the distances from the collapse center to all adjacent edges of node i and node j, respectively. The collapse center refers to the midpoint of node i and node j.

[0083] Among them, calculate ε(p) i The way can be:

[0084]

[0085] here, are the homogeneous coordinates of the three-dimensional vector p, and T represents the transpose.

[0086] Furthermore, the matrix K ij The calculation method can be:

[0087]

[0088] Where a=(n i -n j ) / ||n i -n j ||, and b = a*n i . x, y, and z are used to represent the three directions of the coordinate axis.

[0089] The sampling cost can be obtained by:

[0090]

[0091] Among them, the surface of each joint is sampled to obtain a point cloud. i refers to the centroid of the point cloud of the surface of the joint associated with node i. Similarly, n j refers to the centroid of the point cloud of the surface of the joint associated with node j. That is, n i and n j is a three-dimensional vector. Among them, ε i is the set of all edges connected to node i.

[0092] Among them, the way to obtain the source skeleton can also be to classify the target image, obtain the category of the target object, and select the skeleton model matching the category as the source skeleton. Specifically, the disclosed embodiment can adopt the predicted label mapping, and can also adopt the data set label mapping. The classification result of the predicted label mapping for the target object includes the predicted skeleton topological structure type of the target object, for example, the predicted skeleton topological structure type includes two feet, four feet, etc. That is, the process of predicting the label mapping is mainly to predict the skeleton topological structure characteristics of the target object. The classification result of the data set label mapping needs to give the specific type of the target object in the input image, for example, the target object is a cat, a dog, a giant panda, a bear, etc. The disclosed embodiment chooses to adopt the predicted label mapping. In the specific application process, if the target object is a giant panda, and the target object category given by the predicted label mapping is four feet, and the skeleton model matching the category is selected as the source skeleton, if the selected source skeleton is a four-footed bear. Although the giant panda and the bear are different, they actually have roughly the same skeleton topological structure, therefore, migrating the animation drive data of the bear to the giant panda can also appear in a natural and reasonable form. That is, although the predicted label mapping cannot obtain the correct category of the target object, it does not affect the driving of the final target skeleton. At the same time, because the predicted label mapping does not further know the specific category of the target object, the computational cost is reduced.

[0093] In some disclosed embodiments, the method for determining the first mapping cost between the nodes of the source skeleton and the nodes of the target skeleton can be: first, obtain the difference information between the nodes of the source skeleton and the nodes of the target skeleton about at least one parameter. Then use the difference information about at least one parameter to determine the first mapping cost between the nodes of the source skeleton and the nodes of the target skeleton. Specifically, at least one parameter includes a position parameter representing the position of the node and / or a correlation parameter representing the degree of association between the node and other nodes of the skeleton. Specifically, in some disclosed embodiments, the difference information of the position parameter representing the position of the node is selected to determine the first mapping cost, in some disclosed embodiments, the difference information of the correlation parameter representing the degree of association between the node and other nodes of the skeleton is selected to determine the first mapping cost, and in other disclosed embodiments, the first mapping cost is determined by combining the above two. Determining the first mapping cost through the difference information between the above one or more parameters can integrate information of multiple dimensions, so that the final first mapping cost is more accurate.

[0094] Further, the position parameter includes at least one of the following: the position of the node in the world coordinate system, and the ratio of the distance between the node and the root node to the length of the branch where the node is located. The world coordinate system can be a coordinate system with the root node of the corresponding skeleton as the origin. That is, the position of the node in the world coordinate system is specifically the relative position relationship between the node and the root node. The ratio between the distance between the node and the root node and the length of the branch where the node is located, for example, in the target skeleton, the distance between node S and the root node is z, and the length of the branch where the node is located is x, and the ratio is z / x.

[0095] Among them, the association parameters include centrality and node degree. Among them, the centrality is the number of branches that the node passes through. The node degree is the total number of parent nodes and child nodes of the node. For example, if node A in the target skeleton is in three target branches at the same time, the centrality of node A is 3. If node A has 1 child node, the node degree of node A is 2. That is, the centrality of the node is greater than or equal to its node degree. By obtaining at least one of the above position parameters and association parameters, the first mapping cost obtained is more accurate.

[0096] Among them, the method of determining the first mapping cost between the nodes of the source bone and the nodes of the target bone by using the difference information between at least one parameter can be to weight the difference information between the nodes of at least one target bone to obtain the first mapping cost between the nodes of the source bone and the nodes of the target bone. By setting corresponding weights for each parameter, the emphasis of each parameter can be improved, making the first mapping cost more accurate.

[0097] In some public embodiments, the weight corresponding to the position of the node in the world coordinate system is the largest. Also, the weights corresponding to all parameters in the node position parameters are greater than the weights corresponding to all parameters in the association parameters. The weight corresponding to the centrality is greater than the weight corresponding to the node degree. In some application scenarios, the weight corresponding to the position of the node in the world coordinate system is set to 10, the weight corresponding to the ratio of the distance between the node and the root node to the length of the branch where the node is located is 2, the weight corresponding to the centrality is 0.2, and the weight corresponding to the node degree is 1. Specifically, obtain the first mapping cost between the two nodes The formula is as follows:

[0098]

[0099] in, Refers to the position difference of the node in the world coordinate system. refers to the difference in the above proportions, refers to the difference between the node degrees of the nodes, and Refers to the difference between the centrality of nodes. 1 -a4 Represents weight.

[0100] In some other disclosed embodiments, the first mapping cost may be determined comprehensively by referring to the difference in the mesh surface area corresponding to the node. Specifically, the first mapping cost is calculated by including the mesh surface area corresponding to the node as follows:

[0101]

[0102] Among them, a 5 is the weight, and a 5 What is multiplied is the difference in surface area.

[0103] For some public embodiments, see Figure 6 , Figure 6 FIG. 1 is a schematic diagram showing a sub-process of step S13 in an embodiment of the skeleton mapping method of the present application. Figure 6 As shown, the above step S13 may specifically include the following steps S131 to S132:

[0104] Step S131: Based on the first mapping cost, determine a second mapping cost between the source branch and the target branch.

[0105] Wherein, the source skeleton includes at least one source branch, and the target skeleton includes at least one target branch. Wherein, at least one can be one or two or more. The number of branches of the source skeleton and the target skeleton can be the same or different. For example, in some scenes, the number of branches of the source skeleton is greater than the number of branches of the target skeleton, and in other scenes, the number of branches of the source skeleton is less than or equal to the number of branches of the target skeleton. The source branch and the target branch include at least one node.

[0106] Specifically, several candidate mapping relationships between source branches and target branches are determined. Figure 7 and Figure 8 As shown, Figure 7 is a schematic diagram showing a first mapping cost according to an embodiment of the skeleton mapping method of the present application, Figure 8 1 is a schematic diagram showing some candidate mapping relationships in an embodiment of the skeleton mapping method of the present application. Figure 7 Only the first mapping cost between some nodes is shown. Figure 8 Shown with Figure 7 Some candidate mapping relationships of corresponding nodes. Figure 7In it is assumed that nodes A, B, C, and D in the first column belong to a target branch in the target skeleton, and nodes 1, 2, and 3 in the first row belong to a source branch in the source skeleton. Among them, node A and node 1 can be considered as the nodes closest to the root node in their respective branches, and node D and node 3 can be considered as leaf nodes in their respective branches. A leaf node refers to the node in the branch that is farthest from the root node, that is, a node without child nodes. That is, the root node of the target skeleton is mapped to the root node of the source skeleton by default. As a result, the first mapping cost between the two root nodes can be calculated less in subsequent calculations, thereby reducing resource consumption. Figure 7 and Figure 8 As shown, A1 indicates that a mapping is established between node A and node 1, and the same applies to B2, C3, D3 and C2. The "-" here connects two groups of mappings, where A1-B2-C3 represents the candidate mapping relationship between the target branch and the source branch. The first mapping cost between node 1 and node A is 0.1, the first mapping cost between node 1 and node B is 0.4, the first mapping cost between node 2 and node B is 0.2, the first mapping cost between node 2 and node C is 0.6, the first mapping cost between node 3 and node C is 0.3, the first mapping cost between node 3 and node D is 0.1, and the first mapping cost between the remaining nodes is not shown. Some of the candidate mapping relationships shown include A1-B2-C3, A1-B2-D3, A1-C2-D3 and B1-C2-D3.

[0107] Among them, the candidate mapping relationship includes the second node mapping relationship between the source branch and the target branch. Specifically, the second node mapping relationship can be considered as part of the first node mapping relationship. Following the above example, the second node mapping relationship can be A1, B2, and C3, that is, A1 represents that node A and node 1 construct the second node mapping relationship, and B2 and C3 are the same. In some public embodiments, there is no intersection between all the second node mapping relationships in the candidate mapping relationship. That is, in each group of candidate mapping relationships, there is no intersection between each second node mapping relationship. For example, assuming that node A in the target skeleton establishes a second node mapping relationship with node 2 in the source skeleton, and node B establishes a second node mapping relationship with node 1, it is considered that A2 and B1 have an intersection. In other public embodiments, in each group of candidate mapping relationships, there is no overlap between each second node mapping relationship, that is, there is no situation where multiple nodes in the target skeleton and the same node in the source skeleton construct a second node mapping relationship. In short, when determining the candidate mapping relationship, it can be constructed from the root node to the leaf node in sequence, or from the leaf node to the root node in sequence. By limiting the second node mapping relationships to have no intersection, the determined second node mapping relationships are made more accurate.

[0108] Then, based on the first mapping cost, the candidate mapping cost required for each candidate mapping relationship is determined. The mapping cost corresponding to each second node mapping relationship is the first mapping cost. Specifically, based on the first mapping cost required for each second node mapping relationship in the candidate mapping relationship, the candidate mapping cost required for the candidate branch mapping relationship is determined. That is, in the above example, when the candidate mapping relationship is A1-B2-C3, the second node mapping relationships included in the candidate mapping relationship are A1, B2, and C3. At this time, the first mapping cost of A1 is 0.1, the first mapping cost of B2 is 0.2, and the first mapping cost of C3 is 0.1. At this time, the candidate mapping cost corresponding to the candidate mapping relationship is 0.4. The candidate mapping cost required for the candidate branch mapping relationship can be determined by the first mapping cost required for each second node mapping relationship.

[0109] After obtaining the candidate mapping costs required for each candidate mapping relationship, the candidate mapping costs that meet the first preset condition are selected as the second mapping costs between the source branch and the target branch. By selecting the candidate mapping costs that meet the first preset condition as the second mapping costs between the source branch and the target branch, the obtained second mapping costs are more accurate.

[0110] The first preset condition can be that the candidate mapping cost is the smallest. That is, the minimum value is selected from all candidate mapping costs as the second mapping cost. In the above example, Figure 3 and Figure 4 In the partial candidate mapping relationships shown, the candidate mapping cost corresponding to A1-B2-C3 is 0.6, the candidate mapping cost corresponding to A1-B2-D3 is 0.4, the candidate mapping cost corresponding to A1-C2-D3 is 0.8, and the candidate mapping cost corresponding to B1-C2-D3 is 1.1. Among them, the minimum value is 0.4, and 0.4 is used as the second mapping cost between the source branch and the target branch. And, the second node mapping relationship between the source branch and the target branch is determined to be A1-B2-D3. Optionally, the second node mapping relationship between the source branch and the target branch is represented by a first matrix, wherein each element in the first matrix represents a specific branch.

[0111] As described above, in the embodiment of the present disclosure, the first preset condition is that the first mapping cost is the minimum, so the minimum second mapping cost is obtained. The way is as follows:

[0112]

[0113] Step S132: Determine a first node mapping relationship between the source bone and the target bone according to the second mapping cost.

[0114] Specifically, one of the source branch and the target branch is used as the first branch, and the other is used as the second branch. For each first branch, a second branch whose second mapping cost between the first branches satisfies the second preset condition is determined. The second preset condition may be a minimum value. That is, the second branch with the smallest second mapping cost with the first branch is selected. As described above, the second mapping cost is determined based on the first mapping cost required for the second node mapping relationship. In some disclosed embodiments, a second matrix is ​​constructed. Specifically, a second matrix is ​​constructed according to each branch of the two skeletons, wherein each element in the second matrix is ​​the second mapping cost corresponding to the branch mapping group.

[0115] Specifically, the second matrix B can be expressed as:

[0116]

[0117] in, is the first matrix, used to represent the branch and The second mapping cost between branches is the branch of bone h, It is a branch of bone a.

[0118] Then, the first node mapping relationship between the source skeleton and the target skeleton is determined by using the second node mapping relationship corresponding to different first branches. Because the second node mapping relationship between the branches is known, the first node mapping relationship between the nodes in the skeleton can be determined after the mapping relationship between the branches is determined. The mapping cost between the branches is determined by the mapping relationship between the nodes. The mapping cost between the branches is first determined, and a more accurate first node mapping relationship can be obtained based on the mapping cost between the branches.

[0119] Further, by selecting the second branch whose second mapping cost with the first branch satisfies the second preset condition, and determining the second node mapping relationship between the first branch and the second branch, the first node mapping relationship obtained is more accurate. In other disclosed embodiments, in addition to determining the mapping relationship between each branch with reference to the second mapping cost between each branch, the structure of the source skeleton and / or the target skeleton can also be referenced. For example, if two bone branches in the source skeleton are in a symmetrical state, the target skeleton should use two symmetrical branches to establish a mapping relationship with it, and cannot use asymmetric branches to establish a mapping relationship with two symmetrical branches of the source skeleton. For example, if the source skeleton and the target skeleton are both human skeletons, the branches of the two arms in the source skeleton are symmetrical branches, and the branches of the two legs are symmetrical branches. At this time, the target skeleton cannot use the branches of one arm and one leg to establish a mapping relationship with the branches of the two arms in the source skeleton. Of course, this is just an example. In practice, the first node mapping relationship can be established for the two specifically according to the structure of the source skeleton or the target skeleton.

[0120] In some disclosed embodiments, at least one of the source bone and the target bone is used as the bone to be tested. For example, only the source bone is used as the bone to be tested, or only the target bone is used as the bone to be tested, or both the source bone and the target bone can be used as the bones to be tested.

[0121] The node without the first node mapping relationship in the skeleton to be checked is taken as the target node. And based on the first node mapping relationship of the node adjacent to the target node, the first node mapping relationship of the target node is determined. The node adjacent to the target node can be the parent node of the target node, or it can be the child node of the target node. For example, the first node mapping relationship of its child node is taken as the mapping relationship of the target node. By constructing the first node mapping relationship for the node without the first node mapping relationship, the first node mapping relationship between the two skeletons is more complete for subsequent operations.

[0122] Optionally, based on the first node mapping relationship of the node adjacent to the target node, the pending node mapping relationship of the target node is determined. Wherein, the method for determining the pending node mapping relationship is as described above. If the child nodes of the current node are greater than 1, if the pending node mapping relationship determined according to the mapping relationship of one of the child nodes may cause the first node mapping relationships of other branches to cross, therefore, the pending mapping relationship in which the first node mapping relationships in all branches do not cross is obtained as the first node mapping relationship of the target node. Specifically, in response to the fact that there is no intersection between the pending node mapping relationship and the first node mapping relationship of the skeleton to be tested, the pending node mapping relationship of the target node is used as the first node mapping relationship of the target node. Of course, if all the pending node mapping relationships will cause the first node mapping relationship to cross, the first node mapping relationship will not be established for the target node. By limiting the non-intersection between the first node mapping relationships, the determined first node mapping relationship is made more accurate.

[0123] In some application scenarios, if the skeleton to be tested is the target skeleton, and the target node has only one child node, the mapping relationship of the target node can be determined based on the first node mapping relationship of its child nodes, for example, the target node and its child nodes are mapped to the same node in the source skeleton. In some application scenarios, if the target node has multiple child nodes, the nodes according to multiple branches are merged to remove the upper limit node. Specifically, if there are redundant nodes in the skeleton to be tested, and there are idle nodes in another skeleton, a first node mapping relationship is established between the redundant nodes and the idle nodes. If two or more nodes connected in the skeleton to be tested are mapped to the same node of another skeleton, it is considered that there are redundant nodes in the skeleton to be tested. The existence of an idle node in another skeleton refers to that the adjacent nodes of the node having a mapping relationship with the redundant node do not have a first node mapping relationship. For example, when the skeleton to be tested is the target skeleton, if the continuous nodes J and their child nodes K in the first branch of the target skeleton are simultaneously mapped to the same node H in the second branch of the source skeleton, and the child node I of node H does not have a first node mapping relationship, it is determined whether establishing a first mapping relationship between node K and node I will cause an intersection between the first node mapping relationships. If so, keep mapping node J and its child node K to the same node H in the second branch of the source skeleton at the same time. If not, establish a first mapping relationship between node K and node I. In other disclosed embodiments, when the skeleton to be tested is the target skeleton, if the source skeleton carries animation data, it is determined whether the nodes in the source skeleton that meet the preset requirements have constructed a first node mapping relationship. If not, a first node mapping relationship is constructed for the node. Among them, the preset requirement can be that the node degree is greater than 3, and the premise for establishing the first node mapping relationship for the node is the same as above, that is, there is no intersection between each first node mapping relationship.

[0124] In some disclosed embodiments, if the undetermined node mapping relationship intersects with the first node mapping relationship of the skeleton to be tested, a first node mapping relationship is established between the target node and the root node of another skeleton. At this time, the root node of the skeleton where the target node is located and the target node are simultaneously mapped to the root node of another skeleton.

[0125] In some disclosed embodiments, before determining the first mapping cost between the nodes of the source bone and the target bone, the following steps may also be performed: normalizing the size parameters of the source bone and the target bone. The size parameters include the distance between two adjacent nodes. For example, if the target bone and the source bone are both humanoid bones, if the distance between the two nodes in one of the bones is 100 mm, and the distance between the two nodes in the other bone is 1 mm, the distance between the two nodes of the two is too different, which may cause the established first node mapping relationship to be inaccurate. Therefore, normalizing the size parameters of the source bone and the target bone can reduce the gap between the two, thereby improving the accuracy of the first node mapping relationship.

[0126] In some public embodiments, before determining the first mapping cost between the nodes of the source skeleton and the target skeleton, the following steps may also be performed: unify the directions of the source skeleton and the target skeleton. Wherein, the unification method may be manually adjusted, or may be adjusted to a third direction according to the direction of the root nodes of the current source skeleton and the target skeleton, or the direction of the root node of one of the skeletons is adjusted to the direction of the root node of another skeleton, thereby achieving the unification of the direction of the two skeletons. If the directions of the two skeletons are different, the mapping relationship may be inaccurate. For example, if the source skeleton and the target skeleton are both human skeletons. If the root node of the source skeleton faces north, and the root node of the target skeleton faces south, that is, the source skeleton and the target skeleton stand back to back, if skeleton mapping is performed according to the current direction, it is likely to cause the left hand of the source skeleton to be mapped to the right hand of the target skeleton.

[0127] Wherein, after determining the first node mapping relationship between the source skeleton and the target skeleton using the first mapping cost, the following steps can also be performed: the first animation driving data related to the source skeleton is migrated to the target skeleton to obtain the second animation driving data of the target skeleton. Wherein, when the source skeleton has not been modified, the first animation driving data related to the source skeleton is the first animation driving data created when constructing the source skeleton. If the source skeleton has been modified, the first animation driving data related to the source skeleton is also the animation driving data after the modification.

[0128] After obtaining the first node mapping relationship between the source bone and the target bone, node alignment is performed.

[0129] Among them, node alignment is mainly performed to determine the first pose transformation relationship between the source skeleton and the target skeleton. Specifically, in the order from the root source node to the leaf source node, each source node in the source skeleton is aligned with the corresponding mapped target node in the target skeleton to obtain the first pose transformation relationship between each source node and the mapped target node. The root node is the node with the largest number of bone branches. The root node refers to the root node in the final source skeleton, and similarly, the root target node refers to the root node of the final target skeleton. The final source skeleton and the final target skeleton refer to the source skeleton and the target skeleton after topological structure alignment. Among them, a leaf node refers to a node with a parent node but no child node. The leaf source node refers to the leaf node in the final source skeleton, and the leaf target node refers to the leaf node in the final target skeleton.

[0130] The first pose transformation relationship is the transformation relationship between the source node and the mapped target node in the first coordinate system. By translating the root source node of the final source skeleton and the root target node of the final target skeleton to the origin of the first coordinate system, the offset between the root source node of the final source skeleton and the root target node of the final target skeleton can be obtained. Specifically, for each source node in the final source skeleton, the offset required to align the source node with the mapped target node is obtained. Among them, the offset includes a translation component and a rotation component. Generally speaking, the translation component includes a scaling component. Then, based on the offset corresponding to the source node, the first pose transformation relationship of the source node is obtained.

[0131] Based on the offset corresponding to the source node, the first pose transformation relationship of the source node is obtained. Specifically, based on the offsets corresponding to the source node and the parent node of the source node, the first pose transformation relationship of the source node is obtained. Among them, the parent node of the source node is the first parent node of the source node in the final source skeleton, the root node, and the node between the first parent node and the root node. Among them, the offsets can be represented by a matrix. Specifically, by performing matrix multiplication on the offsets corresponding to the source node and the parent node of the source node, the first pose transformation relationship of the source node can be obtained.

[0132] Thus, the animation data on the source skeleton can be transferred to the target skeleton to drive the target in the image to be processed to move.

[0133] The above scheme can ensure the animation driving stability of the target bone by migrating the first animation driving data related to the source bone to the target bone after constructing the first mapping relationship.

[0134] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0135] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0136] The skeletal mapping method may be executed by a skeletal mapping device, for example, the skeletal mapping method may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the skeletal mapping method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0137] See also Fig. 9 , Fig. 9 It is a structural diagram of an embodiment of the skeleton mapping device of the present application. The skeleton mapping device 10 includes an acquisition module 11, a mapping cost determination module 12 and a mapping module 13. The acquisition module 11 is used to acquire the source skeleton and the target skeleton; the mapping cost determination module 12 is used to determine the first mapping cost between the node of the source skeleton and the node of the target skeleton; the mapping module 13 is used to determine the first node mapping relationship between the source skeleton and the target skeleton using the first mapping cost.

[0138] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0139] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0140] In some disclosed embodiments, the source skeleton includes at least one source branch, the target skeleton includes at least one target branch, and the source branch and the target branch each include at least one node; the mapping module 13 uses a first mapping cost to determine a first node mapping relationship between the source skeleton and the target skeleton, including: determining a second mapping cost between the source branch and the target branch based on the first mapping cost; determining the first node mapping relationship between the source skeleton and the target skeleton based on the second mapping cost.

[0141] In the above scheme, the mapping costs between the branches are determined by the mapping relationship between the nodes. The mapping costs between the branches are determined first, and a more accurate first node mapping relationship can be obtained according to the mapping costs between the branches.

[0142] In some disclosed embodiments, the mapping module 13 determines a second mapping cost between a source branch and a target branch based on the first mapping cost, including: determining several candidate mapping relationships between the source branch and the target branch; determining a candidate mapping cost required for each candidate mapping relationship based on the first mapping cost; and selecting a candidate mapping cost that meets a first preset condition as the second mapping cost between the source branch and the target branch.

[0143] The above solution selects a candidate mapping cost that meets the first preset condition as the second mapping cost between the source branch and the target branch, so that the obtained second mapping cost is more accurate.

[0144] In some disclosed embodiments, the candidate mapping relationships include second node mapping relationships between source branches and target branches; wherein no intersection exists between all second node mapping relationships in the candidate mapping relationships.

[0145] The above solution makes the determined second node mapping relationship more accurate by limiting the non-intersection between the second node mapping relationships.

[0146] In some disclosed embodiments, the mapping module 13 determines the candidate mapping cost required for each candidate mapping relationship based on the first mapping cost, including: determining the candidate mapping cost required for the candidate branch mapping relationship based on the first mapping cost required for each second node mapping relationship in the candidate mapping relationship.

[0147] In the above solution, the candidate mapping costs required for the candidate branch mapping relationships can be determined through the first mapping costs required for each second node mapping relationship.

[0148] In some disclosed embodiments, the mapping module 13 determines the first node mapping relationship between the source bone and the target bone based on the second mapping cost, including: taking one of the source branch and the target branch as the first branch and the other as the second branch; for each first branch, determining a second branch whose second mapping cost with the first branch satisfies a second preset condition, and obtaining the second node mapping relationship between the first branch and the determined second branch, wherein the second mapping cost is determined based on the first mapping cost required for the second node mapping relationship; and determining the first node mapping relationship between the source bone and the target bone using the second node mapping relationships corresponding to different first branches.

[0149] The above scheme selects the second branch whose second mapping cost with the first branch satisfies the second preset condition, and thereby determines the second node mapping relationship between the first branch and the second branch, so that the acquired first node mapping relationship is more accurate.

[0150] In some disclosed embodiments, the mapping cost determination module 12 determines a first mapping cost between a node of a source bone and a node of a target bone, including: obtaining difference information between the nodes of the source bone and the nodes of the target bone regarding at least one parameter, wherein the at least one parameter includes a position parameter indicating the position of the node and / or an association parameter indicating the degree of association between the node and other nodes of the bone in which it is located; and determining a first mapping cost between the nodes of the source bone and the nodes of the target bone using the difference information between the at least one parameter.

[0151] The above solution determines the first mapping cost by using the difference information between the above one or more parameters, and can integrate information from multiple dimensions to make the final first mapping cost more accurate.

[0152] In some disclosed embodiments, the location parameters include at least one of the following: the position of the node in the world coordinate system, and the ratio of the distance between the node and the root node to the length of the branch where the node is located, and the association parameters include centrality and node degree, wherein the centrality is the number of branches the node passes through, and the node degree is the total number of parent nodes and child nodes of the node.

[0153] The above solution makes the first mapping cost more accurate by obtaining at least one of the above position parameters and correlation parameters.

[0154] In some disclosed embodiments, the mapping cost determination module 12 uses difference information about at least one parameter to determine a first mapping cost between the nodes of the source bone and the nodes of the target bone, including: weighting the difference information about at least one parameter to obtain the first mapping cost between the nodes of the source bone and the nodes of the target bone.

[0155] The above scheme can improve the emphasis of each parameter by setting corresponding weights for each parameter, so that the first mapping cost is more accurate.

[0156] In some disclosed embodiments, after the mapping module 13 determines the first node mapping relationship between the source bone and the target bone using the first mapping cost, the mapping module 13 is also used to: use at least one of the source bone and the target bone as the bone to be tested; use the node in the bone to be tested that does not have a first node mapping relationship as the target node, and determine the first node mapping relationship of the target node based on the first node mapping relationship of the nodes adjacent to the target node.

[0157] The above scheme makes the first node mapping relationship between two bones more complete by constructing the first node mapping relationship for the nodes that have not established the first node mapping relationship, so as to facilitate subsequent operations.

[0158] In some disclosed embodiments, the mapping module 13 determines the first node mapping relationship of the target node based on the first node mapping relationship of nodes adjacent to the target node, including: determining the pending node mapping relationship of the target node based on the first node mapping relationship of nodes adjacent to the target node; in response to the absence of intersection between the pending node mapping relationship and the first node mapping relationship of the skeleton to be inspected, using the pending node mapping relationship of the target node as the first node mapping relationship of the target node.

[0159] The above solution makes the determined first node mapping relationship more accurate by limiting the non-intersection between the first node mapping relationships.

[0160] In some disclosed embodiments, the acquisition module 11 is used to acquire the target skeleton, including: performing skeleton extraction on the target image to obtain the initial skeleton; performing node clustering on the initial skeleton to obtain the target skeleton.

[0161] The above scheme clusters the extracted initial skeleton to make the final target skeleton more concise.

[0162] In some disclosed embodiments, the initial skeleton includes multiple nodes and at least one edge, wherein the edge is formed by connecting two adjacent nodes; the initial skeleton is clustered to obtain a target skeleton, including: in response to an edge in the initial skeleton satisfying a third preset condition, merging two nodes corresponding to the edge into one node; wherein the third preset condition includes any one or any combination of the following: the center degree of one or two nodes corresponding to the edge is greater than a first threshold, the number of edges between the two nodes corresponding to the edge is greater than a second threshold, the joint surface area corresponding to one or two nodes corresponding to the edge is less than a third threshold, and the length of the edge is less than a fourth threshold.

[0163] The above scheme can simplify the initial skeleton by satisfying any one or more of the above conditions.

[0164] In some disclosed embodiments, before determining the first mapping cost between the nodes of the source bone and the nodes of the target bone, the mapping cost determination module 12 is further used to: normalize the size parameters of the source bone and the target bone, wherein the size parameters include the distance between two adjacent nodes.

[0165] The above solution can reduce the size difference between the two bones by normalizing the sizes of the two bones first, thereby improving the accuracy of constructing the first mapping relationship.

[0166] In some disclosed embodiments, after the mapping module 13 uses the first mapping cost to determine the first node mapping relationship between the source bone and the target bone, it is also used to: migrate the first animation driving data related to the source bone to the target bone to obtain the second animation driving data of the target bone.

[0167] The above scheme can ensure the animation driving stability of the target bone by migrating the first animation driving data related to the source bone to the target bone after constructing the first mapping relationship.

[0168] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0169] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0170] See also Fig.10 , Fig.10 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 20 includes a memory 21 and a processor 22, and the processor 22 is used to execute program instructions stored in the memory 21 to implement the steps in any of the above-mentioned bone mapping method embodiments. In a specific implementation scenario, the electronic device 20 may include but is not limited to: medical equipment, microcomputers, desktop computers, servers, and in addition, the electronic device 20 may also include mobile devices such as laptops and tablet computers, which are not limited here.

[0171] Specifically, the processor 22 is used to control itself and the memory 21 to implement the steps in any of the above-mentioned skeletal mapping method embodiments. The processor 22 can also be called a CPU (Central Processing Unit). The processor 22 may be an integrated circuit chip with signal processing capabilities. The processor 22 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 22 can be implemented by an integrated circuit chip.

[0172] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0173] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0174] See also Fig.11 , Fig.11 The computer-readable storage medium 30 stores program instructions 31 that can be executed by a processor, and the program instructions 31 are used to implement the steps in any of the above-mentioned bone mapping method embodiments.

[0175] The above scheme can determine the first node mapping relationship between the source bone and the target bone by using the first mapping cost between the nodes of the source bone and the nodes of the target bone, without manual determination, thereby improving the mapping efficiency of the bone nodes, saving labor costs and making the whole process convenient and fast.

[0176] In addition, the use of methods other than machine learning to achieve bone node mapping does not require pre-training of the machine learning model, which simplifies the mapping implementation process, improves the mapping efficiency of bone nodes, and does not require manual annotation of training samples, saving labor costs. Moreover, considering the differences in the node structures of different bones, the machine learning model obtained by training with certain samples for bone node mapping may not be suitable for bones with different structures and has certain limitations.

[0177] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0178] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0179] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0180] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A skeleton mapping method, characterized in that: include: Acquire a source skeleton and a target skeleton, wherein the source skeleton includes at least one source branch, the target skeleton includes at least one target branch, and each of the source branch and the target branch includes at least one node; Determine a first mapping cost between the nodes of the source skeleton and the nodes of the target skeleton, wherein the first mapping cost is used to characterize a similarity relationship between the nodes of the source skeleton and the nodes of the target skeleton; Determine a first node mapping relationship between the source bone and the target bone using the first mapping cost; Among them, the use of the first mapping cost to determine the first node mapping relationship between the source bone and the target bone includes: based on the first mapping cost, determining the second mapping cost between the source branch and the target branch; based on the second mapping cost, determining the first node mapping relationship between the source bone and the target bone.

2. The method according to claim 1, characterized in that The determining, based on the first mapping cost, a second mapping cost between the source branch and the target branch comprises: Determine several candidate mapping relationships between the source branch and the target branch; Determining a candidate mapping cost required for each of the candidate mapping relationships based on the first mapping cost; The candidate mapping cost that meets the first preset condition is selected as the second mapping cost between the source branch and the target branch.

3. The method according to claim 2, characterized in that The candidate mapping relationship includes a second node mapping relationship between the source branch and the target branch; wherein, There is no intersection between all the second node mapping relationships in the candidate mapping relationships; and / or, determining the candidate mapping cost required for each of the candidate mapping relationships based on the first mapping cost includes: Based on the first mapping costs required by each of the second node mapping relationships in the candidate mapping relationships, a candidate mapping cost required by the candidate mapping relationship is determined.

4. The method according to claim 2, characterized in that The determining, based on the second mapping cost, a first node mapping relationship between the source bone and the target bone comprises: One of the source branch and the target branch is used as a first branch, and the other one is used as a second branch; For each of the first branches, determine the second branch whose second mapping cost with the first branch satisfies a second preset condition, and obtain a second node mapping relationship between the first branch and the determined second branch, wherein the second mapping cost is determined based on the first mapping cost required for the second node mapping relationship; The first node mapping relationship between the source bone and the target bone is determined by using the second node mapping relationships corresponding to different first branches.

5. The method according to any one of claims 1 to 4, characterized in that: The determining of a first mapping cost between a node of the source bone and a node of the target bone comprises: Obtaining difference information between the nodes of the source skeleton and the nodes of the target skeleton about at least one parameter, wherein the at least one parameter includes a position parameter indicating the position of the node and / or a correlation parameter indicating the degree of correlation between the node and other nodes of the skeleton; Using the difference information about at least one parameter, a first mapping cost between the nodes of the source bone and the nodes of the target bone is determined.

6. The method according to claim 5, characterized in that The position parameter includes at least one of the following: the position of the node in the world coordinate system, and the ratio of the distance between the node and the root node to the length of the branch where the node is located; the association parameter includes the centrality and the node degree, wherein the centrality is the number of branches passed by the node, and the node degree is the total number of parent nodes and child nodes of the node; And / or, the determining a first mapping cost between the nodes of the source bone and the nodes of the target bone by using the difference information between at least one parameter comprises: The difference information about at least one parameter is weighted to obtain a first mapping cost between the nodes of the source bone and the nodes of the target bone.

7. The method according to any one of claims 1 to 4, characterized in that: After determining the first node mapping relationship between the source bone and the target bone by using the first mapping cost, the method further includes: Taking at least one of the source bone and the target bone as a bone to be inspected; A node in the skeleton to be checked that has no first node mapping relationship is taken as a target node, and the first node mapping relationship of the target node is determined based on the first node mapping relationship of the nodes adjacent to the target node.

8. The method according to claim 7, characterized in that The determining the first node mapping relationship of the target node based on the first node mapping relationship of the nodes adjacent to the target node includes: Determine a pending node mapping relationship of the target node based on the first node mapping relationship of the nodes adjacent to the target node; In response to the fact that there is no intersection between the pending node mapping relationship and the first node mapping relationship of the skeleton to be checked, the pending node mapping relationship of the target node is used as the first node mapping relationship of the target node.

9. The method according to any one of claims 1 to 4, characterized in that: Get the target skeleton, including: Extract bones from the target image to obtain the initial bones; Perform node clustering on the initial skeleton to obtain the target skeleton.

10. The method according to claim 9, characterized in that The initial skeleton includes a plurality of nodes and at least one edge, wherein the edge is formed by connecting two adjacent nodes; performing node clustering on the initial skeleton to obtain the target skeleton includes: In response to the edge in the initial skeleton satisfying a third preset condition, merging the two nodes corresponding to the edge into one node; Among them, the third preset condition includes any one or any combination of the following: the node degree of one or two nodes corresponding to the edge is greater than a first threshold, the number of edges between two nodes corresponding to the edge is greater than a second threshold, the joint surface area corresponding to one or two nodes corresponding to the edge is less than a third threshold, and the length of the edge is less than a fourth threshold.

11. The method according to claim 1, characterized in that: Before determining the first mapping cost between the nodes of the source bone and the nodes of the target bone, the method further includes: Normalizing the size parameters of the source bone and the target bone, wherein the size parameters include the distance between two adjacent nodes; And / or, after determining the first node mapping relationship between the source bone and the target bone by using the first mapping cost, the method further includes: The first animation driving data related to the source bone is migrated to the target bone to obtain the second animation driving data of the target bone.

12. A skeleton mapping device, characterized in that: include: An acquisition module, used for acquiring a source skeleton and a target skeleton, wherein the source skeleton includes at least one source branch, the target skeleton includes at least one target branch, and the source branch and the target branch each include at least one node; A mapping cost determination module, used to determine a first mapping cost between the nodes of the source skeleton and the nodes of the target skeleton, wherein the first mapping cost is used to characterize the similarity relationship between the nodes of the source skeleton and the nodes of the target skeleton; A mapping module, used to determine a first node mapping relationship between the source bone and the target bone using the first mapping cost; The mapping module determines a first mapping cost of a first node mapping relationship between the source bone and the target bone using a first mapping cost, and the manner of determining the first node mapping relationship between the source bone and the target bone includes: determining a second mapping cost between the source branch and the target branch based on the first mapping cost; According to the second mapping cost, a first node mapping relationship between the source bone and the target bone is determined.

13. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images