3D Model Processing Method, Apparatus, Device, and Storage Medium

Abstract

The present disclosure relates to a three-dimensional model processing method, apparatus, device, and storage medium. The method includes: obtaining a three-dimensional model corresponding to a fitting object; determining a plurality of key points at a target position in the three-dimensional model; and generating an auxiliary model at the target position according to the plurality of key points, where the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during three-dimensional rendering. The three-dimensional model processing method provided by the present disclosure can adjust the occlusion relationship between the three-dimensional model of the fitting object and the virtual fitting part of the fitting user, so as to obtain a more realistic virtual fitting effect.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular, to a three-dimensional model processing method, apparatus, device, and storage medium. Background Art

[0002] With the continuous development of augmented reality technology, a try-on user can apply an application program on an electronic device such as a terminal to achieve the effect of virtual try-on. For example, try on shoes, clothes, jewelry, etc., so that the try-on user can select satisfactory products without leaving home.

[0003] However, the inventors of the present application found that during the virtual try-on process of the try-on user, there are certain problems with the occlusion relationship between the try-on object displayed in the picture and the try-on part of the try-on user, resulting in an unrealistic try-on effect. Summary of the Invention

[0004] To solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a three-dimensional model processing method, apparatus, device, and storage medium, which can adjust the occlusion relationship between the three-dimensional model of the try-on object and the virtual try-on part of the try-on user, so as to obtain a more realistic virtual try-on effect.

[0005] In a first aspect, an embodiment of the present disclosure provides a three-dimensional model processing method, including:

[0006] Obtaining a three-dimensional model corresponding to a try-on object;

[0007] Determining a plurality of key points at a target position in the three-dimensional model;

[0008] Generating an auxiliary model at the target position according to the plurality of key points, where the auxiliary model is used to adjust the occlusion relationship between the virtual try-on object generated by rendering and the virtual try-on part of the try-on user during three-dimensional rendering.

[0009] In a second aspect, an embodiment of the present disclosure provides a three-dimensional model processing apparatus, including:

[0010] An obtaining module, configured to obtain a three-dimensional model corresponding to a try-on object;

[0011] A determining module, configured to determine a plurality of key points at a target position in the three-dimensional model;

[0012] A generating module, configured to generate an auxiliary model at the target position according to the plurality of key points, where the auxiliary model is used to adjust the occlusion relationship between the virtual try-on object generated by rendering and the virtual try-on part of the try-on user during three-dimensional rendering.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0014] Memory;

[0015] Processor; and

[0016] Computer program;

[0017] wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the method according to the first aspect.

[0019] The three-dimensional model processing method, device, equipment and storage medium provided by the embodiments of the present disclosure determine a plurality of key points at a target position in the three-dimensional model corresponding to the fitting object, generate an auxiliary model at the target position according to the plurality of key points, and the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the three-dimensional rendering process, realizing the adjustment of the occlusion relationship between the three-dimensional model of the fitting object and the virtual fitting part of the fitting user, so as to obtain a more realistic virtual fitting effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure and used together with the specification to explain the principles of the present disclosure.

[0021] To more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0022] Figure 1 A page schematic diagram of an AR fitting provided by an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of a three-dimensional model processing method provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of an auxiliary model provided by an embodiment of the present disclosure;

[0025] Figure 4 A flowchart of a three-dimensional model processing method provided by another embodiment of the present disclosure;

[0026] Figure 5Schematic diagram of a reference plane provided by an embodiment of the present disclosure;

[0027] Figure 6 Schematic diagram of some nodes provided by an embodiment of the present disclosure;

[0028] Figure 7 Flowchart of a 3D model processing method provided by another embodiment of the present disclosure;

[0029] Figure 8 Schematic diagram of generating an auxiliary model provided by an embodiment of the present disclosure;

[0030] Figure 9 Flowchart of a 3D model processing method provided by another embodiment of the present disclosure;

[0031] Figure 10 Schematic diagram of multiple nodes provided by an embodiment of the present disclosure;

[0032] Figure 11 Schematic diagram of a group of nodes provided by an embodiment of the present disclosure;

[0033] Figure 12 Flowchart of a 3D model processing method provided by another embodiment of the present disclosure;

[0034] Figure 13 Schematic structural diagram of a 3D model processing device provided by an embodiment of the present disclosure;

[0035] Figure 14 Schematic structural diagram of an embodiment of an electronic device provided by an embodiment of the present disclosure. Detailed implementation manners

[0036] In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

[0037] Many specific details are set forth in the following description in order to fully understand the present disclosure, but the present disclosure may be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present disclosure, rather than all of the embodiments.

[0038] With the continuous development of AR (Augmented Reality) technology, a try-on user (also referred to as a "user") can apply an application program on an electronic device such as a terminal to achieve the effect of AR try-on. For example, try on shoes, clothes, jewelry, etc., so that the try-on user can select a satisfactory product without leaving home.

[0039] Taking shoes as an example, the scenario of AR shoe try-on is as follows: The user trying on shoes triggers the AR try-on icon on the shoe details page of the online mall to enter the AR try-on. The device automatically turns on the camera. The user trying on shoes aims the camera at their feet. The feet of the user trying on shoes that are captured in real time will be displayed on the shooting page. The application installed on the device recognizes the position and posture of the feet displayed on the shooting page, adjusts the three-dimensional model of the shoes to be tried on currently according to the position and posture of the feet, and renders the image of the shoes to be tried on. From the try-on screen, it seems as if the user trying on shoes has put on the new shoes they intend to buy, as Figure 1 shown in a schematic diagram of a page for AR shoe try-on. Additionally, since the feet of the user trying on shoes are captured in real time by the camera, if the feet of the user trying on shoes move, the feet displayed in the try-on screen will also move in real time. At this time, the application synchronously adjusts the position and / or posture of the three-dimensional model worn on the feet and renders it in real time, so as to achieve the effect that the three-dimensional model of the shoes can track the feet of the user trying on shoes.

[0040] It can be understood that after all, AR try-on is virtual try-on rather than real physical try-on. Therefore, there is currently a problem that the three-dimensional model of the shoes and the feet of the user block each other inappropriately. For example, for the three-dimensional model of the shoes, the sole inside the shoes will be revealed at the shoe opening, that is, the sole inside the shoes will appear at the user's ankle in the try-on screen. That is, during the try-on process, the sole inside the shoes is displayed in the try-on screen, resulting in a decrease in the realism of the try-on effect. To address this problem, the embodiments of the present disclosure provide a method for processing a three-dimensional model, aiming to adjust the occlusion relationship between the three-dimensional model of the object to be tried on and the part to be tried on, so as to achieve the purpose of improving the try-on effect. The method will be introduced below in combination with specific embodiments.

[0041] Figure 2 The flowchart of the three-dimensional model processing method provided by the embodiments of the present disclosure is shown. This embodiment is applicable to the scenario of AR try-on in a client. This method can be executed by a three-dimensional model processing device, which can be implemented in software and / or hardware, and can be configured in an electronic device, such as a terminal, specifically including but not limited to a smart phone, a personal digital assistant, a tablet computer, a wearable device with a display screen and a camera, a smart home device, etc. Or, this embodiment is applicable to a server, and this method can be executed by a three-dimensional model processing device, which can be implemented in software and / or hardware, and can be configured in an electronic device, such as a server. The three-dimensional model processing method will be introduced below taking a terminal as an example. As Figure 2 shown, the specific steps of this method are as follows:

[0042] S201. Obtain the three-dimensional model corresponding to the object to be tried on.

[0043] Among them, the try-on object can be an item for wearing such as clothes, accessories, or shoes. The corresponding three-dimensional model of the try-on object is usually pre-made and stored. For example, the three-dimensional model corresponding to the try-on object can be stored locally on the terminal or on the server. When the application installed on the terminal needs to obtain the three-dimensional model corresponding to the try-on object, it can obtain the pre-made three-dimensional model according to the specified storage path.

[0044] S202. Determine multiple key points at the target position in the three-dimensional model.

[0045] Among them, the target position of the three-dimensional model is usually the position where there is an inappropriate occlusion relationship with the user's try-on part during the try-on. For example, in the three-dimensional model of shoes, the sole inside the shoe will be revealed at the shoe opening, that is, the sole inside the shoe will appear at the user's ankle in the try-on picture. That is, during the try-on process, the sole inside the shoe is displayed in the try-on picture, resulting in a decrease in the realism of the try-on effect. Therefore, it is necessary to adjust the occlusion relationship between the three-dimensional model of the shoes at the shoe opening and the try-on part (the user's ankle) to prevent the sole inside the shoe from being revealed at the user's ankle to improve the try-on effect. In this application scenario, the shoe opening is the target position in the three-dimensional model.

[0046] S203. Generate an auxiliary model at the target position according to the multiple key points. The auxiliary model is used to adjust the occlusion relationship between the virtual try-on object generated by rendering and the virtual try-on part of the try-on user during the three-dimensional rendering process.

[0047] Taking the three-dimensional model of shoes as an example, a shielding body that fits the shoe opening is generated at the position of the shoe opening. This shielding body is the auxiliary model and is used to establish a correct occlusion relationship with the virtual try-on part during AR shoe try-on to prevent the sole inside the shoe in the three-dimensional model from being revealed at the virtual ankle of the user displayed in the try-on picture.

[0048] Exemplarily, reference can be made to Figure 3 the schematic diagram of an auxiliary model as shown. It includes the three-dimensional model 310 of the shoes and the auxiliary model 320 that fits the shoe opening. The auxiliary model 320 is used to establish a correct occlusion relationship with the virtual try-on part (the relevant positions of the user's foot and ankle) during AR shoe try-on to prevent the sole inside the shoe from being revealed in the try-on picture.

[0049] In one implementation, the way to generate the auxiliary model can be to first determine multiple points located at the shoe opening position of the three-dimensional model. The multiple points located at the shoe opening position are the multiple key points. Then, determine the plane where the shoe opening is located according to the multiple key points, and the points on this plane can form the auxiliary model that fits the shoe opening.

[0050] In another implementation, the auxiliary model can also be generated in the following way: first, determine a reference plane parallel to the plane where the shoe opening of the 3D model is located, then determine multiple key points at the position of the shoe opening of the 3D model based on the reference plane, and then determine the auxiliary model that fits the shoe opening according to the multiple key points.

[0051] In the embodiments of the present disclosure, by determining multiple key points at the target position in the 3D model corresponding to the fitting object, and generating an auxiliary model at the target position according to the multiple key points, the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the 3D rendering process, realizing the adjustment of the occlusion relationship between the 3D model of the fitting object and the virtual fitting part of the fitting user, so as to obtain a more realistic fitting effect.

[0052] Figure 4 The flowchart of the 3D model processing method provided by another embodiment of the present disclosure is shown. In this embodiment, an optional implementation manner is given for the above step S202 "determine multiple key points at the target position in the 3D model", achieving the purpose of accurately determining multiple key points at the target position of the 3D model. As Figure 4 shown, the specific steps of this method are as follows:

[0053] S401. Obtain the 3D model corresponding to the fitting object.

[0054] S402. Determine a reference plane whose position relative to the 3D model is a preset position.

[0055] Taking the 3D model of a shoe as an example, since the positions with inappropriate occlusion relationships with the fitting part during AR fitting include the shoe opening position, an auxiliary model needs to be generated at the shoe opening position, and the corresponding key points for generating the auxiliary model are the points located at the shoe opening position. How to find multiple key points at the shoe opening position in the 3D model is one of the problems to be solved in this embodiment.

[0056] In one implementation, a plane above the shoe opening position in the 3D model can be preferentially determined, and this plane is used as the reference plane. As Figure 5 shown in a schematic diagram of a reference plane, which includes the 3D model 510 of the shoe and the reference plane 520 above the shoe opening 511. The purpose of determining the reference plane 520 is to determine the data corresponding to the key points belonging to the shoe opening position 511 from the data of the 3D model based on the relationship between the points at the shoe opening position 511 and the reference plane.

[0057] S403. According to the reference plane, determine multiple key points at the target position in the 3D model, and the sum of the distances between each key point in the multiple key points and the reference plane is less than or equal to a preset threshold.

[0058] Among them, the meaning that the sum of the distances between each key point among the multiple key points and the reference plane is less than or equal to a preset threshold is as follows: Assume that the number of multiple key points at the target position in the three-dimensional model is 3, that is, there are 3 such key points, namely key point a, key point b, and key point c. The distance between key point a and the reference plane is la, the distance between key point b and the reference plane is lb, and the distance between key point c and the reference plane is lc. Then la + lb + lc ≤ L, where L is the preset threshold.

[0059] Taking the three-dimensional model of shoes as an example, the target position is the shoe mouth position. According to the characteristics of the shoe mouth position, the constraint conditions that the multiple key nodes located at the shoe mouth position need to satisfy can be determined. The constraint conditions are, for example: Each adjacent two key points among the multiple key points are connected, and two non-adjacent key points among the multiple key points are not connected; The first key point and the last key point among the multiple key points are connected; Any key point among the multiple key points is not connected to the nodes in the first set, where the first set is the set obtained by removing the multiple key points from the second set, and the second set is the set composed of the adjacent nodes of each key point among the multiple key points.

[0060] Generally speaking, the multiple key points satisfy at least one of the following constraint conditions: The multiple key points are points in the three-dimensional model; Each adjacent two key points among the multiple key points are connected, and two non-adjacent key points among the multiple key points are not connected; The first key point and the last key point among the multiple key points are connected; Any key point among the multiple key points is not connected to the nodes in the first set, where the first set is the set obtained by removing the multiple key points from the second set, and the second set is the set composed of the adjacent nodes of each key point among the multiple key points; The multiple key points include known points.

[0061] Therefore, a set of key points that satisfy the above constraint conditions can be determined from all the nodes of the three-dimensional model based on the above constraint conditions, and this set of key points is the multiple key points at the target position.

[0062] In order to solve the multiple key points at the target position in the above three-dimensional model, in one implementation, the problem of solving the multiple key points is converted into an optimization equation solving problem. Specifically, the optimization equation includes an objective function and constraint conditions. The obtained solution realizes the objective function on the basis of satisfying the constraint conditions. There may be multiple solutions that satisfy the constraint conditions. By setting the objective function, an optimal solution can be determined from the multiple solutions, thereby determining a set of the above multiple key points.

[0063] A common structure of a 3D model is a mesh. The surface of a 3D model is usually composed of multiple interconnected triangular faces. Each triangular face consists of three nodes and three edges. Therefore, a 3D model can be understood as being composed of nodes and edges, and the set of nodes and edges that make up the 3D model is the mesh. The data structure corresponding to the 3D model is an undirected graph, which is used to store the coordinates of the nodes that make up the 3D model and the connection relationships of the edges between different nodes. The coordinates of the nodes that make up the 3D model are three-dimensional coordinates. For example, the node s i The three-dimensional coordinates of can be expressed as Taking the 3D model of a shoe as an example, the target position is the shoe mouth position, and the multiple key points are the points located at the shoe mouth position. The objective function of the corresponding optimization equation is where s1…s n represents multiple key points located at the shoe mouth position in the 3D model, which are the optimal solutions to be solved, and s i represents s1…s n The i-th node in. Ax + By + Cz + D = 0 represents the reference plane located above the shoe mouth, represents the node s i The distance to the reference plane, represents s1…s n The sum of the distances from each of these n nodes to the reference plane.

[0064] The mathematical meaning of the above objective function is: the value of the objective function calculated based on these n nodes s1…s n is the smallest. The physical meaning of the above objective function is: the optimal solution of the optimization equation is a set of points where the sum of the distances from each node to the reference plane is the minimum.

[0065] The constraint conditions of the corresponding optimization equation are:

[0066]

[0067] where G represents the set of all nodes in the 3D model, or, in other words, G represents the 3D model. The meaning of constraint condition (1) is that any one of the multiple key points s1…s n s i belongs to the points in the 3D model, or, in other words, the multiple key points s1…s n are part of the points in the 3D model. For example, if the 3D model includes 10,000 nodes, the multiple key points s1…s n are part of the 10,000 nodes. When G represents the 3D model, G can be expressed as G[s, e], where s represents the nodes and e represents the edges between the nodes.

[0068] The meaning of the above constraint condition (2) is: the node si Is there an edge connection with node s j If there is an edge connection with node s i Is there an edge connection with node s j If j = i + 1, that is, when s and s are two adjacent nodes, there is an edge connection between node s i Is there an edge connection with node s j For example, there is an edge connection between s1 and s2, and there is an edge connection between s2 and s3, and so on. There is an edge connection between s n-1 and s n That is, there is an edge connection between two adjacent nodes, and there is no edge connection between two non - adjacent nodes. It should be noted that the value range of i in the above constraint condition (2) is from 1 to n - 1, and the corresponding value range of j is from 2 to n. Constraint condition (2) is determined according to the characteristics of the shoe mouth. The edge of the shoe mouth is theoretically a closed loop composed of nodes connected end to end in sequence.

[0069] The meaning of the above constraint condition (3) is: there is an edge connection between the first key point s1 and the last key point s n among the multiple key points, and there is only one edge connection between the first key point s1 and the last key point s n

[0070] Regarding the above constraint condition (4), where S represents the set composed of these n nodes s1…s n That is, S = {s1, s2, …, s n}. s p represents a node selected arbitrarily from these n nodes s1…s n . neighbor(s1) represents the neighbor nodes of node s1, that is, the nodes adjacent to node s1, such as node s2 and node s3. neighbor(s2) represents the neighbor nodes of node s2, and so on. neighbor(s n ) represents the neighbor nodes of node s n . For example, denote [{neighbor(s1)+neighbor(s2)+....+neighbor(s n )}-S] as the first set, that is, the first set is the set remaining after subtracting the S set from the second set composed of the neighbor nodes corresponding to these n nodes s1…s n (that is, {neighbor(s1)+neighbor(s2)+....+neighbor(s n )}). s q represents an arbitrary node in the first set, and N pq = 0 means that there is no edge connection between node s p and node s q . For example​Figure 6 As shown, assume that node s1, node s2, node s3, and node s4 are the n key points located at the shoe mouth position obtained by solving. Node s5, node s6, node s2, and node s4 are all neighbor nodes of node s1. There is an edge connecting node s5 and node s1, an edge connecting node s6 and node s1, an edge connecting node s2 and node s1, and an edge connecting node s4 and node s1, that is, neighbor(s1) = {s5, s6, s2, s4}. Similarly, node s1 and node s3 are both neighbor nodes of node s2, that is, neighbor(s2) = {s1, s3}; node s2 and node s4 are both neighbor nodes of node s3, that is, neighbor(s3) = {s2, s4}; node s1 and node s3 are both neighbor nodes of node s4, that is, neighbor(s4) = {s1, s3}. Then {neighbor(s1) + neighbor(s2) + neighbor(s3) + neighbor(s4)} = {s5, s6, s2, s4, s1, s3}, S = {s1, s2, s3, s4}, and the first set [{neighbor(s1) + neighbor(s2) +.... + neighbor(s n ) - S] = {s5, s6}. That is, the first set includes node s5 and node s6. Since node s5 and node s6 do not belong to set S, it is necessary to disconnect the edge between node s5 and node s1, and disconnect the edge between node s6 and node s1, so as to prevent forming other closed loops with other nodes outside the closed loop formed by node s1, node s2, node s3, and node s4. That is, any key point among the multiple key points is not connected to the nodes in the first set, and the first set is the set obtained by removing the multiple key points from the second set, and the second set is the set formed by the adjacent nodes of each key point among the multiple key points.

[0071] The meaning of the above constraint condition (5) is that set S includes known nodes, that is, the n key points obtained by solving include known nodes. By adding known nodes, the computational complexity during the solution can be reduced, the solution speed can be improved, and the optimal solution of the optimization equation can be obtained relatively quickly. Among them, the way to obtain known nodes can be: divide the three-dimensional model into multiple parts, determine one known node from each part, so that multiple known nodes can be obtained, and then determine one of them from the multiple known nodes. Specifically, calculate the distance from the known node determined from each part to the reference plane, and determine the node with the smallest distance to the reference plane as the final known node s k .

[0072] In summary, the problem of determining multiple key nodes at the target position of the 3D model is transformed into the problem of solving the optimal solution of the optimization equation. The constraint conditions of the optimization equation are the five constraint conditions listed above, and the optimization objective, that is, the objective function, is The sum of the distances from the multiple key points obtained by solution to a preset plane is minimized.

[0073] S404. Generate an auxiliary model at the target position according to the multiple key points. The auxiliary model is used to adjust the occlusion relationship between the virtual try-on object generated by rendering and the virtual try-on part of the try-on user during the 3D rendering process.

[0074] Specifically, the implementation manners and specific principles of S401 and S201 are the same, and the implementation manners and specific principles of S404 and S203 are the same, so they will not be elaborated here.

[0075] In this embodiment, an optional implementation manner is given for step S202 "determine multiple key nodes at the target position in the 3D model" above. Specifically, the problem of determining multiple key points is transformed into the problem of solving the optimal solution of the optimization equation. Specifically, multiple groups of solutions that satisfy all the constraint conditions of the optimization equation are determined from the 3D model, and then the optimal solution is determined from multiple groups of solutions according to the objective function. The multiple nodes corresponding to the optimal solution are the multiple key points located at the shoe mouth position sought, achieving the purpose of accurately determining multiple key points at the target position of the 3D model.

[0076] Figure 7 The flowchart of the 3D model processing method provided by another embodiment of the present disclosure. In this embodiment, an optional implementation manner is given for step S203 "generate an auxiliary model at the target position according to the multiple key points" above. As Figure 7 shown, the specific steps of this method are as follows:

[0077] S701. Obtain the 3D model corresponding to the try-on object.

[0078] S702. Determine multiple key nodes at the target position in the 3D model.

[0079] S703. Obtain a third set and a fourth set according to the multiple key points. The points in the third set are the replicated points of the multiple key points, and the points in the fourth set are the replicated points of the multiple key points.

[0080] S704. Move the fourth set a preset distance relative to the third set, and generate an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

[0081] Among them, the auxiliary model cannot intersect with the three-dimensional model of the fitting object. Therefore, the auxiliary model cannot be directly generated using multiple key points located at the target position of the three-dimensional model. In one implementation, the determined multiple key points are copied twice, and the copied points of the two times are respectively determined as the third set and the fourth set. Then, the fourth set is moved a preset distance relative to the third set, and the third set remains at the target position. An auxiliary model is generated at the target position according to each point in the moved fourth set and each point in the third set.

[0082] Exemplarily, referring to a schematic diagram of generating an auxiliary model as shown in Figure 8 generating an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set includes: generating a first plane 811 according to each point in the moved fourth set (assuming the fourth set includes 3 points, respectively marked as 810a, 810b, and 810c); generating a second plane 821 according to each point in the third set (correspondingly, the fourth set includes 3 points, respectively marked as 820a, 820b, and 820c); connecting each point (810a, 810b, and 810c) in the moved fourth set with its corresponding point in the third set (the point 810a in the fourth set is correspondingly connected to the point 820a in the third set, the point 810b in the fourth set is correspondingly connected to the point 820b in the third set, and the point 810c in the fourth set is correspondingly connected to the point 820c in the third set) to obtain a side surface 830. The auxiliary model is composed of the first plane 811, the second plane 821, and the side surface 830. It should be noted that since each point is a point in the three-dimensional model and thus has three-dimensional coordinates, copying a point does not change the coordinates of the point in the three-dimensional coordinate system. Therefore, each point in the third set and the fourth set coincides with each point in the initial multiple key points, and each point in the moved fourth set only changes in the Z-direction coordinate relative to each point in the initial multiple key points or rather relative to each point in the third set. As shown in Figure 8 for example, for the point 810a in the fourth set relative to the point 820a in the third set, only the z-direction coordinate changes, while the coordinates in the x-direction and y-direction do not change.

[0083] Generally speaking, the generation process of the auxiliary model can be divided into: replication - separation - generation. Specifically, for replication: the auxiliary model cannot intersect with the 3D model, so it is impossible to directly generate the auxiliary model using the multiple key points obtained. Therefore, the multiple key points obtained are replicated. Specifically, the multiple key points obtained are marked as original nodes, the original nodes are replicated for the first time to obtain the first set of replicated nodes, and the original nodes are replicated for the second time to obtain the second set of replicated nodes. After replication, the original nodes, the first set of replicated nodes, and the second set of replicated nodes overlap. For separation: assume that the 3D model is located in a 3D coordinate system. Move the second set of replicated nodes upward by a certain distance along the Z - axis direction of the 3D coordinate system so that the second set of replicated nodes and the first set of replicated nodes are separated from each other, while the first set of replicated nodes remains at the position of the original nodes, that is, the first set of replicated nodes coincides with the original nodes. For generation: generate the side surfaces according to the planes formed by the first set of replicated nodes and the planes formed by the second set of replicated nodes. Specifically, the nodes in the first set of replicated nodes can be correspondingly connected to the nodes in the second set of replicated nodes to form the side surfaces.

[0084] Specifically, the implementation manners and specific principles of step S701 and step S201 are the same, and the implementation manners and specific principles of step S702, step S402, and step S403 are the same, which will not be elaborated here.

[0085] In this embodiment, an alternative implementation manner is given for the above - mentioned step S203 "generate an auxiliary model at the target position according to the multiple key points". Specifically, the determined multiple key nodes are replicated twice, and then the nodes replicated twice are separated, and the auxiliary model is generated from the two sets of replicated points after separation.

[0086] Figure 9 It is a flowchart of a 3D model processing method provided by another embodiment of the present disclosure. In this embodiment, an alternative implementation manner is given for the above - mentioned step S203 "generate an auxiliary model at the target position according to the multiple key points". As Figure 9 shown, the specific steps of this method are as follows:

[0087] S901. Obtain the 3D model corresponding to the fitting object.

[0088] S902. Determine multiple key points at the target position in the 3D model.

[0089] Among them, determining multiple key points at the target position in the 3D model includes:

[0090] Determine a reference plane whose position relative to the 3D model is a preset position. According to the reference plane, determine multiple key points at the target position in the 3D model, and the sum of the distances between each key point in the multiple key points and the reference plane is less than or equal to a preset threshold.

[0091] Among them, the multiple key points satisfy at least one of the following constraint conditions: (1) The multiple key points are points in the three-dimensional model; (2) Every two adjacent key points among the multiple key points are connected, and two non-adjacent key points among the multiple key points are not connected; (3) The first key point and the last key point among the multiple key points are connected; (4) Any key point among the multiple key points is not connected to the nodes in the first set, where the first set is the set obtained by removing the multiple key points from the second set, and the second set is the set composed of the adjacent nodes of each key point among the multiple key points; (5) The multiple key points include known points. S903. Process the multiple key points to obtain the processed multiple key points; according to the processed multiple key points, obtain a third set and a fourth set, where the points in the third set are the replicated points of the processed multiple key points, and the points in the fourth set are the replicated points of the processed multiple key points.

[0092] Among them, the key point is also called a node, and the multiple key points are called a set of nodes.

[0093] Since the optimization equation (objective function + constraint conditions) is a general equation, in the actual application process, it may not be possible to find a set of nodes s1...s that simultaneously satisfy the above 5 constraint conditions n , that is, the multiple key points. Then, a set of nodes s1...s that simultaneously satisfy any 4 of the above 5 constraint conditions can be selected n , and some nodes in this set of nodes s1...s n do not satisfy the remaining 1 constraint condition, but the other nodes in this set of nodes s1...s n except the above-mentioned part of the nodes satisfy the remaining 1 constraint condition. At this time, the nodes that do not satisfy the remaining 1 constraint condition can be processed so that the processed set of nodes simultaneously satisfies the above 5 constraint conditions, and the processed set of nodes is determined as the solution of the optimization equation.

[0094] Exemplarily, processing the multiple key points to obtain the processed multiple key points includes:

[0095] Performing pruning processing on the part of the key points among the multiple key points that do not satisfy the preset constraint conditions to obtain the processed multiple key points; or, if the multiple key points cannot form a closed loop, connecting the key points with a degree less than 2 among the multiple key points to obtain the processed multiple key points.

[0096] For example, the above constraint condition (2) requires multiple key points s1...s n (multiple key points s1...s nare connected between every two adjacent key points in a group of nodes), and among multiple key points s1…s n nodes that are not adjacent among them are not connected. In other words, among multiple key points s1…s n the degree of each node cannot be greater than 2, that is, among multiple key points s1…s n each node corresponds to two branches, or each node has only two neighbors. However, in some cases, multiple key points s1…s n may simultaneously satisfy other constraint conditions except constraint condition (2), that is, multiple key points s1…s n simultaneously satisfy the above constraint conditions (1), (3), (4), and (5), and among multiple key points s1…s n some nodes do not satisfy constraint condition (2). Exemplarily, referring to the schematic diagram of multiple nodes as shown in Figure 10 In addition to being connected to the adjacent nodes s1 and s3, the node s2 is also connected to the nodes s4 and s5 respectively, resulting in four branches for the node s2, and the degree of the node s2 is greater than 2. Therefore, the node s2 does not satisfy the above constraint condition (2). In response to this, the node to be deleted and the edges therebetween can be determined according to the shortest path. For example, the neighbor nodes closer to the node s2 are retained, and the neighbor nodes farther from the node s2 and the edges therebetween are deleted. As shown in Figure 10 it can be to delete the nodes s4, s5, the connection line between the node s2 and the node s4, and the connection line between the node s2 and the node s5, so that the node s2 after the deletion process satisfies the above constraint condition (2). This deletion process is the above-mentioned pruning process.

[0097] After deleting the nodes s4, s5, the connection line between the node s2 and the node s4, and the connection line between the node s2 and the node s5, multiple key points s1…s n are the processed multiple key points. The implementation manners of obtaining the third set and the fourth set according to the processed multiple key points are similar to those of obtaining the third set and the fourth set according to the unprocessed multiple key points in the above embodiments, and will not be specifically described in this embodiment.

[0098] According to the above constraint condition (2) "every two adjacent key points among the multiple key points are connected, and two non-adjacent key points among the multiple key points are not connected" and constraint condition (3) "the first key point and the last key point among the multiple key points are connected", it can be determined that the multiple key points form a closed loop. However, in some cases, the multiple key points obtained by solving do not form a closed loop. At this time, certain processing needs to be performed on the obtained multiple key points to make the processed multiple key points form a closed loop. Specifically, it can be to determine the nodes with a degree less than 2. For example Figure 11Schematic diagram of a set of nodes shown, assuming Figure 11 The four nodes shown are a set of nodes searched, that is, the multiple key points. From Figure 11 It can be seen that the degrees of node s3 and node s1 are both less than 2. At this time, a connection line can be established between node s3 and node s1 according to the distance relationship between node s3 and node s1, so that Figure 11 the four nodes in form a closed loop.

[0099] S904. Move the fourth set relative to the third set by a preset distance; generate an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

[0100] In this embodiment, an optional implementation manner is given for the above step S203 "generate an auxiliary model at the target position according to the multiple key points". Specifically, when a set of determined nodes cannot simultaneously meet the above 5 constraint conditions, the nodes that do not meet the constraint conditions are appropriately processed so that the processed nodes meet the constraint conditions, thereby realizing the determination of multiple key points and providing a data basis for generating an auxiliary model.

[0101] Generally, referring to the flowchart of the processing method of the three-dimensional model shown in Figure 12 Taking the three-dimensional model of shoes as an example of the three-dimensional model, first obtain the three-dimensional model of shoes, and then determine the key points located at the shoe mouth position of the shoes. Specifically, first obtain a set of key points, then perform pruning processing on the obtained key points through a pruning module, and perform processing on the pruned key points through a closed-loop module to obtain the key points located at the shoe mouth position. Then, perform the first copy on the shoe mouth key points, separate the shoe mouth key points obtained by the first copy from the points at the shoe mouth position in the three-dimensional model to generate the bottom surface of the auxiliary model. Then, perform the second copy on the shoe mouth key points and move the points of the second copy upward by a certain distance to generate the upper surface of the auxiliary model. The bottom surface and the upper surface generate the side surface of the auxiliary model, thereby obtaining an auxiliary model that fits the shoe mouth of the three-dimensional model, which can also be called an occlusion body three-dimensional model.

[0102] Figure 13 It is a schematic structural diagram of the three-dimensional model processing device provided by the embodiments of the present disclosure. The three-dimensional model processing device provided by the embodiments of the present disclosure can execute the processing flow provided by the embodiments of the three-dimensional model processing method. As Figure 13 shown, the three-dimensional model processing device 1300 includes: an acquisition module 1310, a determination module 1320, and a generation module 1330.

[0103] Among them, the acquisition module 1310 is configured to acquire a three-dimensional model corresponding to a fitting object; the determination module 1320 is configured to determine a plurality of key points at a target position in the three-dimensional model; the generation module 1330 is configured to generate an auxiliary model at the target position according to the plurality of key points, and the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the three-dimensional rendering process.

[0104] Optionally, the determination module 1320 includes:

[0105] The first determination unit is configured to determine a reference plane whose position relative to the three-dimensional model is a preset position; the second determination unit is configured to determine a plurality of key points at a target position in the three-dimensional model according to the reference plane, and the sum of the distances between each key point in the plurality of key points and the reference plane is less than or equal to a preset threshold.

[0106] Optionally, the plurality of key points satisfy at least one of the following constraint conditions: the plurality of key points are points in the three-dimensional model; every two adjacent key points in the plurality of key points are connected, and two non-adjacent key points in the plurality of key points are not connected; the first key point and the last key point in the plurality of key points are connected; any key point in the plurality of key points is not connected to a node in a first set, and the first set is a set obtained by removing the plurality of key points from a second set, and the second set is a set composed of adjacent nodes of each key point in the plurality of key points; the plurality of key points include known points.

[0107] Optionally, the generation module 1330 includes: a first copying unit configured to obtain a third set and a fourth set according to the plurality of key points, the points in the third set being copied points of the plurality of key points, and the points in the fourth set being copied points of the plurality of key points; a first moving unit configured to move the fourth set a preset distance relative to the third set; a first generating unit configured to generate an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

[0108] Optionally, the generation module 1330 includes: a processing unit configured to process the multiple key points to obtain processed multiple key points; a second copying unit configured to: obtain a third set and a fourth set according to the processed multiple key points, wherein the points in the third set are the copied points of the processed multiple key points, and the points in the fourth set are the copied points of the processed multiple key points; a second moving unit configured to move the fourth set a preset distance relative to the third set; and a second generation unit configured to generate an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

[0109] Optionally, the processing unit includes: a pruning processing subunit configured to perform pruning processing on some of the multiple key points that do not meet the preset constraint conditions to obtain processed multiple key points; or a connection processing subunit configured to, if the multiple key points cannot form a closed loop, perform connection processing on the key points with a degree less than 2 among the multiple key points to obtain processed multiple key points.

[0110] Optionally, the first generation unit and the second generation unit include: a first generation subunit configured to generate a first plane according to each point in the moved fourth set; a second generation subunit configured to generate a second plane according to each point in the third set; and a connection subunit configured to connect each point in the moved fourth set with its corresponding point in the third set to obtain a side surface, and the auxiliary model is composed of the first plane, the second plane, and the side surface.

[0111] The three-dimensional model processing device according to the embodiments of the present disclosure determines multiple key points at a target position in a three-dimensional model corresponding to a fitting object, and generates an auxiliary model at the target position according to the multiple key points. The auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the three-dimensional rendering process, thereby realizing the adjustment of the occlusion relationship between the three-dimensional model of the fitting object and the virtual fitting part of the fitting user, and thus a more realistic virtual fitting effect can be obtained.

[0112] Figure 13 The three-dimensional model processing device in the illustrated embodiment can be used to execute the technical solutions of the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here.

[0113] The internal functions and structures of the three-dimensional model processing device are described above, and the device can be implemented as an electronic device. Figure 14 It is a schematic structural diagram of an electronic device embodiment provided by the embodiments of the present disclosure. As Figure 14 shown, the electronic device includes a memory 151 and a processor 152.

[0114] A memory 151 for storing programs. In addition to the above programs, the memory 151 can also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application program or method for operating on the electronic device, contact data, phone book data, messages, pictures, videos, and the like.

[0115] The memory 151 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0116] A processor 152, coupled to the memory 151, executes the programs stored in the memory 151 for:

[0117] Obtaining a three-dimensional model corresponding to a fitting object; determining a plurality of key points at a target position in the three-dimensional model; and generating an auxiliary model at the target position according to the plurality of key points, where the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the three-dimensional rendering process.

[0118] Further, as Figure 14 shown, the electronic device may further include other components such as a communication component 153, a power supply component 154, an audio component 155, a display 156, etc. Figure 14 Only some components are schematically shown in Figure 14 and it does not mean that the electronic device only includes

[0119] The communication component 153 is configured to facilitate communication between the electronic device and other devices in a wired or wireless manner. The electronic device can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 153 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 153 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0120] The power supply component 154 provides power for various components of the electronic device. The power supply component 154 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device.

[0121] The audio component 155 is configured to output and / or input audio signals. For example, the audio component 155 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 151 or transmitted via the communication component 153. In some embodiments, the audio component 155 further includes a speaker for outputting audio signals.

[0122] The display 156 includes a screen, and the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations.

[0123] In addition, embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the three-dimensional model processing method described in the above embodiments.

[0124] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0125] The above are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional model processing method, wherein, The method includes: Obtaining a three-dimensional model corresponding to the fitting object; Determining a plurality of key points at a target position in the three-dimensional model; Generating an auxiliary model at the target position according to the plurality of key points, where the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during the three-dimensional rendering process; Determining a plurality of key points at a target position in the three-dimensional model includes: Determining a reference plane whose position relative to the three-dimensional model is a preset position; According to the reference plane, determining a plurality of key points at the target position in the three-dimensional model, and the sum of the distances between each key point in the plurality of key points and the reference plane is less than or equal to a preset threshold; The plurality of key points satisfy at least one of the following constraint conditions: The plurality of key points are points in the three-dimensional model; Each adjacent two key points in the plurality of key points are connected, and non-adjacent two key points in the plurality of key points are not connected; The first key point and the last key point in the plurality of key points are connected; Any key point in the plurality of key points is not connected to the nodes in the first set, where the first set is a set obtained by removing the plurality of key points from the second set, and the second set is a set composed of the adjacent nodes of each key point in the plurality of key points; The plurality of key points include known points; Generating an auxiliary model at the target position according to the plurality of key points includes: According to the plurality of key points, obtaining a third set and a fourth set, where the points in the third set are duplicate points of the plurality of key points, and the points in the fourth set are duplicate points of the plurality of key points; Moving the fourth set a preset distance relative to the third set; Generating an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

2. The method according to claim 1, wherein Generating an auxiliary model at the target position according to the plurality of key points includes: Processing the plurality of key points to obtain processed key points; According to the processed key points, obtaining a third set and a fourth set, where the points in the third set are duplicate points of the processed key points, and the points in the fourth set are duplicate points of the processed key points; Moving the fourth set a preset distance relative to the third set; Generating an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

3. The method according to claim 2, wherein Processing the plurality of key points to obtain processed key points includes: Performing pruning processing on some key points in the plurality of key points that do not meet the preset constraint conditions to obtain processed key points; or If the plurality of key points cannot form a closed loop, connecting the key points with a degree less than 2 in the plurality of key points to obtain processed key points.

4. The method according to claim 1 or 2, wherein Generating an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set includes: Generating a first plane according to each point in the moved fourth set; Generate a second plane according to each point in the third set; Connect each point in the moved fourth set with its corresponding point in the third set to obtain a side surface, and the auxiliary model is composed of the first plane, the second plane, and the side surface.

5. A three-dimensional model processing device, wherein, Comprising: An acquisition module for acquiring a three-dimensional model corresponding to a fitting object; A determination module for determining a plurality of key points at a target position in the three-dimensional model; A generation module for generating an auxiliary model at the target position according to the plurality of key points, and the auxiliary model is used to adjust the occlusion relationship between the virtual fitting object generated by rendering and the virtual fitting part of the fitting user during three-dimensional rendering; The determination module includes: a first determination unit for determining a reference plane whose position relative to the three-dimensional model is a preset position; a second determination unit for determining a plurality of key points at a target position in the three-dimensional model according to the reference plane, and the sum of the distances between each key point in the plurality of key points and the reference plane is less than or equal to a preset threshold; The plurality of key points satisfy at least one of the following constraints: the plurality of key points are points in the three-dimensional model; every two adjacent key points in the plurality of key points are connected, and every two non-adjacent key points in the plurality of key points are not connected; the first key point and the last key point in the plurality of key points are connected; any key point in the plurality of key points is not connected to a node in the first set, and the first set is a set obtained by removing the plurality of key points from the second set, and the second set is a set composed of adjacent nodes of each key point in the plurality of key points; the plurality of key points include known points; The generation module includes: a first copying unit for obtaining a third set and a fourth set according to the plurality of key points, the points in the third set being copied points of the plurality of key points, and the points in the fourth set being copied points of the plurality of key points; a first moving unit for moving the fourth set a preset distance relative to the third set; a first generating unit for generating an auxiliary model at the target position according to each point in the moved fourth set and each point in the third set.

6. An electronic device, wherein, Comprising: A memory; A processor; And A computer program; Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, wherein, The computer program, when executed by the processor, implements the method according to any one of claims 1-4.

Images