A method and related apparatus for mesh reconstruction based on a three-dimensional object

Abstract

The application discloses a three-dimensional object-based mesh reconstruction method and related device. The method comprises: on the basis that each vertex in the first reconstruction mesh of the three-dimensional object has an attention weight representing importance, for each line segment in the first reconstruction mesh of the three-dimensional object, obtaining two attention weights corresponding to two vertices connected by the line segment, so as to determine the importance of the line segment through the importance of the two vertices; updating the first length of the line segment to obtain a second length through the two attention weights representing the importance of the line segment, so that the ratio between the second length and the first length is positively correlated with the size of the two attention weights; judging whether the second length is greater than the maximum length determined by the best length of the three-dimensional object in the mesh reconstruction round, if yes, segmenting the line segment to obtain a line segment segmentation mesh, and obtaining a more fine line segment segmentation mesh of the important line segment; and performing mesh reconstruction based on the line segment segmentation mesh to obtain a second reconstruction mesh of the three-dimensional object which is more fine in the important region.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a mesh reconstruction method and related apparatus based on three-dimensional objects. Background Technology

[0002] With the continuous development of computer graphics and computer vision technologies, mesh reconstruction of 3D objects has become increasingly important. Mesh reconstruction of 3D objects refers to the process of recovering the 3D model of a 3D object from a 2D image of that object.

[0003] In related technologies, the mesh reconstruction method for three-dimensional objects refers to: reconstructing the mesh of a three-dimensional object using multiple images taken from multiple perspectives, updating the vertices in the mesh of the three-dimensional object, and segmenting, collapsing, and flipping the line segments in the mesh of the three-dimensional object to obtain the reconstructed mesh of the three-dimensional object.

[0004] However, the mesh reconstruction process of the 3D object in the above method is isotropic, which cannot highlight the important areas of the 3D object. As a result, the reconstructed mesh of the important areas of the 3D object is not fine enough, thus resulting in poor mesh reconstruction effect of the 3D object. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a mesh reconstruction method and related apparatus based on 3D objects. By using two attention weights corresponding to two vertices connected by line segments in the first reconstructed mesh of the 3D object, the importance of the line segments in the first reconstructed mesh is determined. The first length of the line segments is updated to obtain a second length, ensuring that more important line segments in the first reconstructed mesh are longer. This makes it easier for excessively long line segments to be divided into more segments, resulting in a finer segmentation mesh for important line segments. This highlights the important regions to which important line segments of the 3D object belong. Based on the segmentation mesh, the mesh reconstruction of the 3D object is achieved, making the reconstructed mesh of important regions of the 3D object more refined, thereby improving the mesh reconstruction effect of the 3D object.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] On one hand, embodiments of this application provide a mesh reconstruction method based on three-dimensional objects, the method comprising:

[0008] For each line segment in the first reconstructed mesh of a 3D object, two attention weights are obtained corresponding to the two vertices connected by the line segment; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh.

[0009] The first length of the line segment is updated based on the two attention weights to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights.

[0010] If the second length is greater than the maximum length, the line segment is segmented to obtain the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction cycle;

[0011] The line segment mesh is reconstructed to obtain the second reconstructed mesh of the three-dimensional object.

[0012] On the other hand, embodiments of this application provide a mesh reconstruction device based on three-dimensional objects, the device comprising: an acquisition unit, an update unit, a segmentation unit, and a reconstruction unit;

[0013] The acquisition unit is used to acquire two attention weights corresponding to the two vertices connected by each line segment in the first reconstructed mesh of the 3D object; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh.

[0014] The update unit is used to update the first length of the line segment according to the two attention weights to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights.

[0015] The segmentation unit is used to segment the line segment if the second length is greater than the maximum length, to obtain the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction cycle;

[0016] The reconstruction unit is used to reconstruct the line segment mesh to obtain a second reconstructed mesh of the three-dimensional object.

[0017] On the other hand, embodiments of this application provide a computer device, the computer device including a processor and a memory:

[0018] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0019] The processor is configured to execute the method described in any of the foregoing aspects according to instructions in the computer program.

[0020] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program that, when run on a computer device, causes the computer device to perform the method described in any of the foregoing aspects.

[0021] On the other hand, embodiments of this application provide a computer program product, including a computer program that, when run on a computer device, causes the computer device to perform the method described in any of the foregoing aspects.

[0022] As can be seen from the above technical solution, each vertex in the first reconstructed mesh of the 3D object has an attention weight, representing the importance of each vertex in the first reconstructed mesh. For each line segment in the first reconstructed mesh of the 3D object, firstly, the two attention weights corresponding to the two vertices connected by the line segment are obtained, so that the importance of the line segment in the first reconstructed mesh can be determined by the importance of the two vertices connected by the line segment in the first reconstructed mesh. The first length of the line segment is updated to obtain the second length by using the two attention weights representing the importance of the line segment in the first reconstructed mesh, so that the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. That is, the larger the two attention weights are, the larger the ratio between the second length and the first length is, which makes the length of the more important line segments in the first reconstructed mesh longer. Then, it is determined whether the second length of the line segment is greater than the maximum length determined by the optimal length of the 3D object in the mesh reconstruction round. If so, the line segment is divided to obtain a line segmentation mesh, indicating that the more important line segments in the first reconstructed mesh are more likely to be too long and can be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. Based on the line segmentation mesh, mesh reconstruction is performed to obtain a more refined second reconstructed mesh for the important regions of the 3D object.

[0023] Based on this, the method determines the importance of line segments in the first reconstructed mesh by using two attention weights corresponding to the two vertices connected by line segments in the first reconstructed mesh of the 3D object. The first length of the line segment is updated to obtain the second length, so that the more important line segments in the first reconstructed mesh are longer. The longer the length, the more likely it is to be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. This highlights the important regions to which the important line segments of the 3D object belong. The mesh reconstruction of the 3D object is achieved based on the line segmentation mesh, making the reconstructed mesh of the important regions of the 3D object more refined, thereby improving the mesh reconstruction effect of the 3D object. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This application provides a schematic diagram of a mesh reconstruction method based on three-dimensional objects.

[0026] Figure 2 A flowchart illustrating a mesh reconstruction method based on three-dimensional objects provided in this application embodiment;

[0027] Figure 3 A schematic diagram showing attention weights for each vertex in a first reconstructed mesh of a 3D object, provided as an embodiment of this application;

[0028] Figure 4 A schematic diagram illustrating line segmentation based on a first reconstructed mesh of a three-dimensional object, provided as an embodiment of this application;

[0029] Figure 5 A schematic diagram illustrating attention flow under two-dimensional vision, provided as an embodiment of this application;

[0030] Figure 6 A schematic diagram of a normal vector and a reference normal vector of a surface under two-dimensional vision, provided for an embodiment of this application;

[0031] Figure 7 A schematic diagram illustrating the sum of squared degree errors before and after line segment flipping under two-dimensional vision, provided for an embodiment of this application;

[0032] Figure 8 A schematic diagram illustrating a mesh reconstruction step based on a three-dimensional object, provided for an embodiment of this application;

[0033] Figure 9 A schematic diagram of a reconstruction mesh of a three-dimensional object in a related art provided for embodiments of this application, and a second reconstruction mesh of a three-dimensional object in embodiments of this application;

[0034] Figure 10 A structural diagram of a mesh reconstruction device based on a three-dimensional object provided in this application embodiment;

[0035] Figure 11 A structural diagram of a server provided in an embodiment of this application;

[0036] Figure 12 This is a structural diagram of a terminal provided in an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application will now be described with reference to the accompanying drawings.

[0038] Currently, 3D object mesh reconstruction typically involves reconstructing the 3D object's mesh from multiple images taken from various perspectives, updating the vertices in the mesh, and segmenting, collapsing, and flipping the line segments within the mesh to obtain the reconstructed mesh. However, research has revealed that the 3D object mesh reconstruction process is isotropic, with both important and minor regions exhibiting the same reconstruction precision. This leads to insufficient allocation of mesh resources, failing to highlight the critical areas of the 3D object. Consequently, the reconstructed meshes for these important regions lack detail, resulting in poor overall 3D object mesh reconstruction quality.

[0039] This application provides a mesh reconstruction method based on 3D objects. By using two attention weights corresponding to two vertices connected by line segments in the first reconstructed mesh of the 3D object, the importance of the line segments in the first reconstructed mesh is determined. The first length of the line segments is updated to obtain the second length, so that the more important line segments in the first reconstructed mesh are longer. The longer the length, the more likely it is to be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. This highlights the important regions to which the important line segments of the 3D object belong. The mesh reconstruction of the 3D object is achieved based on the line segmentation mesh, making the reconstructed mesh of the important regions of the 3D object more refined, thereby improving the mesh reconstruction effect of the 3D object.

[0040] Next, the system architecture of the mesh reconstruction method based on 3D objects will be introduced. See [link / reference] Figure 1 ,Should Figure 1 This is a schematic diagram of a mesh reconstruction method based on three-dimensional objects provided in an embodiment of this application. The system includes a computer device 100, which is used to execute the mesh reconstruction method based on three-dimensional objects.

[0041] For each line segment in the first reconstructed mesh of a 3D object, the computer device 100 obtains two attention weights corresponding to the two vertices connected by the line segment; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh.

[0042] As an example, the 3D object is a game character, the first reconstructed mesh is reconstructed mesh 1, the line segment is l, and the two vertices connected by l are x. i x j Then, for each line segment l in the reconstructed mesh 1 of the game character, computer device 100 obtains the x connected to l. i x j The two corresponding attention weights are wi w j .

[0043] The computer device 100 updates the first length of the line segment based on the two attention weights corresponding to the two vertices connected by the line segment, and obtains the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights.

[0044] As an example, based on the above example, if the first length of l is l1, then the computer device 100 is connected to x via l. i x j The corresponding w i w j Update l to l1 to get the second length l2, such that the ratio between l2 and l1 is equal to w. i w j The magnitudes are positively correlated.

[0045] If the second length is greater than the maximum length, the computer device 100 performs line segmentation on the line segments to obtain the line segmentation mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction round.

[0046] As an example, based on the above example, the optimal length of the game character in the grid reconstruction round is l. ref The maximum length is l max Then computer device 100 determines whether l2 is greater than l in the grid reconstruction round determined by the game character. ref Determined l max If so, the segmented mesh is obtained by dividing and reconstructing mesh 1 into l.

[0047] Computer device 100 performs mesh reconstruction on the line segmented mesh to obtain a second reconstructed mesh of the three-dimensional object.

[0048] As an example, based on the above example, computer device 100 performs mesh reconstruction based on line segmentation mesh, and the second reconstructed mesh of the game character is reconstructed mesh 2.

[0049] In other words, each vertex in the first reconstructed mesh of the 3D object has an attention weight, representing the importance of each vertex in the first reconstructed mesh. For each line segment in the first reconstructed mesh of the 3D object, two attention weights corresponding to the two vertices connected by the line segment are obtained. This allows the importance of the line segment in the first reconstructed mesh to be determined by the importance of the two vertices connected by the line segment. The first length of the line segment is updated using the two attention weights representing the importance of the line segment in the first reconstructed mesh to obtain the second length. The ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. That is, the larger the two attention weights, the larger the ratio between the second length and the first length, which allows the more important line segments in the first reconstructed mesh to be longer. It is then determined whether the second length of the line segment is greater than the maximum length determined by the optimal length of the 3D object in the mesh reconstruction round. If so, the line segment is divided to obtain a line segmentation mesh. This indicates that the more important line segments in the first reconstructed mesh are more likely to be too long and can be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. Mesh reconstruction is then performed based on the line segmentation mesh to obtain a more refined second reconstructed mesh for the important regions of the 3D object.

[0050] It should be noted that, in the embodiments of this application, the computer device can be a server or a terminal. The method provided in the embodiments of this application can be executed by the terminal or the server alone, or it can be executed by the terminal and the server in cooperation. Specifically, when the method provided in the embodiments of this application is executed by the terminal or the server alone, its execution method is similar to... Figure 1 The corresponding embodiments are similar, mainly replacing the computer device with a terminal or server. Furthermore, when the method provided in this application is executed by a terminal and a server, steps that need to be displayed on the front-end interface can be executed by the terminal, while steps that require background calculations and do not need to be displayed on the front-end interface can be executed by the server.

[0051] The terminal can be a smartphone, tablet, laptop, desktop computer, intelligent voice interaction device, in-vehicle terminal, extended reality device, or gaming device, but is not limited to these. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, but is not limited to these. The terminal and server can be connected directly or indirectly through wired or wireless communication, and this application does not impose any restrictions. For example, the terminal and server can be connected through a network, which can be wired or wireless.

[0052] Next, taking the method provided in the embodiments of this application as an example, and in conjunction with the accompanying drawings, we will give a detailed description of the mesh reconstruction method based on three-dimensional objects provided in the embodiments of this application.

[0053] See Figure 2 ,Should Figure 2 A flowchart of a mesh reconstruction method based on three-dimensional objects provided in this application embodiment, the method including:

[0054] S201: For each line segment in the first reconstructed mesh of the 3D object, obtain two attention weights corresponding to the two vertices connected by the line segment; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh.

[0055] In related technologies, the mesh reconstruction of 3D objects typically involves reconstructing the 3D object's mesh from multiple images taken from various perspectives, updating the vertices in the mesh, and segmenting, collapsing, and flipping the line segments within the mesh to obtain the reconstructed mesh. However, the mesh reconstruction process for 3D objects is isotropic, and the allocation of reconstruction mesh resources between important and secondary regions of the 3D object is often insufficient, failing to highlight the important regions. This results in imprecise reconstruction meshes for the important regions, leading to poor mesh reconstruction results for the 3D object.

[0056] In this embodiment, to address the aforementioned issues, after performing a preliminary mesh reconstruction on the 3D object to obtain a first reconstructed mesh, attention annotation is applied to each vertex in the first reconstructed mesh, assigning an attention weight to each vertex to represent its importance. Based on this, to achieve a more refined mesh reconstruction of important regions of the 3D object, it is necessary to refine the important line segments, which requires determining the importance of each line segment in the first reconstructed mesh. Therefore, for each line segment in the first reconstructed mesh, considering the two attention weights corresponding to the two vertices connected by the line segment, which represent the importance of the line segment in the first reconstructed mesh, it is first necessary to obtain the two attention weights corresponding to the two vertices connected by the line segment.

[0057] The first reconstructed mesh of the 3D object is the reconstructed mesh of the object contour of the 3D object. The number of vertices in the first reconstructed mesh of the 3D object is less than the number of vertices in the actual reconstructed mesh of the 3D object. The two attention weights corresponding to the two vertices connected by the line segment are determined based on the attention labels corresponding to the two vertices connected by the line segment.

[0058] Based on the fact that each vertex in the first reconstructed mesh of the 3D object has an attention weight, which represents the importance of each vertex in the first reconstructed mesh, S201 determines the importance of two vertices connected by a line segment in the first reconstructed mesh by obtaining the two attention weights corresponding to the two vertices connected by the line segment in the first reconstructed mesh, thus laying the foundation for determining the importance of the line segment in the first reconstructed mesh in the subsequent process.

[0059] As an example of S201, the 3D object is a game character, the first reconstructed mesh is reconstructed mesh 1, the line segment is l, and the two vertices connected by l are x. i x j For each line segment l in the reconstructed mesh 1 of the game character, the computer device obtains the x connected to l. i x j The two corresponding attention weights are w i w j .

[0060] See Figure 3 ,Should Figure 3 This is a schematic diagram illustrating attention weights at each vertex in the first reconstructed mesh of a 3D object, provided as an embodiment of this application; based on the example in S201, Figure 3 Represents the reconstructed mesh 1 of the game character. In this reconstructed mesh 1, the vertices of darker colored regions have greater attention weights. That is, the vertices of important regions such as the head region and hand region in this reconstructed mesh 1 have greater attention weights.

[0061] S202: Update the first length of the line segment based on the two attention weights corresponding to the two vertices connected by the line segment to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights.

[0062] In this embodiment, after executing S201 to obtain the two attention weights corresponding to the two vertices connected by the line segment, since the two attention weights corresponding to the two vertices connected by the line segment can represent the importance of the line segment in the first reconstructed mesh, in order to make the important line segments of the 3D object more refined, it is necessary to divide the length of the important line segments into more segments if the length of the important line segments is too long. Therefore, the length of each line segment in the first reconstructed mesh of the 3D object needs to be modified to make the length of the important line segments longer. Therefore, by using the two attention weights corresponding to the two vertices connected by the line segment, the first length of the line segment is updated to obtain the second length, so that the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights, that is, the larger the two attention weights are, the larger the ratio between the second length and the first length is.

[0063] Specifically, updating the first length of a line segment involves recalculating the length of the line segment based on its first length to modify the first length of the line segment.

[0064] S202 updates the first length of a line segment to obtain the second length by representing the importance of the line segment in the first reconstructed grid with two attention weights. The larger the two attention weights are, the larger the ratio between the second length and the first length is, which can make the length of the more important line segments in the first reconstructed grid longer, providing a basis for subsequent segmentation of important line segments that are too long into more line segments.

[0065] As an example of the above S202, based on the example of S202, the first length l1 of l, then the computer device is connected to x through l. i x j The corresponding w i w j Update l to l1 to get the second length l2, such that the ratio between l2 and l1 is equal to w. i w j The magnitudes are positively correlated.

[0066] S203: If the second length is greater than the maximum length, the line segment is segmented to obtain the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction round.

[0067] In this embodiment of the application, after executing the above-mentioned S202, which updates the first length of the line segment to obtain the second length by connecting the two attention weights of the two vertices connected by the line segment, in order to make the important line segments of the 3D object more refined, it is necessary to divide the important line segments that are too long into more line segments. Therefore, it is necessary to divide the important line segments that are too long in the first reconstructed mesh of the 3D object into more line segments. Therefore, it is determined whether the second length of the line segment is greater than the maximum length determined by the best length of the 3D object in the mesh reconstruction round. If so, it means that the line segment is an important line segment that is too long in the first reconstructed mesh of the 3D object, and the line segment is divided to obtain the line segmentation mesh of the first reconstructed mesh.

[0068] Among them, segmenting a line segment is based on inserting a vertex at the midpoint of the line segment to divide the line segment; the optimal length of a 3D object in a mesh reconstruction cycle is the optimized and best line segment length of the 3D object in the mesh reconstruction cycle.

[0069] S203 determines that the second length of a line segment is greater than the maximum length, and then divides the line segment to obtain the line segment segmentation mesh of the first reconstructed mesh. This means that the more important the line segment in the first reconstructed mesh, the more likely it is to be too long and can be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments, so that the important areas of the 3D object can be refined in the future.

[0070] As an example of S203 above, based on the example of S202 above, the optimal length of the game character in the grid reconstruction round is l. refThe maximum length is l max The computer then determines whether l2 is greater than l determined by the game character in the grid reconstruction round. ref Determined l max If so, the segmented mesh is obtained by dividing and reconstructing mesh 1 into l.

[0071] See Figure 4 ,Should Figure 4 This application provides a schematic diagram of line segmentation based on a first reconstructed mesh of a three-dimensional object, as provided in an embodiment of the present application; based on the example in S203, Figure 4 This represents a portion of the reconstructed mesh 1 for the game character. Based on the midpoint of l, a vertex is inserted to divide l, and the original partial reconstructed mesh is updated to obtain a partial reconstructed mesh with segmented meshes. The black vertices are the inserted vertices, and the dashed lines are the new line segments obtained by updating the original partial reconstructed mesh.

[0072] S204: Reconstruct the mesh from the segmented line mesh to obtain the second reconstructed mesh of the 3D object.

[0073] In this embodiment of the application, after performing the above-mentioned S203 to determine that the second length of the line segment is greater than the maximum length and dividing the line segment to obtain the first reconstructed mesh, since the important line segments with excessive length in the line segmented mesh are divided into more line segments, the important line segments of the three-dimensional object are more refined. Therefore, based on the line segmented mesh, mesh reconstruction can be performed to obtain a second reconstructed mesh with more refined important regions of the three-dimensional object.

[0074] Among them, the mesh reconstruction of the segmented mesh is the collapse and flipping of the segmented mesh of the line segments.

[0075] The S204 achieves mesh reconstruction of 3D objects by dividing the mesh into finer segments for important line segments, making the important areas of the 3D objects more detailed and improving the mesh reconstruction effect of the 3D objects.

[0076] As an example of S204 above, based on the example of S203 above, the computer device performs mesh reconstruction based on the line segmentation mesh, and the second reconstructed mesh of the game character is reconstructed mesh 2.

[0077] As can be seen from the above technical solution, by using the two attention weights corresponding to the two vertices connected by the line segments in the first reconstructed mesh of the 3D object, the importance of the line segments in the first reconstructed mesh is determined, and the first length of the line segments is updated to obtain the second length. This makes the more important line segments in the first reconstructed mesh longer, and the longer they are, the more likely they are to be divided into more line segments. This results in a more refined line segmentation mesh for important line segments, highlighting the important regions to which the important line segments of the 3D object belong. Based on the line segmentation mesh, the mesh reconstruction of the 3D object is realized, making the reconstructed mesh of the important regions of the 3D object more refined, thereby improving the mesh reconstruction effect of the 3D object.

[0078] In this embodiment, in the above-mentioned S202, the first length of the line segment is updated to obtain the second length by using the two attention weights corresponding to the two vertices connected by the line segment, such that the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. Specifically, considering the two attention weights corresponding to the two vertices connected by the line segment, the line segment weight can be determined to update the first length of the line segment to obtain the second length. Therefore, the average value of the two attention weights corresponding to the two vertices connected by the line segment can be calculated first as the line segment weight; then, the first length of the line segment is updated to obtain the second length using the line segment weight, such that the ratio between the second length and the first length is positively correlated with the magnitude of the line segment weight, that is, the larger the line segment weight, the larger the ratio between the second length and the first length. Based on this, this application provides a possible implementation, and the above-mentioned S202 may include, for example, the following S2021-S2022 (not shown in the figure).

[0079] S2021: The line segment weight is obtained by averaging the two attention weights corresponding to the two vertices connected by the line segment.

[0080] S2022: Update the first length according to the segment weight to obtain the second length.

[0081] The line segment weight represents the importance of the line segment in the first reconstructed grid.

[0082] S2021-S2022 uses the average of the two attention weights corresponding to the two vertices connected by the line segment as the line segment weight. Based on the two attention weights corresponding to the two vertices connected by the line segment, it intuitively and evenly represents the importance of the line segment in the first reconstructed mesh as a whole. It can accurately update the first length of the line segment to obtain the second length. When the line segment weight is larger, the ratio between the second length and the first length is larger. It can accurately make the length of the more important line segments in the first reconstructed mesh longer, and provide an accurate segmentation basis for subsequent segmentation of important line segments that are too long into more line segments.

[0083] As an example of S2021-S2022, based on the example of S202 above, the computer device first calculates the x connected to l. i x j The corresponding w i w j The average value of l is used as the line segment weight of l. l1 is updated by weight to obtain l2, so that the ratio between l2 and l1 is positively correlated with the size of weight.

[0084] Furthermore, in this embodiment, after performing S2021 to calculate the average of the two attention weights corresponding to the two vertices connected by the line segment as the line segment weight, and before performing S2022 to update the first length of the line segment using the line segment weight to obtain the second length, considering the different actual situations of different line segments in the first reconstructed mesh of the 3D object, an offset for the line segment weight can be introduced to adjust the line segment weight to suit the determination of the importance of different line segments in the first reconstructed mesh of the 3D object. Therefore, based on the line segment weight, the first weight can also be obtained by offset calculation using the first offset. Correspondingly, in the specific implementation of S2022, the first weight can update the length of the line segment more accurately than the line segment weight, so the second length is obtained by updating the first length of the line segment using the first weight. Based on this, this application provides a possible implementation method. After executing the above S2021 and before executing the above S2022, the method may further include S1 (not shown in the figure): calculating the offset of the line segment weight according to the first offset to obtain the first weight; correspondingly, the above S2022 may include the following S2022a (not shown in the figure): updating the length of the first length according to the first weight to obtain the second length.

[0085] The first offset is a pre-configured offset; the offset calculation of the segment weight based on the first offset is based on modifying the segment weight based on the first offset.

[0086] Based on the average of the two attention weights corresponding to the two vertices connected by the line segment, S1 and S2022a introduce a first offset applicable to different line segments in the first reconstructed mesh of the 3D object, and more accurately calculate the weights used to update the length of the line segment, so as to update the length of the line segment more accurately.

[0087] As an example of S1 and S2022a, based on the examples of S2021-S2022 above, the first offset is β1; then, based on the weight, the computer device can also calculate the first weight, e, through offset calculation using β1. weight -β1, through e weight-β1 updates l to l1, resulting in l2, i.e., l2 = l1 × (e weight -β1); where β1 can be 0.4.

[0088] In this embodiment, the maximum length in S203 is actually an acceptable maximum length determined by the optimal length of the 3D object in the mesh reconstruction cycle. Therefore, the maximum length is greater than the optimal length of the 3D object in the mesh reconstruction cycle. Thus, the maximum length can be calculated based on the optimal length using a tolerance value greater than 1. Based on this, this application provides a possible implementation method. The step of determining the maximum length in S203 may include, for example, S2 (not shown in the figure): calculating the length of the optimal length based on the tolerance value to obtain the maximum length; the maximum length is greater than the optimal length.

[0089] The tolerance value is a pre-configured tolerance parameter value.

[0090] This method determines the maximum acceptable length greater than the optimal length by using a tolerance value and the optimal length of the 3D object in a mesh reconstruction cycle. This represents the upper limit of the acceptable line segment length for the 3D object in a mesh reconstruction cycle, providing an accurate basis for subsequent judgments on whether a line segment is too long and needs to be divided into more segments.

[0091] As an example, based on the example of S203 above, the tolerance value is tol, tol > 0; then the computer device in l ref Based on this, l is calculated using 1+tol max That is, l max =l ref ×(1+tol), l max >l ref .

[0092] Furthermore, in this embodiment, after obtaining the segmented mesh of the first reconstructed mesh by performing the above S203, it is further considered that the number of vertices and the distribution of attention weights of important and minor regions of the three-dimensional object are uneven in the segmented mesh of the three-dimensional object. In order to avoid neglecting the mesh reconstruction of minor regions in the future, a voxel-level attention flow mechanism can be introduced. It is necessary to make attention flow from more important voxels to the least important voxels around them, so that attention is gradually and evenly distributed in the segmented mesh of the three-dimensional object.

[0093] Therefore, for each first voxel in the reconstruction space of a 3D object, the flow heuristic value of the first voxel is first determined by the number of vertices within the first voxel and the attention weights corresponding to those vertices. This flow heuristic value is negatively correlated with the number of vertices and the attention weights corresponding to those vertices within the first voxel; that is, the more vertices and attention weights within the first voxel, the more important the first voxel, and the smaller the flow heuristic value. Next, it is determined whether the flow heuristic value is less than the heuristic threshold. If so, it indicates that the first voxel has a large number of vertices and a large attention weight, making it more important. Attention needs to be shifted from the first voxel to the voxel with the fewest vertices and the lowest attention weight among its surrounding voxels. In other words, attention should be shifted from the first voxel to the second voxel corresponding to the largest flow heuristic value among its surrounding voxels to update the attention weights corresponding to the vertices in the segmented mesh. Based on this, this application provides a possible implementation method. After performing the above S203 to obtain the line segmentation mesh of the first reconstructed mesh, the method may further include the following S3-S4 (not shown in the figure).

[0094] S3: For each first voxel in the reconstruction space of a 3D object, determine the flow heuristic value of the first voxel based on the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel; the flow heuristic value is negatively correlated with the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel.

[0095] S4: If the flow heuristic value is less than the heuristic threshold, perform attention flow on the second voxel corresponding to the largest flow heuristic value among the first voxel and the surrounding voxels of the first voxel, and update the multiple attention weights corresponding to multiple vertices in the segmented mesh.

[0096] Here, the first voxel is a cube in the reconstruction space; the heuristic threshold is a pre-configured upper limit value of the flow heuristic; attention flow on the second voxel corresponding to the maximum flow heuristic value among the first voxel and the surrounding voxels of the first voxel is to exchange multiple attention weights corresponding to multiple vertices within the first voxel and the second voxel.

[0097] S3-S4 calculates the flow heuristic value of the first voxel based on the number of vertices within the first voxel and the attention weights corresponding to those vertices. A higher number of vertices and larger attention weights result in a lower flow heuristic value, ensuring that more important first voxels in the segmented mesh have lower flow heuristic values. This provides a flow basis for subsequent attention flow implementation to prevent excessively low flow heuristic values ​​in more important first voxels. By determining if the flow heuristic value of a segment is less than a threshold, attention is shifted from the first voxel to the second voxel with the highest flow heuristic value among its surrounding voxels. This updates the attention weights corresponding to multiple vertices in the segmented mesh, indicating that attention flows from the more important first voxel to the least important surrounding second voxels. This gradually evenly distributes attention in the segmented mesh of the 3D object, preventing subsequent mesh reconstruction that neglects secondary regions of the 3D object.

[0098] See Figure 5 ,Should Figure 5 This is a schematic diagram of attention flow under two-dimensional vision provided in an embodiment of this application; the diagram shows a partial voxel map of multiple voxels in the reconstruction space of a three-dimensional object under two-dimensional vision. The flow inspiration value of the black first voxel is less than the inspiration threshold. The second voxel corresponding to the largest flow inspiration value among the surrounding voxels of the first voxel is the white second voxel. Attention flows from the first voxel to the white second voxel.

[0099] In this embodiment, in the above-described S3, the flow heuristic value of the first voxel is determined by the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel. Specifically, considering the attention weights corresponding to the vertices within the first voxel, the average vertex weight of the vertices within the first voxel can be calculated to represent the voxel weight, thus determining the flow heuristic value of the first voxel. Therefore, the average value of the attention weights corresponding to the vertices within the first voxel can be calculated first as the average vertex weight of the vertices within the first voxel. Then, the flow heuristic value of the first voxel is determined by the number of vertices within the first voxel and the average vertex weight of the vertices within the first voxel, such that the flow heuristic value is negatively correlated with the number of vertices within the first voxel and the average vertex weight of the vertices within the first voxel. That is, the more vertices within the first voxel and the greater the average vertex weight of the vertices within the first voxel, the smaller the flow heuristic value. Based on this, this application provides a possible implementation, and the above S3 may include, for example, the following S31-S32 (not shown in the figure).

[0100] S31: Average the attention weights corresponding to multiple vertices within the first voxel to obtain the average vertex weights of multiple vertices within the first voxel.

[0101] S32: Determine the flow heuristic value based on the number of vertices in the first voxel and the average weight of the vertices in the first voxel.

[0102] The average vertex weight of multiple vertices within the first voxel represents the importance of the first voxel in the reconstruction space of the 3D object.

[0103] S31-S32 uses the average value of the multiple attention weights corresponding to multiple vertices within the first voxel as the vertex average weight of the multiple vertices within the first voxel. Based on the multiple attention weights corresponding to multiple vertices within the first voxel, it intuitively and evenly represents the importance of the first voxel in the reconstruction space of the 3D object as a whole. It can accurately determine the flow heuristic value of the first voxel. The more vertices and the greater the multiple attention weights corresponding to multiple vertices within the first voxel, the smaller the flow heuristic value. It can accurately make the flow heuristic value of the more important first voxel in the reconstruction space of the 3D object smaller, providing an accurate flow basis for subsequent flow of attention from the more important first voxel to the least important second voxel around it.

[0104] As an example of S31-S32, the number of vertices in the first voxel is 'number'; then the computer device first calculates the average of the multiple attention weights corresponding to the multiple vertices in the first voxel, and the average vertex weight in the first voxel is 'weight'. av Then through number and weight av The flow heuristic value of the first voxel is determined to be the score, such that the score is correlated with the number and weight. av Negative correlation.

[0105] Furthermore, in this embodiment, after performing S31 to calculate the average value of multiple attention weights corresponding to multiple vertices within the first voxel as the vertex average weight of multiple vertices within the first voxel, and before performing S32 to determine the flow heuristic value of the first voxel using the number of vertices and the average vertex weight of multiple vertices within the first voxel, considering the different situations of different first voxels in the reconstruction space of the 3D object, an offset for the average vertex weight of multiple vertices within the first voxel can be introduced to adjust the average vertex weight of multiple vertices within the first voxel to be suitable for determining the importance of different first voxels in the reconstruction space of the 3D object. Therefore, based on the average vertex weight of multiple vertices within the first voxel, a second weight can be obtained by offset calculation using a second offset. Correspondingly, in the specific implementation of 32 above, the second weight can more accurately determine the flow heuristic value of the first voxel compared to the average vertex weight of multiple vertices within the first voxel. Therefore, the flow heuristic value of the first voxel is determined using the number of vertices and the second weight. Based on this, this application provides a possible implementation method. After executing S31 and before executing S32, the method may further include S5 (not shown in the figure): calculating the offset of the average weight of the vertices according to the second offset to obtain the second weight; correspondingly, S32 may include S32a (not shown in the figure): determining the flow heuristic value according to the number of vertices of multiple vertices in the first voxel and the second weight.

[0106] The second offset is a pre-configured offset; the offset calculation of the vertex average weight based on the second offset is based on modifying the vertex average weight.

[0107] Based on the average value of multiple attention weights corresponding to multiple vertices within the first voxel, S5 and S32a introduce a second offset applicable to different first voxels in the reconstruction space of the 3D object, to more accurately calculate the weights used to determine the flow heuristic value of the first voxel, so as to more accurately determine the flow heuristic value of the first voxel.

[0108] As an example of S5 and S32a, based on the examples of S31-S32 above, the second offset is β2; then the computer device is in weight av Based on this, the second weight can also be obtained by offset calculation using β2. av +β2, through number and weight av +β2 determines the score, i.e., score = f(number) / (weight) av +β2); where β2 can be 0.4.

[0109] In this embodiment, when the above-described S4 process of flowing attention from the first voxel to the second voxel corresponding to the maximum flow heuristic value among the surrounding voxels of the first voxel to update the multiple attention weights corresponding to multiple vertices in the segmented mesh is specifically implemented, it actually involves flowing the average vertex weight of the multiple attention weights corresponding to multiple vertices within the first voxel to each vertex within the second voxel, and vice versa. Therefore, based on the attention weights of each vertex within the second voxel, attention flow is achieved through the average vertex weight of the multiple attention weights corresponding to multiple vertices within the first voxel to update the attention weights of each vertex within the second voxel; similarly, based on the attention weights of each vertex within the first voxel, attention flow is achieved through the average vertex weight of the multiple attention weights corresponding to multiple vertices within the second voxel to update the attention weights of each vertex within the first voxel. Based on this, this application provides a possible implementation method in which attention flow is performed on the second voxel corresponding to the maximum flow heuristic value among the first voxel and the surrounding voxels of the first voxel in S4 above, and multiple attention weights corresponding to multiple vertices in the segmented mesh are updated. For example, it may include the following S41-S42 (not shown in the figure).

[0110] S41: Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the first voxel, perform attention flow on the attention weight of each vertex in the second voxel and update the attention weight of each vertex in the second voxel.

[0111] S42: Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the second voxel, perform attention flow on the attention weight of each vertex in the first voxel and update the attention weight of each vertex in the first voxel.

[0112] Specifically, the vertex average weight of the multiple attention weights corresponding to multiple vertices within the first voxel is the average value of the multiple attention weights corresponding to multiple vertices within the first voxel; attention flow for the attention weights of each vertex within the second voxel involves modifying the attention weights of each vertex within the second voxel based on the attention weights of each vertex within the second voxel; the vertex average weight of the multiple attention weights corresponding to multiple vertices within the second voxel is the average value of the multiple attention weights corresponding to multiple vertices within the second voxel; attention flow for the attention weights of each vertex within the first voxel involves modifying the attention weights of each vertex within the first voxel based on the attention weights of each vertex within the first voxel.

[0113] S41-S42 transfers the average vertex weight of multiple attention weights corresponding to multiple vertices in the first voxel to each vertex in the second voxel. The average vertex weight of multiple attention weights corresponding to multiple vertices in the first voxel is relatively large, which makes the attention weight of each vertex in the second voxel larger. Conversely, the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel is transferred to each vertex in the first voxel. The average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel is relatively small, which makes the attention weight of each vertex in the first voxel smaller. This truly achieves the gradual and uniform distribution of attention in the line segmentation mesh of the 3D object.

[0114] As an example of S41-S42, the attention weight of each vertex within the first voxel is weight1, and the average vertex weight of the multiple attention weights corresponding to multiple vertices within the first voxel is weight1. av1 The attention weight of each vertex within the second voxel is weight2, and the average vertex weight of the attention weights corresponding to multiple vertices within the second voxel is weight2. av2 Then the computer device, based on weight2, through weight av1 Implement attention flow to update weight2; based on weight1, update weight2... av2 Implement attention flow to update weight1.

[0115] In this embodiment, S41 above, based on the attention weight of each vertex in the second voxel, achieves attention flow by using the vertex average weight of multiple attention weights corresponding to multiple vertices in the first voxel, so as to update the attention weight of each vertex in the second voxel. In specific implementation, considering that the actual situation of different vertices in the second voxel is different, a penalty amount can also be introduced for the vertex average weight of multiple attention weights corresponding to multiple vertices in the first voxel, and the vertex average weight of multiple attention weights corresponding to multiple vertices in the first voxel can be adjusted to be suitable for the attention flow of different vertices in the second voxel. Therefore, based on the attention weight of each vertex in the second voxel, attention flow is achieved by using the vertex average weight of multiple attention weights corresponding to multiple vertices in the first voxel, combined with the penalty amount, so as to update the attention weight of each vertex in the second voxel.

[0116] Correspondingly, in the above-mentioned S42, attention flow is achieved by using the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel, based on the attention weight of each vertex in the first voxel, to update the attention weight of each vertex in the first voxel. In specific implementation, considering that the actual situation of different vertices in the first voxel is different, a penalty amount can also be introduced for the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel to adjust the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel to be applicable to the attention flow of different vertices in the first voxel. Therefore, attention flow is achieved by using the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel, combined with the penalty amount, based on the attention weight of each vertex in the first voxel, to update the attention weight of each vertex in the first voxel. Based on this, this application provides a possible implementation, wherein S41 may include, for example, S41a (not shown in the figure): according to the average vertex weight of the multiple attention weights corresponding to multiple vertices in the first voxel and the penalty amount, attention flow is performed on the attention weight of each vertex in the second voxel to update the attention weight of each vertex in the second voxel; S42 may include, for example, S42a (not shown in the figure): according to the average vertex weight of the multiple attention weights corresponding to multiple vertices in the second voxel and the penalty amount, attention flow is performed on the attention weight of each vertex in the first voxel to update the attention weight of each vertex in the first voxel.

[0117] The penalty amount is a pre-configured parameter.

[0118] S41a-S42a, based on the principle of achieving attention flow for each vertex within the second voxel by averaging the vertex weights of multiple attention weights corresponding to multiple vertices within the first voxel, introduces a penalty amount applicable to attention flow for different vertices within the second voxel, thus achieving more accurate attention flow for each vertex within the second voxel.

[0119] As an example of S41a-S42a, based on the examples of S41-S42 above, if the penalty is subsidy, then the computer device, based on weight2, through weight av1 Integrating with subsidy enables attention flow to update weight2, i.e., weight2 = α × weight2 + (1 - α) × weight av1 +subsidy; Based on weight1, through weightav2 Integrating with subsidy enables attention flow to update weight1, i.e., weight1 = α × weight1 + (1 - α) × weight av2 -subsidy; where subsidy can be 0.001.

[0120] In this embodiment, since the first voxel in S3 is a cube in the reconstruction space, dividing the reconstruction space according to the cube formed by the voxel size can obtain each first voxel in the reconstruction space. Based on this, this application provides a possible implementation, and the step of determining the first voxel in S3 may include, for example, S5 (not shown in the figure): dividing the reconstruction space into voxels according to the voxel size to obtain each first voxel in the reconstruction space.

[0121] Among them, dividing the reconstruction space into voxels based on voxel size means dividing the reconstruction space into cubes formed by voxel size.

[0122] This method precisely divides the reconstruction space according to the cube formed by the voxel size, obtaining each first voxel in the reconstruction space. To avoid neglecting the mesh reconstruction of secondary areas in the future, a voxel-level attention flow mechanism is introduced to lay the foundation.

[0123] In this embodiment of the application, when the above-mentioned S204 is specifically implemented to reconstruct the mesh based on the segmented mesh to obtain the second reconstructed mesh of the three-dimensional object, it is considered that it is also necessary to perform segment collapse, segment flipping and regularization smoothing based on the segmented mesh to complete the mesh reconstruction of the three-dimensional object and obtain the second reconstructed mesh; therefore, firstly, segment collapse is performed based on the segmented mesh to obtain the segment collapsed mesh, then segment flipping is performed based on the segment collapsed mesh to obtain the segment flipped mesh, and finally, regularization smoothing is performed based on the segment flipped mesh to obtain the second reconstructed mesh. Based on this, this application provides a possible implementation method, and the above-mentioned S204 may include, for example, the following S2041-S2043 (not shown in the figure).

[0124] S2041: Perform line segment collapse on the segment-divided mesh to obtain the segment-collapsed mesh of the segment-divided mesh.

[0125] The segment collapse method involves repairing defects in the segmented mesh. Specifically, candidate faces are identified within the segmented mesh. These candidate faces are either faces with an area less than a threshold area, or they are flipped faces. Segment collapse is then performed based on the shortest segment among these candidate faces.

[0126] A flipped face is one where the dot product of the face's normal vector and the face's reference normal vector is less than 0. The face's reference normal vector is the average of the normal vectors of the face's three vertices. The normal vector of each vertex is the weighted sum of the areas of the multiple faces connected to that vertex and the normal vectors of those multiple faces.

[0127] See Figure 6 ,Should Figure 6 This application provides a schematic diagram of a normal vector and a reference normal vector of a surface under two-dimensional vision, as shown in an embodiment of the present application. The diagram represents a portion of the mesh of a three-dimensional object's line segmentation mesh under two-dimensional vision, and the normal vector n of the surface... f With the reference normal vector n of the surface N A dot product less than 0 indicates that the face is a flipped face.

[0128] S2042: Perform segment flipping on the collapsed mesh to obtain the segment flipped mesh of the collapsed mesh.

[0129] The segment flipping process for the collapsed mesh involves calculating the squared degree error of each segment after flipping and minimizing the sum of these squared degree errors to minimize the overall degree loss. Specifically, for each segment in the collapsed mesh, the sum of the squared degree errors before and after flipping is calculated. The vertices connected by the segments are the middle vertices of the collapsed mesh, with an optimal degree of 6. The vertices connected by the segments are the boundary vertices of the collapsed mesh, with an optimal degree of 4. Based on the sum of the squared degree errors before and after flipping, it is determined whether a segment needs to be flipped and its flipping priority. The segment flipping is then performed according to this priority.

[0130] See Figure 7 ,Should Figure 7 This is a schematic diagram illustrating the sum of square degree errors of a line segment before and after flipping in a two-dimensional vision, provided as an embodiment of this application. The diagram represents a portion of the collapsed mesh of a three-dimensional object in a two-dimensional vision. Line segment AB can be flipped into line segment CD. The sum of square degree errors of line segment AB before flipping is the sum of the square degree errors of vertex A (1), vertex B (0), vertex C (4), and vertex D before flipping, i.e., 5. The sum of square degree errors of line segment AB after flipping is the sum of the square degree errors of vertex A (0), vertex B (1), vertex C (1), and vertex D (1) after flipping, i.e., 3. It is determined that line segment AB needs to be flipped into line segment CD.

[0131] S2043: Regularize and smooth the line segment flip mesh to obtain the second reconstructed mesh.

[0132] The S2041-S2043, based on segmented meshes, undergoes segment collapse, segment flipping, and regularization smoothing to repair defects in the segmented meshes, minimize overall degree loss, and smoothly reconstruct the mesh. This enables more accurate mesh reconstruction of important segments in 3D objects, resulting in more refined important areas of 3D objects and a more precise improvement in the mesh reconstruction effect of 3D objects.

[0133] In summary, see [link / reference] Figure 8 ,Should Figure 8 This application provides a schematic diagram of a mesh reconstruction step based on a three-dimensional object; in the above examples and above Figure 3 -the above Figure 7 Based on this, mesh reconstruction based on 3D objects includes the following steps:

[0134] Step 1: Pre-reconstruction.

[0135] By capturing multiple images of a 3D object from multiple perspectives, a mesh reconstruction (e.g., 50 mesh reconstructions) is performed on the 3D object. The vertices in the 3D object's mesh are updated, and line segments in the 3D object's mesh are segmented, collapsed, and flipped to obtain the first reconstructed mesh of the 3D object. Each mesh reconstruction updates the vertices in the 3D object's mesh using a mesh optimizer, and the mesh reconstruction is achieved through the line segmentation, line collapse, and line flipping methods described in the above embodiments.

[0136] Step 2: Attention annotation.

[0137] Attention annotations are applied to each vertex in the first reconstructed mesh of the 3D object, giving each vertex an attention weight. The attention annotations control the distance between the center and radius of a sphere by selecting it, and the vertices inside the sphere are color-coded, with darker colors indicating greater attention weight.

[0138] Step 3: Vertex update.

[0139] Update the vertices in the mesh of a 3D object using the mesh optimizer.

[0140] Step 4: Line segment division.

[0141] In the above embodiments, S201-S203 determines the importance of line segments in the first reconstructed mesh by using the two attention weights corresponding to the two vertices connected by line segments in the first reconstructed mesh of the 3D object. The first length of the line segment is updated to obtain the second length, so that the more important line segments in the first reconstructed mesh are longer, and the longer they are, the more likely they are to be divided into more line segments, resulting in a more refined line segmentation mesh for the important line segments of the 3D object.

[0142] Step 5: Attention Flow.

[0143] In the above embodiments, S3-S4 calculates the flow heuristic value of the first voxel by using the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel. The more vertices within the first voxel and the greater the attention weights corresponding to the vertices within the first voxel, the smaller the flow heuristic value, resulting in a lower flow heuristic value for the more important first voxel in the segmented mesh. Attention is then shifted from the more important first voxel to the least important second voxel among the surrounding voxels of the first voxel, updating the attention weights corresponding to the vertices in the segmented mesh, so that the attention in the segmented mesh of the 3D object is gradually and evenly distributed.

[0144] Step Six: Grid Reconstruction.

[0145] In the above embodiments, S2041-S2043, based on the segmented mesh, performs segment collapse, segment flipping, and regularization smoothing to complete the mesh reconstruction of the three-dimensional object and obtain the second reconstructed mesh.

[0146] See Figure 9 ,Should Figure 9 This document provides a schematic diagram of a reconstructed mesh of a 3D object in a related art and a second reconstructed mesh of a 3D object in an embodiment of this application. Based on the above example, in this figure (a), the head mesh and arm mesh of the game character in the second reconstructed mesh of this application are shown; in this figure (b), the head mesh and left arm mesh of the game character in the reconstructed mesh of a related art are shown; in this figure (c), the right arm mesh of the game character in the second reconstructed mesh of this application are shown; and in this figure (d), the right arm mesh of the game character in the reconstructed mesh of a related art is shown. This embodiment of the application achieves better mesh reconstruction of the game character with fewer mesh resources, especially with more refined and detailed reconstructed meshes in important areas such as the head and arm regions of the game character.

[0147] The mesh reconstruction method based on 3D objects provided in this application can be applied to the mesh reconstruction of game characters. In the process of mesh reconstruction of game characters, the importance of line segments in the first reconstructed mesh is determined by the two attention weights corresponding to the two vertices connected by line segments in the first reconstructed mesh of the game character. The first length of the line segment is updated to obtain the second length, so that the more important line segments in the first reconstructed mesh are longer. The longer the length, the more likely it is to be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. This highlights the important areas to which the important line segments of the game character belong. Mesh reconstruction of the game character is achieved based on the line segmentation mesh, making the reconstructed mesh of the important areas of the game character more refined, thereby improving the mesh reconstruction effect of the game character.

[0148] Without significantly increasing the size of the game character's rebuilt file, more detail was successfully added to the important areas of the game character, thereby optimizing memory usage, improving the visual effects of the game character, enhancing the game character's expressiveness, and ensuring the running efficiency of the game system.

[0149] It should be noted that, based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods.

[0150] based on Figure 2 Corresponding to the mesh reconstruction method based on three-dimensional objects provided in the embodiments, this application also provides a mesh reconstruction apparatus based on three-dimensional objects, see [link to relevant documentation]. Figure 10 , Figure 10 The present application provides a structural diagram of a mesh reconstruction device based on a three-dimensional object. The mesh reconstruction device 1000 based on a three-dimensional object includes: an acquisition unit 1001, an update unit 1002, a segmentation unit 1003, and a reconstruction unit 1004.

[0151] The acquisition unit 1001 is used to acquire two attention weights corresponding to the two vertices connected by each line segment in the first reconstructed mesh of the 3D object; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh.

[0152] Update unit 1002 is used to update the first length of the line segment according to the two attention weights to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights.

[0153] The segmentation unit 1003 is used to segment the line segment if the second length is greater than the maximum length, thereby obtaining the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction cycle;

[0154] Reconstruction unit 1004 is used to reconstruct the line segment mesh to obtain the second reconstructed mesh of the three-dimensional object.

[0155] In one possible implementation, update unit 1002 is used for:

[0156] The line segment weight is obtained by averaging the two attention weights corresponding to the two vertices connected by the line segment.

[0157] The first length is updated based on the segment weights to obtain the second length.

[0158] In one possible implementation, the device further includes: a first computing unit;

[0159] The first calculation unit is used to perform offset calculation on the line segment weight based on the first offset to obtain the first weight;

[0160] Update unit 1002 is used for:

[0161] The first length is updated based on the first weight to obtain the second length.

[0162] In one possible implementation, the device further includes: a second computing unit;

[0163] The second calculation unit is used to calculate the length of the optimal length based on the tolerance value to obtain the maximum length; the maximum length is greater than the optimal length.

[0164] In one possible implementation, the apparatus further includes: a determining unit and a flow unit;

[0165] The determination unit is used to determine the flow heuristic value of each first voxel in the reconstruction space of the 3D object based on the number of vertices in the first voxel and the attention weights corresponding to the vertices in the first voxel. The flow heuristic value is negatively correlated with the number of vertices in the first voxel and the attention weights corresponding to the vertices in the first voxel.

[0166] The flow unit is used to perform attention flow on the second voxel corresponding to the largest flow heuristic value among the first voxel and the surrounding voxels of the first voxel if the flow heuristic value is less than the heuristic threshold, and update multiple attention weights corresponding to multiple vertices in the line segmentation mesh.

[0167] In one possible implementation, a unit is defined for:

[0168] The average attention weights corresponding to multiple vertices within the first voxel are calculated to obtain the average vertex weights of multiple vertices within the first voxel.

[0169] The flow heuristic value is determined based on the number of vertices in the first voxel and the average weight of the vertices in the first voxel.

[0170] In one possible implementation, the device further includes: a second computing unit;

[0171] The second calculation unit is used to perform offset calculation on the average weight of the vertices based on the second offset to obtain the second weight;

[0172] Determine the unit, used for:

[0173] The flow heuristic value is determined based on the number of vertices in the first voxel and the second weight.

[0174] In one possible implementation, the flow unit is used for:

[0175] Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the first voxel, attention flow is performed on the attention weight of each vertex in the second voxel to update the attention weight of each vertex in the second voxel.

[0176] Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the second voxel, attention flow is performed on the attention weight of each vertex in the first voxel to update the attention weight of each vertex in the first voxel.

[0177] In one possible implementation, the flow unit is used for:

[0178] Based on the average vertex weight of multiple attention weights corresponding to multiple vertices in the first voxel and the penalty amount, attention flow is performed on the attention weight of each vertex in the second voxel to update the attention weight of each vertex in the second voxel.

[0179] Based on the average vertex weight of multiple attention weights corresponding to multiple vertices in the second voxel, and the penalty amount, attention flow is performed on the attention weight of each vertex in the first voxel to update the attention weight of each vertex in the first voxel.

[0180] In one possible implementation, determining the unit is also used for:

[0181] The reconstruction space is divided into voxels based on the voxel size to obtain the first voxel in the reconstruction space.

[0182] In one possible implementation, reconstruction unit 1004 is used for:

[0183] Perform line segment collapse on the line segment segmented mesh to obtain the line segment collapsed mesh of the line segment segmented mesh;

[0184] Perform segment flipping on the collapsed segment mesh to obtain the segment flipped mesh of the collapsed segment mesh;

[0185] The line segment flip mesh is regularized and smoothed to obtain the second reconstructed mesh.

[0186] As can be seen from the above technical solution, each vertex in the first reconstructed mesh of the 3D object has an attention weight, representing the importance of each vertex in the first reconstructed mesh. For each line segment in the first reconstructed mesh of the 3D object, firstly, the two attention weights corresponding to the two vertices connected by the line segment are obtained, so that the importance of the line segment in the first reconstructed mesh can be determined by the importance of the two vertices connected by the line segment in the first reconstructed mesh. The first length of the line segment is updated to obtain the second length by using the two attention weights representing the importance of the line segment in the first reconstructed mesh, so that the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. That is, the larger the two attention weights are, the larger the ratio between the second length and the first length is, which makes the length of the more important line segments in the first reconstructed mesh longer. Then, it is determined whether the second length of the line segment is greater than the maximum length determined by the optimal length of the 3D object in the mesh reconstruction round. If so, the line segment is divided to obtain a line segmentation mesh, indicating that the more important line segments in the first reconstructed mesh are more likely to be too long and can be divided into more line segments, resulting in a more refined line segmentation mesh for important line segments. Based on the line segmentation mesh, mesh reconstruction is performed to obtain a more refined second reconstructed mesh for the important regions of the 3D object.

[0187] Based on this, the device determines the importance of line segments in the first reconstructed mesh by using two attention weights corresponding to the two vertices connected by line segments in the first reconstructed mesh of the 3D object. It then updates the first length of the line segments to obtain the second length, so that the more important line segments in the first reconstructed mesh are longer. This makes it easier for the length to be too long and need to be divided into more line segments, resulting in a finer line segmentation mesh for important line segments. This highlights the important regions to which the important line segments of the 3D object belong. The mesh reconstruction of the 3D object is achieved based on the line segmentation mesh, making the reconstructed mesh of the important regions of the 3D object more refined, thereby improving the mesh reconstruction effect of the 3D object.

[0188] This application also provides a computer device, which may be a server, see [link to previous document]. Figure 11 , Figure 11This application provides a structural diagram of a server 1100. The server 1100 can vary significantly due to different configurations or performance. It may include one or more processors, such as a central processing unit (CPU) 1122, and a memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1122 may be configured to communicate with the storage media 1130 and execute the series of instruction operations stored in the storage media 1130 on the server 1100.

[0189] Server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0190] In this embodiment, the central processing unit 1122 in the server 1100 can execute the methods provided in the various optional implementations of the above embodiments.

[0191] The computer device provided in this application embodiment can also be a terminal, see [link to relevant documentation]. Figure 12 , Figure 12 This is a structural diagram of a terminal provided in an embodiment of this application. Taking a smartphone as an example, the smartphone includes components such as a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless Fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290. The input unit 1230 may include a touch panel 1231 and other input devices 1232, the display unit 1240 may include a display panel 1241, and the audio circuit 1260 may include a speaker 1261 and a microphone 1262. Those skilled in the art will understand that... Figure 12 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0192] The memory 1220 can be used to store software programs and modules. The processor 1280 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 1220. The memory 1220 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 1220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0193] The processor 1280 is the control center of the smartphone, connecting various parts of the smartphone via various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1220 and calling data stored in the memory 1220. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1280.

[0194] In this embodiment, the processor 1280 in the smartphone can execute the methods provided in the various optional implementations of the above embodiments.

[0195] According to one aspect of this application, a computer-readable storage medium is provided for storing a computer program that, when run on a computer device, causes the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0196] According to one aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above embodiments.

[0197] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0198] The terms "first," "second," etc., used in this application's specification and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0199] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0200] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0201] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0202] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), RAM, magnetic disks, or optical disks.

[0203] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0204] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A mesh reconstruction method based on three-dimensional objects, characterized in that, The method includes: For each line segment in the first reconstructed mesh of a 3D object, two attention weights are obtained corresponding to the two vertices connected by the line segment; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh. Based on the two attention weights corresponding to the two vertices connected by the line segment, the first length of the line segment is updated to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. If the second length is greater than the maximum length, the line segment is segmented to obtain the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction cycle; The line segment mesh is reconstructed to obtain the second reconstructed mesh of the three-dimensional object.

2. The method according to claim 1, characterized in that, The step of updating the first length of the line segment based on the two attention weights corresponding to the two vertices connected by the line segment to obtain the second length of the line segment includes: The line segment weight is obtained by averaging the two attention weights corresponding to the two vertices connected by the line segment. The first length is updated according to the line segment weight to obtain the second length.

3. The method according to claim 2, characterized in that, The method further includes: The weight of the line segment is offset based on the first offset to obtain the first weight; The step of updating the first length according to the line segment weight to obtain the second length includes: The first length is updated according to the first weight to obtain the second length.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the maximum length includes: The maximum length is obtained by calculating the length of the optimal length based on the tolerance value; the maximum length is greater than the optimal length.

5. The method according to claim 1, characterized in that, After obtaining the line segmentation mesh of the first reconstructed mesh, the method further includes: For each first voxel in the reconstruction space of the 3D object, a flow heuristic value for the first voxel is determined based on the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel. The flow heuristic value is negatively correlated with the number of vertices within the first voxel and the attention weights corresponding to the vertices within the first voxel. If the flow heuristic value is less than the heuristic threshold, attention flow is performed on the second voxel corresponding to the maximum flow heuristic value among the first voxel and the surrounding voxels of the first voxel, and the multiple attention weights corresponding to multiple vertices in the segmented mesh are updated.

6. The method according to claim 5, characterized in that, The step of determining the flow heuristic value of the first voxel based on the number of vertices in the first voxel and the multiple attention weights corresponding to the multiple vertices in the first voxel includes: The average attention weights corresponding to multiple vertices within the first voxel are calculated to obtain the average vertex weights of multiple vertices within the first voxel. The flow heuristic value is determined based on the number of vertices in the first voxel and the average weight of the vertices in the first voxel.

7. The method according to claim 6, characterized in that, The method further includes: The second weight is obtained by offsetting the average weight of the vertices based on the second offset; The step of determining the flow heuristic value based on the number of vertices in the first voxel and the average vertex weight of the vertices in the first voxel includes: The flow heuristic value is determined based on the number of vertices in the first voxel and the second weight.

8. The method according to claim 5, characterized in that, The step of performing attention flow on the second voxel corresponding to the maximum flow heuristic value among the first voxel and its surrounding voxels, and updating multiple attention weights corresponding to multiple vertices in the segmented mesh, includes: Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the first voxel, attention flow is performed on the attention weight of each vertex in the second voxel to update the attention weight of each vertex in the second voxel. Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the second voxel, attention flow is performed on the attention weight of each vertex in the first voxel to update the attention weight of each vertex in the first voxel.

9. The method according to claim 8, characterized in that, The step of performing attention flow on the attention weight of each vertex in the second voxel based on the average vertex weight of multiple attention weights corresponding to multiple vertices in the first voxel, and updating the attention weight of each vertex in the second voxel, includes: Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the first voxel and the penalty amount, attention flow is performed on the attention weight of each vertex in the second voxel to update the attention weight of each vertex in the second voxel. The step of performing attention flow on the attention weights of each vertex in the first voxel based on the multiple attention weights corresponding to multiple vertices in the second voxel, and updating the attention weights of each vertex in the first voxel, includes: Based on the average vertex weight of the multiple attention weights corresponding to multiple vertices in the second voxel and the penalty amount, attention flow is performed on the attention weight of each vertex in the first voxel to update the attention weight of each vertex in the first voxel.

10. The method according to claim 5, characterized in that, The steps for determining the first voxel include: The reconstruction space is divided into voxels according to the voxel size to obtain each first voxel in the reconstruction space.

11. The method according to claim 1, characterized in that, The step of reconstructing the line segment mesh to obtain the second reconstructed mesh of the 3D object includes: The segmented mesh is collapsed to obtain the collapsed mesh of the segmented mesh. The collapsed line segment mesh is flipped to obtain the flipped line segment mesh of the collapsed line segment mesh; The line segment flip mesh is regularized and smoothed to obtain the second reconstructed mesh.

12. A mesh reconstruction device based on three-dimensional objects, characterized in that, The device includes: an acquisition unit, an update unit, a segmentation unit, and a reconstruction unit; The acquisition unit is used to acquire two attention weights corresponding to the two vertices connected by each line segment in the first reconstructed mesh of the 3D object; the two attention weights represent the importance of the two vertices connected by the line segment in the first reconstructed mesh. The update unit is used to update the first length of the line segment according to the two attention weights to obtain the second length of the line segment; the ratio between the second length and the first length is positively correlated with the magnitude of the two attention weights. The segmentation unit is used to segment the line segment if the second length is greater than the maximum length, to obtain the segmented mesh of the first reconstructed mesh; the maximum length is determined based on the optimal length of the 3D object in the mesh reconstruction cycle; The reconstruction unit is used to reconstruct the line segment mesh to obtain a second reconstructed mesh of the three-dimensional object.

13. A computer device, characterized in that, The computer device includes a processor and memory: The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute the method according to any one of claims 1-11 according to instructions in the computer program.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer device, causes the computer device to perform the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is run on a computer device, it causes the computer device to perform the method according to any one of claims 1-11.

Images