Three-dimensional modeling method

Abstract

The invention provides a three-dimensional modeling method which can be applied to the technical field of computer graphic processing. The method comprises the following steps: generating a model by using a three-dimensional structure sequence, and generating a modeling sequence corresponding to data features based on the data features of to-be-modeled data; the modeling sequence is decoded, a model geometric element set of a target three-dimensional model corresponding to the to-be-modeled data is determined, and the model geometric element set comprises a point element set, an edge element set determined according to the point element set and the corresponding relation between point elements and edge elements; and performing three-dimensional reconstruction according to the model geometric element set to obtain a target three-dimensional model.

Description

Technical Field

[0001] This disclosure relates to the field of computer graphics processing technology, and more specifically to a three-dimensional modeling method. Background Technology

[0002] 3D modeling technology has wide and important applications in many fields such as industrial design, film and television production, architectural planning, and virtual reality. Its core goal is to construct 3D digital models that accurately reflect real objects or virtual concepts through digital means. With the rapid development of related fields, increasingly higher demands are being placed on the accuracy, efficiency, and ability to process complex data in 3D modeling.

[0003] In existing 3D modeling methods, some rely on manual construction, which is not only time-consuming and labor-intensive, but also struggles to guarantee accuracy and consistency when modeling complex structures, making it difficult to meet the needs of large-scale, high-precision modeling. Automated modeling methods, on the other hand, often directly extract features from the data for modeling, lacking a systematic approach to the relationship between data features and the geometric elements of the 3D model. This can result in the generated 3D model failing to accurately reflect the essential characteristics of the data. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a three-dimensional modeling method.

[0005] According to a first aspect of this disclosure, a three-dimensional modeling method is provided, comprising: generating a model using a three-dimensional structure sequence; generating a modeling sequence corresponding to the data features based on the data features of the data to be modeled; decoding the modeling sequence to determine a set of model geometric elements of a target three-dimensional model corresponding to the data to be modeled, wherein the set of model geometric elements includes a set of point elements, a set of edge elements determined based on the set of point elements, and a correspondence between each point element and each edge element; and performing three-dimensional reconstruction based on the set of model geometric elements to obtain a target three-dimensional model.

[0006] According to embodiments of this disclosure, the target 3D model includes multiple 3D components, and the model geometric feature set includes the component geometric feature sets of each of the multiple 3D components. Decoding the modeling sequence to determine the model geometric feature set of the target 3D model corresponding to the data to be modeled includes: dividing the modeling sequence according to the delimiters in the modeling sequence to obtain multiple modeling sub-sequences, wherein the multiple modeling sub-sequences correspond one-to-one with the multiple 3D components; decoding each modeling sub-sequence according to a preset encoding rule to obtain the component geometric feature set of the 3D component corresponding to the modeling sub-sequence; and determining the model geometric feature set based on the component geometric feature sets of each of the multiple 3D components.

[0007] According to embodiments of this disclosure, for each modeling subsequence, the modeling subsequence is decoded according to a preset encoding rule to obtain a set of component geometric elements of the 3D component corresponding to the modeling subsequence. This includes: determining the encoding length and encoding composition of each point element according to the preset encoding rule; segmenting the modeling subsequence according to the encoding length to obtain the encodings of multiple point elements; parsing the encoding of each point element based on the encoding composition to determine the point element features, wherein the point element features include the position of the point element, the connection relationship between the point element and other point elements, and the edge elements corresponding to the point element, wherein the edge elements are determined based on two interconnected point elements; and determining the set of component geometric elements based on the point element features of multiple point elements.

[0008] According to embodiments of this disclosure, a target 3D model is obtained by performing 3D reconstruction based on a geometric feature set, including: determining an edge element set based on the connectivity relationships between multiple point elements included in the geometric feature set; determining at least one closed loop composed of multiple edge elements based on the positional relationships between multiple edge elements included in the edge element set, wherein the positional relationships are determined based on the distances between the multiple edge elements; constructing surface elements based on each closed loop; and determining the target 3D model based on the positional relationships of the multiple surface elements in space.

[0009] According to embodiments of this disclosure, for each closed loop, surface elements are constructed based on the closed loop, including: for each closed loop, arbitrarily selecting an edge element constituting the closed loop, and arbitrarily selecting a point element corresponding to the edge element as a starting point element; starting from the starting point element, diffusion is carried out through the point elements corresponding to the multiple edge elements constituting the closed loop to the multiple edge elements constituting the closed loop respectively, determining the diffusion range traversed when diffusion is carried out from the starting point element to each edge element; and determining the surface element corresponding to the closed loop based on the diffusion range traversed and the closed loop.

[0010] According to embodiments of this disclosure, the 3D modeling method further includes: for a current closed loop, determining the distances between multiple edge elements constituting the current closed loop and surface elements constructed based on other closed loops; taking the surface element corresponding to the smallest distance among the multiple distances as the target surface element; and updating the positional relationship of the surface elements in space using the relationship between the surface elements constructed based on the current closed loop and the target surface element.

[0011] According to embodiments of this disclosure, a three-dimensional structure sequence generation model is used to generate a modeling sequence corresponding to the data features of the data to be modeled. This includes: predicting the next prediction data based on the current prediction result, and updating the current prediction result using the next prediction data. The current prediction result includes the data features and multiple next prediction data obtained by iterative prediction based on the data features. When making the first prediction, the data features are used as the current prediction result. If the next prediction data is empty, the encoding format of the current prediction result is verified. If the current prediction result passes the encoding format verification, the current prediction result is used as the modeling sequence.

[0012] According to embodiments of this disclosure, the data to be modeled includes a three-dimensional point cloud; the three-dimensional modeling method further includes: when it is determined that the modality of the input data input to the three-dimensional model sequence to generate the model is not a three-dimensional point cloud, converting the input data into a three-dimensional stereograph; and extracting the three-dimensional point cloud from the three-dimensional stereograph to obtain the data to be modeled.

[0013] According to embodiments of this disclosure, a three-dimensional structure sequence generation model is trained by: obtaining a sample geometric feature set of a sample three-dimensional model; encoding the sample geometric feature set using the sample three-dimensional structure sequence generation model to obtain a sample modeling sequence; determining a standard modeling sequence of the sample three-dimensional model based on the sample geometric feature set; and adjusting the model parameters of the sample three-dimensional structure sequence generation model based on the modeling loss determined by the sample modeling sequence and the standard modeling sequence to obtain a three-dimensional structure sequence generation model.

[0014] According to embodiments of this disclosure, the modeling loss determined based on the sample modeling sequence and the standard modeling sequence is determined by: determining a geometric topology loss based on the sample modeling sequence and the standard modeling sequence, wherein the geometric topology loss represents the loss caused by the sample modeling sequence's incorrect prediction of the topological relationships between the sample geometric feature sets compared to the standard modeling sequence; determining a coding accuracy loss based on the sample coding rules applied when coding the sample geometric feature sets; and determining a modeling loss based on the geometric topology loss and the coding accuracy loss.

[0015] A second aspect of this disclosure provides a three-dimensional modeling apparatus, comprising: a sequence generation module for generating a model using a three-dimensional structure sequence, generating a modeling sequence corresponding to the data features based on the data features of the data to be modeled; a sequence decoding module for decoding the modeling sequence to determine a set of model geometric elements of a target three-dimensional model corresponding to the data to be modeled, wherein the set of model geometric elements includes a set of point elements, a set of edge elements determined based on the set of point elements, and a correspondence between each point element and each edge element; and a three-dimensional reconstruction module for performing three-dimensional reconstruction based on the set of model geometric elements to obtain a target three-dimensional model.

[0016] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0017] A fourth aspect of this disclosure also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0018] The fifth aspect of this disclosure also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0019] According to embodiments of this disclosure, a modeling sequence is generated by generating a model through a 3D structure sequence, and then decoded to obtain a set of geometric elements of the model and complete 3D reconstruction. By transforming the features of the data to be modeled into an ordered modeling sequence, geometric elements such as point elements, line elements, and surface elements in the modeling process, as well as the relationships between geometric elements, can be uniformly represented using the modeling sequence. During the decoding of the modeling sequence, topological structure generation and geometric shape restoration can be achieved simultaneously, thereby improving modeling efficiency. Furthermore, introducing a modeling sequence into the intermediate process of 3D modeling based on data features allows for the systematic capture of the correlation between data features and the geometric elements constituting the target 3D model, making the generated target 3D model more closely match the essential features of the data to be modeled, thus improving the accuracy and relevance of 3D modeling. Attached Figure Description

[0020] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 The diagram illustrates an application scenario of the three-dimensional modeling method according to embodiments of the present disclosure.

[0022] Figure 2 A flowchart illustrating a three-dimensional modeling method according to an embodiment of the present disclosure is shown schematically;

[0023] Figure 3 The illustration shows a schematic diagram of dividing sample surface elements into Volonoi segments according to a three-dimensional modeling method based on an embodiment of the present disclosure;

[0024] Figure 4 This illustration schematically shows the process of generating a sample modeling sequence using a three-dimensional modeling method according to an embodiment of the present disclosure;

[0025] Figure 5This illustration schematically shows the process of performing connection topology encoding and geometric relation encoding in a 3D modeling method according to an embodiment of the present disclosure;

[0026] Figure 6 The illustration shows a schematic diagram of closed-loop construction using a three-dimensional modeling method according to an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram illustrating the structure of a three-dimensional modeling apparatus according to an embodiment of the present disclosure; and

[0028] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a three-dimensional modeling method according to an embodiment of the present disclosure. Detailed Implementation

[0029] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0031] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0032] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0033] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this disclosure all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0035] The embodiments of this disclosure provide a three-dimensional modeling method, which uses a three-dimensional structure sequence to generate a model. Based on the data features of the data to be modeled, a modeling sequence corresponding to the data features is generated. The modeling sequence is decoded to determine the model geometric element set of the target three-dimensional model corresponding to the data to be modeled. The model geometric element set includes a set of point elements, a set of edge elements determined based on the set of point elements, and the correspondence between each point element and each edge element. Based on the model geometric element set, three-dimensional reconstruction is performed to obtain the target three-dimensional model.

[0036] Figure 1 The diagram illustrates an application scenario of the three-dimensional modeling method according to an embodiment of the present disclosure.

[0037] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103 to send modeling requirements to the server 105.

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a server that responds to modeling requests uploaded by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103, and generates 3D modeling results.

[0041] It should be noted that the 3D modeling method provided in this embodiment can generally be executed by server 105. Correspondingly, the 3D modeling device provided in this embodiment can generally be located in server 105. The 3D modeling method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the 3D modeling device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] The following will be based on Figure 1 The described scene, through Figures 2-6 The three-dimensional modeling method of the present disclosure will be described in detail.

[0044] Figure 2 A flowchart illustrating a three-dimensional modeling method according to an embodiment of the present disclosure is shown schematically.

[0045] like Figure 2 As shown, the three-dimensional modeling method of this embodiment includes operations S210 to S230.

[0046] In operation S210, a three-dimensional structural sequence is used to generate a model. Based on the data features of the data to be modeled, a modeling sequence corresponding to the data features is generated.

[0047] In operation S220, the modeling sequence is decoded to determine the set of model geometric elements of the target 3D model corresponding to the data to be modeled.

[0048] In operation S230, the target 3D model is obtained by performing 3D reconstruction based on the model's geometric feature set.

[0049] According to embodiments of this disclosure, before performing 3D modeling, feature extraction can be performed on the data to be modeled to obtain data features of the data to be modeled, and the data features can be input into the 3D structure sequence generation model. The data to be modeled includes data in the form of 3D point clouds.

[0050] According to embodiments of this disclosure, a three-dimensional structure sequence generation model can be used to generate a modeling sequence corresponding to data features using an autoregressive prediction method. The autoregressive prediction method utilizes historical values ​​from time series data to predict future values.

[0051] In this embodiment, both the data features and the modeling sequence are time series data. In the process of generating the modeling sequence, a single data point is first predicted based on the data features. Then, based on the data features and at least one data point that has already been predicted, the next data point is predicted. The above prediction process is repeated until the predicted next data point is empty, at which point the iteration stops, and the predicted multiple data points are used as the modeling sequence.

[0052] According to embodiments of this disclosure, the model geometric element set includes a point element set, an edge element set determined based on the point element set, and the correspondence between each point element and each edge element.

[0053] According to embodiments of this disclosure, the target 3D model is a 3D model with the same shape and structure as the point cloud included in the data to be modeled, and the target 3D model can be represented using boundary representation (B-rep).

[0054] According to embodiments of this disclosure, the model geometric element set includes multiple geometric elements, wherein the geometric elements may include point elements in the point element set, edge elements in the edge element set, and surface elements composed of multiple point elements and multiple edge elements, determined according to the correspondence between each point element and each edge element.

[0055] According to embodiments of this disclosure, a target 3D model can be obtained by 3D reconstruction based on multiple face elements, a set of point elements and a set of edge elements included in the model geometric element set, and the positional relationship of the aforementioned geometric elements in space. The positional relationship between any two point elements in space can include connected and disconnected. When two point elements are connected, there are edge elements between the two point elements. The positional relationship between multiple edge elements in space can include coplanar and non-coplanar. When multiple edge elements are coplanar, face elements can be determined based on the coplanar multiple edge elements.

[0056] According to embodiments of this disclosure, a modeling sequence is generated by generating a model through a 3D structure sequence, and then decoded to obtain a set of geometric elements of the model and complete 3D reconstruction. By transforming the features of the data to be modeled into an ordered modeling sequence, geometric elements such as point elements, line elements, and surface elements in the modeling process, as well as the relationships between geometric elements, can be uniformly represented using the modeling sequence. During the decoding of the modeling sequence, topological structure generation and geometric shape restoration can be achieved simultaneously, thereby improving modeling efficiency. Furthermore, introducing a modeling sequence into the intermediate process of 3D modeling based on data features allows for the systematic capture of the correlation between data features and the geometric elements constituting the target 3D model, making the generated target 3D model more closely match the essential features of the data to be modeled, thus improving the accuracy and relevance of 3D modeling.

[0057] According to embodiments of this disclosure, the target 3D model includes multiple 3D components, and the model geometric feature set includes the component geometric feature sets of each of the multiple 3D components. Decoding the modeling sequence to determine the model geometric feature set of the target 3D model corresponding to the data to be modeled includes: dividing the modeling sequence according to the delimiters in the modeling sequence to obtain multiple modeling sub-sequences, wherein the multiple modeling sub-sequences correspond one-to-one with the multiple 3D components; decoding each modeling sub-sequence according to a preset encoding rule to obtain the component geometric feature set of the 3D component corresponding to the modeling sub-sequence; and determining the model geometric feature set based on the component geometric feature sets of each of the multiple 3D components.

[0058] According to embodiments of this disclosure, a three-dimensional component is a substructure constituting a target three-dimensional model. By combining multiple three-dimensional components according to a preset positional relationship, a target three-dimensional model can be obtained. The positional relationship between the three-dimensional components can be used to represent the position of multiple three-dimensional components in space, such as three-dimensional component A being above three-dimensional component B, three-dimensional component C being to the left of three-dimensional component D, etc.

[0059] According to embodiments of this disclosure, the modeling sequence contains multiple delimiters. The modeling sequence can be segmented based on these delimiters. Specifically, starting from the beginning of the modeling sequence, the portion between two adjacent delimiters is defined as a modeling subsequence. By sequentially identifying all delimiters in the modeling sequence and completing the segmentation process based on them, multiple modeling subsequences can be obtained. In particular, for the first delimiter in the modeling sequence, the portion from the beginning of the modeling sequence to that delimiter is defined as a modeling subsequence. For the last delimiter in the modeling sequence, the portion from that delimiter to the end of the modeling sequence is defined as a modeling subsequence.

[0060] According to embodiments of this disclosure, the preset encoding rule may include the length of the encoding of each point element and the physical meaning of each bit in the encoding of the point element. Therefore, when decoding the modeling subsequence according to the preset encoding rule, the modeling subsequence can be further segmented, the encoding of multiple point elements constituting the modeling subsequence can be determined, and multiple point elements, multiple edge elements determined by the multiple point elements, and the correspondence between each point element and each edge element can be determined according to the physical meaning of each bit in the encoding of the point elements, thereby obtaining the component geometric element set.

[0061] According to embodiments of this disclosure, the model geometry set can be obtained by summing up the component geometry sets of multiple 3D components.

[0062] According to embodiments of this disclosure, multiple modeling sub-sequences are obtained based on delimiters in the modeling sequence, and the component geometric element sets of each 3D component are decoded to achieve component-based processing of the target 3D model. This divide-and-conquer approach facilitates accurate modeling of each part of a complex 3D model, improves the flexibility and refinement of modeling, and also facilitates subsequent individual adjustments and optimizations of each component.

[0063] According to embodiments of this disclosure, for each modeling subsequence, the modeling subsequence is decoded according to a preset encoding rule to obtain a set of component geometric elements of the 3D component corresponding to the modeling subsequence. This includes: determining the encoding length and encoding composition of each point element according to the preset encoding rule; segmenting the modeling subsequence according to the encoding length to obtain the encodings of multiple point elements; parsing the encoding of each point element based on the encoding composition to determine the point element features, wherein the point element features include the position of the point element, the connection relationship between the point element and other point elements, and the edge elements corresponding to the point element, wherein the edge elements are determined based on two interconnected point elements; and determining the set of component geometric elements based on the point element features of multiple point elements.

[0064] According to embodiments of this disclosure, the encoding of a point element can include the physical meaning of each bit or bits in the encoding. For example, the encoding of a point element can include the coordinates of the point element in space, the connection relationship between the point element and other point elements, and the correspondence between the point element and edge elements. Therefore, parsing the encoding of point elements can determine the point element characteristics of each point element, and based on the connection relationship between the point element and other point elements in the point element characteristics, edge elements can be determined, and based on the edge elements corresponding to the point elements, surface elements can be determined, thereby determining the component geometric element set.

[0065] According to embodiments of this disclosure, the modeling subsequence is parsed according to preset encoding rules to clarify the encoding length, composition, and characteristics of point elements, enabling accurate extraction of key information such as the position and connection relationships of point elements. This ensures accurate conversion from the modeling subsequence to the component geometric element set, providing reliable basic data for subsequent 3D reconstruction, reducing information loss or misjudgment, and improving the accuracy of component modeling.

[0066] According to embodiments of this disclosure, a target 3D model is obtained by performing 3D reconstruction based on a geometric feature set, including: determining an edge element set based on the connectivity relationships between multiple point elements included in the geometric feature set; determining at least one closed loop composed of multiple edge elements based on the positional relationships between multiple edge elements included in the edge element set, wherein the positional relationships are determined based on the distances between the multiple edge elements; constructing surface elements based on each closed loop; and determining the target 3D model based on the positional relationships of the multiple surface elements in space.

[0067] According to embodiments of this disclosure, based on the connectivity relationships between multiple point elements, it can be determined that there are edge elements between two point elements that are connected and no edge elements between two point elements that are not connected. By traversing all connectivity relationships, the edge element set obtained from the point element set can be determined.

[0068] According to embodiments of this disclosure, based on the positional relationship between multiple edge elements, other edge elements on the same face as each edge element can be determined, and based on the aforementioned positional relationship, a closed loop composed of multiple edge elements can be determined.

[0069] According to embodiments of this disclosure, for each closed loop, surface elements can be determined based on the edge elements constituting the closed loop and the point elements corresponding to the edge elements constituting the closed loop. Based on the positional relationships between multiple edge elements, the positional relationships between multiple closed loops can be determined. Furthermore, the positional relationships between surface elements determined by each of the multiple closed loops are determined. Based on the positional relationships of the multiple surface elements in space, the surface elements are combined and spliced ​​to determine the target three-dimensional model.

[0070] According to embodiments of this disclosure, an edge element set is determined based on the connectivity of point elements, and closed loops are determined and surface elements are constructed based on the positional relationships of the edge elements, ultimately forming the target 3D model. This follows a construction logic from points to edges, from edges to surfaces, and finally to the solid model. This step-by-step construction method ensures the correlation and consistency between various geometric elements, making the generated 3D model structure more reasonable and complete, and improving the reliability of modeling.

[0071] According to embodiments of this disclosure, for each closed loop, surface elements are constructed based on the closed loop, including: for each closed loop, arbitrarily selecting an edge element constituting the closed loop, and arbitrarily selecting a point element corresponding to the edge element as a starting point element; starting from the starting point element, diffusion is carried out through the point elements corresponding to the multiple edge elements constituting the closed loop to the multiple edge elements constituting the closed loop respectively, determining the diffusion range traversed when diffusion is carried out from the starting point element to each edge element; and determining the surface element corresponding to the closed loop based on the diffusion range traversed and the closed loop.

[0072] According to embodiments of this disclosure, for each closed loop, the point elements corresponding to the multiple edge elements constituting the closed loop are determined to form a reference point element set. A point element is arbitrarily selected from the reference point element set as the starting point element. Starting from the starting point element, a search is performed from the reference point element set towards the multiple edge elements constituting the closed loop. The target point element in the reference point element set that is closest to the starting point element is determined. The diffusion is performed from the starting point element to the multiple target point elements, and the diffusion range traversed from the starting point element to the target element is determined.

[0073] According to embodiments of this disclosure, the target point element is used as the starting point element, and the above diffusion process is repeated until the diffusion range expands to include all edge elements forming a closed loop, thus determining the current diffusion range. Based on the diffusion range and the closed loop, the surface element corresponding to the closed loop is determined.

[0074] According to embodiments of this disclosure, the surface elements are diffused from the starting element of the closed loop to each edge element. The surface elements are determined based on the diffusion range and the closed loop, making full use of the spatial distribution relationship of each edge element within the closed loop. Constructing surface elements through diffusion can more accurately fit the actual shape of the area enclosed by the closed loop, making the generated surface elements more consistent with the real structure of the 3D model and improving the accuracy of surface element construction.

[0075] According to embodiments of this disclosure, the 3D modeling method further includes: for a current closed loop, determining the distances between multiple edge elements constituting the current closed loop and surface elements constructed based on other closed loops; taking the surface element corresponding to the smallest distance among the multiple distances as the target surface element; and updating the positional relationship of the surface elements in space using the relationship between the surface elements constructed based on the current closed loop and the target surface element.

[0076] According to embodiments of this disclosure, for each face element, the positional relationships between multiple edge elements in the closed loop constituting that face element and face elements constructed based on other closed loops are determined, and the distance between the edge elements and face elements is determined based on the positional relationships. For example, if an edge element in the closed loop constituting the current face element participates in the construction of another closed loop, it can be determined that the positional relationship between the edge element constituting the current closed loop and the face element determined based on the other closed loop is intersecting or overlapping, and the distance between the edge element constituting the current closed loop and the face element is 0. Therefore, it can be determined that the positional relationship between the face element determined based on the current closed loop and the face element determined based on the other closed loop in space is adjacent.

[0077] According to embodiments of this disclosure, by determining the target surface element corresponding to the minimum distance between the current closed-loop edge element and other closed-loop surface elements, the spatial relationship between surface elements is updated, thereby strengthening the spatial association between different surface elements. This makes the connection between surface elements in the 3D model more consistent with actual spatial logic, reduces conflicts or unreasonable connections between surface elements, and enhances the overall coordination of the 3D model.

[0078] According to embodiments of this disclosure, a three-dimensional structure sequence generation model is used to generate a modeling sequence corresponding to the data features of the data to be modeled. This includes: predicting the next prediction data based on the current prediction result, and updating the current prediction result using the next prediction data. The current prediction result includes the data features and multiple next prediction data obtained by iterative prediction based on the data features. When making the first prediction, the data features are used as the current prediction result. If the next prediction data is empty, the encoding format of the current prediction result is verified. If the current prediction result passes the encoding format verification, the current prediction result is used as the modeling sequence.

[0079] According to embodiments of this disclosure, an autoregressive prediction method can be used to generate a modeling sequence based on data features. Specifically, based on the data features, a first predicted data point is obtained. The data features are then concatenated with the first predicted data point to form the current prediction result. Based on this current prediction result, the next predicted data point is obtained. If the next predicted data point is not empty, it is concatenated with the current prediction result, and the current prediction result is updated before continuing the above prediction process.

[0080] According to an embodiment of this disclosure, when the next predicted data is empty, the encoding format of the current prediction result is verified. The encoding format verification includes verifying the number of bits in the current prediction result according to a preset encoding rule to determine whether the current prediction result includes the encoding of complete multiple point elements.

[0081] According to embodiments of this disclosure, if the current prediction result passes the encoding format verification, the current prediction result is used as the modeling sequence. If the current prediction result fails the encoding format verification, it indicates that the current prediction process is erroneous. The current prediction result is then deleted, and the process of generating the modeling sequence described above is re-executed based on the data characteristics. Alternatively, other feature extraction methods can be used to extract features from the data to be modeled, and the process of generating the modeling sequence described above is re-executed based on the newly extracted data features.

[0082] According to embodiments of this disclosure, the next data point is iteratively predicted based on the current prediction result. A modeling sequence is determined by combining this prediction with encoding format verification. The modeling sequence is continuously optimized through iterative prediction to ensure it fully reflects the characteristics of the data to be modeled. Encoding format verification ensures the standardization of the modeling sequence, providing qualified input for subsequent decoding and improving the effectiveness and reliability of the generated modeling sequence.

[0083] According to embodiments of this disclosure, the data to be modeled includes a three-dimensional point cloud; the three-dimensional modeling method further includes: when it is determined that the modality of the input data input to the three-dimensional model sequence to generate the model is not a three-dimensional point cloud, converting the input data into a three-dimensional stereograph; and extracting the three-dimensional point cloud from the three-dimensional stereograph to obtain the data to be modeled.

[0084] According to embodiments of this disclosure, the 3D model sequence generation model can process input data of multiple modalities. After receiving input data input to the 3D model sequence generation model, the modality of the input data is determined. If it is determined that the input data is not in the form of a 3D point cloud, the input data is converted into a 3D point cloud to obtain the data to be modeled and re-input to the 3D model sequence generation model.

[0085] According to embodiments of this disclosure, in the process of converting input data into a 3D point cloud, the input data can first be converted into a 3D stereo image, and then point cloud extraction can be performed on the 3D stereo image to obtain the 3D point cloud, which is then used as the data to be modeled. Specifically, when converting input data into a 3D stereo image, different expert models for generating the 3D stereo image can be selected based on the different modalities of the input data.

[0086] For example, when the input data is in text form, a text-based image model can be invoked to generate a corresponding 3D stereoscopic image based on the text description of the input data. When the input data consists of multiple images of a 3D object from different angles, a multi-view stereoscopic vision model can be invoked to generate a 3D stereoscopic image of the 3D object based on the images of the 3D object from multiple angles.

[0087] According to embodiments of this disclosure, when the input data modality is not a 3D point cloud, it is converted into a 3D stereo image and the 3D point cloud is extracted as the data to be modeled, thus expanding the method's adaptability to different modal input data. Regardless of the form of the input data, it can be converted into a uniform 3D point cloud form for processing, improving the method's versatility and practicality.

[0088] According to embodiments of this disclosure, a three-dimensional structure sequence generation model is trained by: obtaining a sample geometric feature set of a sample three-dimensional model; encoding the sample geometric feature set using the sample three-dimensional structure sequence generation model to obtain a sample modeling sequence; determining a standard modeling sequence of the sample three-dimensional model based on the sample geometric feature set; and adjusting the model parameters of the sample three-dimensional structure sequence generation model based on the modeling loss determined by the sample modeling sequence and the standard modeling sequence to obtain a three-dimensional structure sequence generation model.

[0089] According to embodiments of this disclosure, the sample 3D model is in the same form as the target 3D model, i.e., a 3D model represented using B-rep. Based on the sample 3D model, sample point elements, sample edge elements, and sample surface elements are extracted to form a sample geometric element set.

[0090] According to embodiments of this disclosure, a modeling sequence for the sample is determined by generating a model using a sample three-dimensional structure sequence based on a sample geometric feature set.

[0091] According to embodiments of this disclosure, based on the sample edge element set in the sample geometric element set, the relationship between multiple sample point elements in the sample point element set is determined, and the correspondence between multiple sample point elements and multiple sample edge elements is determined based on the sample surface element set, thereby determining the sample point element features of each of the multiple sample point elements in the sample point element set. By encoding the sample point element features according to sample encoding rules, a standard modeling sequence can be obtained.

[0092] According to embodiments of this disclosure, a modeling loss is determined based on the reconstruction effect of the sample 3D model using the sample modeling sequence and the standard modeling sequence. Based on a comparison of the modeling loss and the loss threshold, it is determined whether the model parameters of the sample 3D structure sequence generation model need adjustment. If adjustment is necessary, the model parameters are optimized until the modeling loss converges, and a 3D structure sequence generation model is determined based on the current model parameters.

[0093] According to embodiments of this disclosure, a 3D structure sequence generation model is trained using a sample geometric feature set of a sample 3D model. Model parameters are adjusted by comparing the modeling loss between the sample modeling sequence and the standard modeling sequence, enabling the model to learn the modeling patterns found in the samples. The training process ensures the consistency between the model-generated modeling sequence and real-world modeling requirements, improves the accuracy of the generated modeling sequence, and provides a high-quality sequence foundation for subsequent 3D modeling.

[0094] According to embodiments of this disclosure, the modeling sequence of a sample is determined by generating a model based on a sample geometric feature set and a sample three-dimensional structure sequence, which can be achieved in the following way.

[0095] According to embodiments of this disclosure, the sample geometric feature set includes a sample point element set. Sample edge element set and sample face element set According to the manifold assumption, each edge constituting a 3D structure will be shared by one and only two faces, that is, each sample edge element in the sample edge element set will be shared by one and only two faces. It refers to exactly two edges of the sample face elements. For two sample face elements, the sample edge elements can be... Divided into two halves and This is to distinguish the correspondence between the sample edge element and the sample point element present in different sample surface elements.

[0096] For example, sample edge elements Sample surface elements and sample surface elements The edge, the sample edge element Split into half and Sample surface elements The side above is half of the side. Sample surface elements The side above is half of the side. .half of With sample surface elements Some sample points in the data have corresponding relationships, half of the data... With sample surface elements Some sample point elements in the diagram have corresponding relationships. By dividing the diagram into half-edges, a one-to-one mapping relationship between the half-edge and the sample surface element can be achieved, clarifying the attribution relationship between each half-edge and its corresponding sample surface element, and avoiding the ambiguity of the association relationship caused by constructing a mapping relationship between the same sample edge element and two sample surface elements. In addition, by representing the diagram using half-edges, it is still possible to clearly show that the two actual corresponding sample edge elements completely preserve the global association between the edge and the surface.

[0097] According to embodiments of this disclosure, for each sample surface element in the sample surface element set... In the parameter domain of the sample surface element The Volonoi diagram is constructed internally, which involves dividing the parameter domain of the sample surface element into multiple small-grained surfaces using the perpendicular bisectors of each edge. Specifically, the sample surface element is determined... Multiple edges Calculate sample surface elements The shortest distance from each point p to each edge of the sample face element. And according to the shortest distance criterion, the surface Divided into The regions are shown in formula (1):

[0098] (1)

[0099] in, Indicates the relationship with the edge Associated Voronoi Half Patch (VHP).

[0100] According to embodiments of this disclosure, the above-described partitioning process enables each sample surface element to be divided into multiple half-surfaces, and ensures that each sample point element in the sample surface element is partitioned into one half-surface, thus avoiding information overlap.

[0101] Figure 3 The illustration shows a schematic diagram of dividing sample surface elements into Volonoi segments according to a three-dimensional modeling method based on an embodiment of the present disclosure.

[0102] like Figure 3 As shown, the 3D model of the sample is divided according to the sample surface elements to obtain multiple sample surface elements. For each sample surface element, multiple edges of the sample surface element are determined. The distances of all sample point elements in the sample surface element to each edge are determined respectively. Each sample point element is then assigned to the edge with the smallest distance to it. The correspondence between the sample point elements and the edges is established, and the Volonoi segmentation region is determined based on the correspondence.

[0103] According to embodiments of this disclosure, during sequence generation using a sample 3D structure sequence generation model, the connection topology between multiple sample point elements and the geometric topology of the Volonoi segmentation region can be determined first based on the sample geometric feature set. The aforementioned topology can be encoded using a vector quantization variational autoencoder.

[0104] Specifically, when determining the connection topology between multiple sample point elements, the connection graph between the multiple sample point elements can be used. As input, the connection relationship encoder is used through formula (2). Extract the feature vector of each vertex, and fuse the low-level features of adjacent vertices into the learned embedding:

[0105] (2)

[0106] in, For the learned set of connection topology embeddings, for the th The topological embedding of a vertex is represented as: , for A vector space of dimension.

[0107] Similarly, when determining the geometric topology of the Volonoi segmentation region, a directed graph based on the Volonoi segmentation region can be used. As input, the geometric relationship encoder is used through formula (3). Geometric topological information in the local neighborhood is encoded based on the edge feature map attention mechanism:

[0108] (3)

[0109] in, For the learned set of geometric topological embeddings, for the th The set topological embedding of a set of sample point elements is represented as: .

[0110] After obtaining the connectivity topology and geometric topology, residual quantization can be used to further encode the topology and improve its expressive power.

[0111] Specifically, for the connection topology, different codebook sets are used. of A stack of codes, serving as the quantization code assigned to each embedding, for example, the ... The connection topology embedding of vertices It can be expressed in the form shown in formula (4):

[0112] (4)

[0113] in, For the first The topological representation of the connection of each sample point element Indicating the depth in the connected topology The quantization code at the location.

[0114] Similarly, for geometric topology, different codebook sets are used. of A stack of codes, serving as the quantization code assigned to each embedding, for example, the ... Topological embedding of a set of vertices It can be expressed in the form shown in formula (5):

[0115] (5)

[0116] in, For the first The geometric topological encoding representation of each vertex In geometric topology, depth The quantization code at the location.

[0117] Based on the encoding of the connection topology and the geometric topology described above, a model can be generated using the sample 3D structure sequence, and the sample modeling sequence can be determined. The required sample modeling sequence includes multiple placeholders: a start placeholder, an end placeholder, and a separator. The start placeholder is located at the beginning of the sample modeling sequence, and the end placeholder is located at the end, indicating the start and end of the sequence, respectively. The separator is used to divide the component geometric feature sets of multiple 3D components. The start placeholder can be set to... <start>The termination placeholder can be set to <end>The separator can be set to <sep>.

[0118] Based on the encoding of the connection topology and the encoding of the geometric topology, the positional relationships between sample point elements, sample edge elements and sample surface elements can be determined. Based on the above positional relationships, multiple sample 3D components that make up the sample 3D model can be further determined.

[0119] The sample geometric feature set is divided into multiple sample 3D components, resulting in sample component geometric feature sets corresponding to each sample 3D component. Following a first preset rule, the connection topology codes and geometric topology codes of each sample point element constituting the sample component geometric feature set are arranged according to the coordinate order of the sample point elements, resulting in multiple sample modeling subsequences. The first preset rule may include arranging the sample point elements in ascending order of coordinates.

[0120] According to the second preset rule, multiple sample modeling subsequences are arranged in the order of the coordinates of their first sample point elements. A separator is written between the multiple sample modeling subsequences after the arrangement. A start placeholder is written before the first sample modeling subsequence and a stop placeholder is written after the last sample modeling subsequence, thus obtaining the sample modeling sequence.

[0121] According to embodiments of this disclosure, after obtaining a sample modeling sequence using a sample 3D structure sequence, compliance checks can be performed on the sample modeling sequence. For example, it can be verified whether the coordinates of each sample point element in the sample modeling sequence exceed the parameter domain of the sample 3D model. If there are sample point elements exceeding the parameter domain, it can be determined that there is an error in the sample modeling sequence. Furthermore, it can be determined whether the number of data bits between the placeholders is an integer multiple of the number of encoding bits for the sample point elements. The number of encoding bits can be obtained by summing the number of bits for the coordinates of the sample point elements, the number of bits for the connection topology encoding, and the number of bits for the geometric topology encoding.

[0122] Figure 4 The illustration schematically depicts the process of generating a sample modeling sequence using a three-dimensional modeling method according to an embodiment of the present disclosure.

[0123] like Figure 4 As shown, by connecting the coordinates, the topology code, and the geometric topology code of each sample point element, the encoding result of each sample point element can be determined. After arranging multiple sample point elements according to the above sorting logic, the encoding results of each sample point element are arranged sequentially. A start placeholder is written before the first sample modeling subsequence, and an end placeholder is written after the last sample modeling subsequence, thus obtaining the sample modeling sequence.

[0124] exist Figure 4 In the code, the coordinates of the sample point element include 3 bits, corresponding to the 3-bit data of the Coord field; the encoding of the connection topology includes 4 bits, corresponding to the 4-bit data of the Connect field; and the encoding of the geometric topology includes 4 bits, corresponding to the 4-bit data of the VHP field.

[0125] According to embodiments of this disclosure, by using the inverse process of generating the sample modeling sequence as described above, the sample geometric feature set can be determined based on the sample modeling sequence. During the 3D reconstruction process based on the sample geometric feature set, the connection topology code and the geometric topology code can be decoded using formulas (6) and (7):

[0126] (6)

[0127] (7)

[0128] Formula (6) is used to decode the connection topology code. For the first The and the first The prediction results of the connection relationships between elements of each sample point. For connection relationship decoder, This indicates a splicing operation.

[0129] Formula (7) is used to decode geometric topology codes. For the first The and the first The geometric and topological information of the Volonoi segmentation region recovered from each sample point element. It is a geometric relation decoder.

[0130] Figure 5 The illustration schematically shows the process of performing connection topology encoding and geometric relationship encoding in a three-dimensional modeling method according to an embodiment of the present disclosure.

[0131] like Figure 5 As shown, for both connection topology and geometric topology, abstract relationships between sample point elements are obtained through graph neural networks and residual vector quantization, respectively. A connection relationship decoder then determines the connection relationships between sample point elements, represented by 0 / 1. A connection relationship of 0 indicates that the two sample point elements are not connected and there is no sample edge element, while a connection relationship of 1 indicates that the two sample point elements are connected and there is a sample edge element. Based on the predicted connection relationships in the decoding results and the actual connection relationships in the connection topology, the binary cross-entropy loss (BCE) is determined, resulting in the edge reconstruction loss.

[0132] The geometric relation decoder can determine the sample edge element corresponding to each sample point element. Based on the correspondence between the predicted sample edge element and the sample point element in the decoding result and the true correspondence between the sample edge element and the sample point element in the geometric topology, the mean squared error (MSE) between the two is determined, and the edge position loss and the face position loss are obtained.

[0133] Using the above formulas (6) and (7), the connection topology and geometric topology can be obtained. First, the connection relationship between sample point elements is determined according to the connection topology, thereby determining the sample edge elements. According to the geometric topology, for each half of a sample edge element, its next half is determined. Specifically, the next half of the current half can be determined using formula (8):

[0134] (8)

[0135] in, For the half-set, For the first The next half sample of the input half, For the first determined according to the connection topology Geometry of a curve extending halfway, The geometric distance is denoted as . Each sample edge element is divided into two directed halves, namely the outgoing half and the incoming half, which are in opposite directions. For each half, the half with the smallest distance is determined according to formula (8), and it is taken as the next half. The above operation is repeated until the latest determined next half points to the starting point of the first half, that is, the determined halfs form a closed loop.

[0136] In particular, when determining the half-side that is closest to a half-side according to the above formula (8), the determined next half-side cannot be a twin edge that matches that half-side.

[0137] Figure 6 The illustration shows a schematic diagram of a closed-loop construction method according to an embodiment of the present disclosure.

[0138] like Figure 6 As shown, for multiple sample edge elements on a sample face element, each sample edge element is divided into two directed half-edges with opposite directions. During the process of constructing a closed loop, a half-edge that does not form a closed loop is randomly selected, and the geometric distance between the endpoint of this half-edge and other half-edges is determined. From the other half-edges, the half-edge with the smallest geometric distance from this half-edge, and which is not a twin edge of this half-edge, is selected as the next half-edge of this half-edge. The above operation is repeated until a closed loop is formed.

[0139] Repeat the above operation until half of each sample edge element forms a closed loop. Based on these closed loops, use the N-boundary filling algorithm to generate sample surface elements. Specifically, based on the relationship between the closed loops, select outer and inner loops from the closed loops on the same plane, and first construct the sample surface elements corresponding to each of the outer loops.

[0140] For the inner ring, calculate the distances between the multiple sample edge elements constituting the inner ring and the multiple constructed sample surface elements, select the minimum distance, and determine the sample surface element corresponding to the minimum distance as the parent surface of the inner ring. After determining the parent surface, the inner ring can be filled to obtain sample surface elements, and the parent surface can be filled using the sample surface elements to complete the restoration of the 3D model.

[0141] According to embodiments of this disclosure, the modeling loss determined based on the sample modeling sequence and the standard modeling sequence is determined by: determining a geometric topology loss based on the sample modeling sequence and the standard modeling sequence, wherein the geometric topology loss represents the loss caused by the sample modeling sequence's incorrect prediction of the topological relationships between the sample geometric feature sets compared to the standard modeling sequence; determining a coding accuracy loss based on the sample coding rules applied when coding the sample geometric feature sets; and determining a modeling loss based on the geometric topology loss and the coding accuracy loss.

[0142] According to embodiments of this disclosure, geometric topology loss may include edge reconstruction loss caused by prediction errors of connectivity between point elements, edge position loss caused by prediction errors of positional relationships between edge elements, and face position loss caused by prediction errors of positional relationships of face elements in space.

[0143] According to embodiments of this disclosure, encoding precision loss can be used to represent the difference between the representational capability of the encoding result after encoding a sample geometric feature set using sample encoding rules and the expected representational capability required to accurately represent the sample geometric feature set.

[0144] According to embodiments of this disclosure, the modeling loss L can be determined using formula (9) based on the geometric topology loss and the coding precision loss:

[0145] (9)

[0146] Among them, L con L represents the edge reconstruction loss. line L represents the edge position loss. plane L represents the surface position loss. code This represents the loss of coding precision, where α, β, and λ are hyperparameters.

[0147] According to embodiments of this disclosure, during the process of adjusting the model parameters of the sample 3D structure sequence generation model based on the modeling loss, it can be first determined that the cause of the non-convergence of the modeling loss is geometric topology loss and / or encoding accuracy loss. If the geometric topology loss is large, the fitting ability of the sample 3D structure sequence generation model to the sample geometric feature set can be adjusted; if the encoding accuracy loss is large, the sample encoding rules applied by the sample 3D structure sequence generation model can be adjusted; if both the geometric topology loss and the encoding accuracy loss are large, the fitting ability and the sample encoding rules can be adjusted simultaneously.

[0148] According to embodiments of this disclosure, the modeling loss is determined by combining geometric topology loss and coding accuracy loss. Geometric topology loss focuses on the accuracy of topological relationship prediction, while coding accuracy loss focuses on the accuracy of coding rule application. This comprehensive loss function can comprehensively evaluate model performance, guiding the model to simultaneously optimize its ability to capture topological relationships and follow coding rules during training, further improving the modeling effect of the trained model.

[0149] Based on the above-described 3D modeling method, this disclosure also provides a 3D modeling apparatus. The following will be combined with... Figure 7 The device is described in detail.

[0150] Figure 7 A schematic block diagram of a three-dimensional modeling apparatus according to an embodiment of the present disclosure is shown.

[0151] like Figure 7 As shown, the 3D modeling device 700 of this embodiment includes a sequence generation module 710, a sequence decoding module 720, and a 3D reconstruction module 730.

[0152] The sequence generation module 710 is used to generate a model using a three-dimensional structural sequence, generating a modeling sequence corresponding to the data features of the data to be modeled. In one embodiment, the sequence generation module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0153] The sequence decoding module 720 is used to decode the modeling sequence and determine the model geometric element set of the target 3D model corresponding to the data to be modeled. The model geometric element set includes a set of point elements, a set of edge elements determined based on the set of point elements, and the correspondence between each point element and each edge element. In one embodiment, the sequence decoding module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0154] The 3D reconstruction module 730 is used to perform 3D reconstruction based on the model's geometric feature set to obtain the target 3D model. In one embodiment, the 3D reconstruction module 730 can be used to perform the operation S230 described above, which will not be repeated here.

[0155] According to embodiments of this disclosure, the sequence decoding module 720 includes a sequence segmentation submodule, a sequence decoding submodule, and a feature determination submodule.

[0156] The sequence segmentation submodule is used to segment the modeling sequence according to the delimiters in the modeling sequence to obtain multiple modeling subsequences, where each modeling subsequence corresponds one-to-one with a multiple 3D component.

[0157] The sequence decoding submodule is used to decode each modeling subsequence according to a preset encoding rule to obtain the component geometric element set of the 3D component corresponding to the modeling subsequence.

[0158] The Feature Determination submodule is used to determine the model's geometric feature set based on the individual component geometric feature sets of multiple 3D components.

[0159] According to embodiments of this disclosure, the sequence decoding submodule includes an encoding determination unit, a sequence segmentation unit, a feature determination unit, and an element determination unit.

[0160] The encoding determination unit is used to determine the encoding length and encoding composition of each point element according to the preset encoding rules.

[0161] The sequence segmentation unit is used to segment the modeling subsequence according to the encoding length to obtain the encoding of each point element.

[0162] The feature determination unit is used to parse the encoding based on the encoding structure for each point element to determine the point element features. The point element features include the position of the point element, the connection relationship between the point element and other point elements, and the edge elements corresponding to the point element. The edge elements are determined based on two interconnected point elements.

[0163] The element determination unit is used to determine the set of geometric elements of a component based on the individual point element characteristics of multiple point elements.

[0164] According to embodiments of this disclosure, the 3D reconstruction module 730 includes an edge element determination submodule, a loop closure determination submodule, a surface element construction submodule, and a 3D reconstruction submodule.

[0165] The edge element determination submodule is used to determine the edge element set based on the connectivity between multiple point elements included in the geometric feature set.

[0166] The closed-loop determination submodule is used to determine at least one closed loop consisting of multiple edge elements based on the positional relationships between multiple edge elements included in the edge element set, wherein the positional relationships are determined based on the distances between the multiple edge elements.

[0167] The surface element construction submodule is used to construct surface elements based on each closed loop.

[0168] The 3D reconstruction submodule is used to determine the target 3D model based on the positional relationship of multiple surface elements in space.

[0169] According to embodiments of this disclosure, the surface element construction submodule includes a point element selection unit, a point element diffusion unit, and a surface element determination unit.

[0170] The point element selection unit is used to arbitrarily select an edge element that constitutes the closed loop for each closed loop, and arbitrarily select a point element corresponding to the edge element as the starting point element.

[0171] The point element diffusion unit is used to diffuse from the starting point element to the multiple edge elements that constitute the closed loop, through the point elements corresponding to the multiple edge elements that constitute the closed loop, and to determine the diffusion range traversed when diffusing from the starting point element to each edge element.

[0172] The surface element determination unit is used to determine the surface element corresponding to the closed loop based on the diffusion range and the closed loop.

[0173] According to embodiments of this disclosure, the 3D modeling apparatus 700 further includes a distance determination module, a surface element determination module, and a position update module.

[0174] The distance determination module is used to determine the distances between the multiple edge elements constituting the current closed loop and the face elements constructed based on other closed loops, for the current closed loop.

[0175] The surface element determination module is used to select the surface element that corresponds to the minimum distance among multiple distances as the target surface element.

[0176] The position update module is used to update the positional relationship of face elements in space by utilizing the relationship between the face elements constructed based on the current closed loop and the target face elements.

[0177] According to embodiments of this disclosure, the sequence generation module 710 includes a sequence prediction submodule, a result verification submodule, and a sequence determination submodule.

[0178] The sequence prediction submodule is used to predict the next prediction data based on the current prediction result, and update the current prediction result using the next prediction data. The current prediction result includes data features and multiple next prediction data obtained by iterative prediction based on the data features. When making the first prediction, the data features are used as the current prediction result.

[0179] The result verification submodule is used to verify the encoding format of the current prediction result when the next prediction data is empty.

[0180] The sequence determination submodule is used to use the current prediction result as the modeling sequence if the current prediction result is verified by the encoding format.

[0181] According to embodiments of this disclosure, the 3D modeling apparatus 700 further includes a data conversion module and a data determination module.

[0182] The data conversion module is used to convert input data into a 3D stereo image when it is determined that the modality of the input data to the 3D model sequence is not a 3D point cloud.

[0183] The data determination module is used to extract the 3D point cloud from the 3D stereo image to obtain the data to be modeled.

[0184] According to embodiments of this disclosure, the 3D modeling apparatus 700 further includes a sample element acquisition module, a sample sequence generation module, a standard sequence determination module, and a model adjustment module.

[0185] The sample feature acquisition module is used to acquire the sample geometric feature set of the sample 3D model.

[0186] The sample sequence generation module is used to encode the sample geometric element set using the sample 3D structure sequence generation model to obtain the sample modeling sequence.

[0187] The standard sequence determination module is used to determine the standard modeling sequence of the sample 3D model based on the sample geometric feature set.

[0188] The model adjustment module is used to adjust the model parameters of the sample 3D structure sequence generation model based on the modeling loss determined by the sample modeling sequence and the standard modeling sequence, so as to obtain the 3D structure sequence generation model.

[0189] According to embodiments of this disclosure, the 3D modeling apparatus 700 further includes a geometric loss determination module, an encoding loss determination module, and a modeling loss determination module.

[0190] The geometric loss determination module is used to determine the geometric topology loss based on the sample modeling sequence and the standard modeling sequence. The geometric topology loss represents the loss caused by the error in predicting the topological relationship between the sample geometric feature set compared with the standard modeling sequence.

[0191] The encoding loss determination module is used to determine the encoding accuracy loss based on the sample encoding rules applied when encoding the sample geometric feature set.

[0192] The modeling loss determination module is used to determine the modeling loss based on the geometric topology loss and the coding accuracy loss.

[0193] According to embodiments of this disclosure, any plurality of modules among the sequence generation module 710, sequence decoding module 720, and 3D reconstruction module 730 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the sequence generation module 710, sequence decoding module 720, and 3D reconstruction module 730 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the sequence generation module 710, sequence decoding module 720, and 3D reconstruction module 730 can be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0194] Figure 8 A block diagram schematically illustrates an electronic device suitable for implementing a three-dimensional modeling method according to an embodiment of the present disclosure.

[0195] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0196] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0197] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0198] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0199] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0200] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0201] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0202] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0203] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0204] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0205] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0206] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0207] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.< / sep> < / end> < / start>

Claims

1. A three-dimensional modeling method, characterized in that, The method includes: A three-dimensional structure sequence is used to generate a model, which generates a modeling sequence corresponding to the data features of the data to be modeled. The modeling sequence is decoded to determine the model geometric element set of the target 3D model corresponding to the data to be modeled. The model geometric element set includes a point element set, an edge element set determined based on the point element set, and the correspondence between each point element and each edge element. The target 3D model is obtained by performing 3D reconstruction based on the set of geometric elements of the model.

2. The method according to claim 1, characterized in that, The target 3D model includes multiple 3D components, and the model geometric feature set includes the component geometric feature sets of each of the multiple 3D components. Decoding the modeling sequence to determine the model geometric feature set of the target 3D model corresponding to the data to be modeled includes: The modeling sequence is divided according to the delimiter in the modeling sequence to obtain multiple modeling sub-sequences, wherein each of the multiple modeling sub-sequences corresponds one-to-one with a multiple of the three-dimensional components; For each of the modeling subsequences, the modeling subsequences are decoded according to a preset encoding rule to obtain a set of component geometric elements of the 3D component corresponding to the modeling subsequence; and The model geometric feature set is determined based on the component geometric feature sets of each of the multiple 3D components.

3. The method according to claim 2, characterized in that, For each modeling subsequence, the modeling subsequence is decoded according to a preset encoding rule to obtain a set of component geometric elements of the 3D component corresponding to the modeling subsequence, including: According to the preset encoding rules, the encoding length of each point element and the encoding composition of each point element are determined; The modeling subsequence is segmented according to the encoding length to obtain the encoding of each of the multiple point elements; For the encoding of each point element, the encoding is parsed based on the encoding structure to determine the point element features, wherein the point element features include the position of the point element, the connection relationship between the point element and other point elements, and the edge element corresponding to the point element, wherein the edge element is determined based on two interconnected point elements; The component geometric feature set is determined based on the point element characteristics of each of the multiple point elements.

4. The method according to claim 3, characterized in that, The step of obtaining the target 3D model by performing 3D reconstruction based on the geometric feature set includes: Based on the connectivity relationships between the multiple point elements included in the geometric element set, determine the edge element set determined based on the point element set; Based on the positional relationships between the multiple edge elements included in the edge element set, at least one closed loop composed of the multiple edge elements is determined, wherein the positional relationships are determined based on the distance between the multiple edge elements. For each of the closed loops, surface elements are constructed based on the closed loop; and The target 3D model is determined based on the spatial positional relationship of the multiple surface elements.

5. The method according to claim 4, characterized in that, For each of the closed loops, the construction of surface elements based on the closed loops includes: For each closed loop, arbitrarily select an edge element that constitutes the closed loop, and arbitrarily select a point element corresponding to the edge element as the starting point element; Starting from the initial element, the diffusion proceeds through the point elements corresponding to the edge elements constituting the closed loop, and then diffuses towards each of the edge elements constituting the closed loop, determining the diffusion range traversed when diffusing from the initial element to each edge element; and Based on the diffusion range and the closed loop, determine the surface element corresponding to the closed loop.

6. The method according to claim 4, characterized in that, The method further includes: For the current closed loop, determine the distances between the multiple edge elements constituting the current closed loop and the face elements constructed based on other closed loops; The surface element corresponding to the minimum distance among the plurality of said distances is taken as the target surface element; and The spatial positional relationship of the surface elements is updated by utilizing the relationship between the surface elements constructed based on the current closed loop and the target surface element.

7. The method according to claim 1, characterized in that, The method of generating a model using a three-dimensional structure sequence, based on the data features of the data to be modeled, generates a modeling sequence corresponding to the data features, including: Based on the current prediction result, the next prediction data is predicted, and the current prediction result is updated using the next prediction data. The current prediction result includes the data features and multiple next prediction data obtained by iterative prediction based on the data features. When making the first prediction, the data features are used as the current prediction result. If the next predicted data is empty, the encoding format of the current prediction result is verified; and If the current prediction result passes the encoding format verification, the current prediction result is used as the modeling sequence.

8. The method according to claim 1, characterized in that, The data to be modeled includes 3D point clouds; The method further includes: If it is determined that the modality of the input data to the generated model of the 3D model sequence is not a 3D point cloud, the input data is converted into a 3D stereo image; and Extract the 3D point cloud from the 3D solid image to obtain the data to be modeled.

9. The method according to claim 1, characterized in that, The three-dimensional structure sequence generation model was trained in the following way: Obtain the set of sample geometric features of the 3D model of the sample; The sample geometric element set is encoded using a sample 3D structure sequence generation model to obtain a sample modeling sequence; Based on the set of geometric features of the sample, determine the standard modeling sequence of the three-dimensional model of the sample; as well as Based on the modeling loss determined according to the sample modeling sequence and the standard modeling sequence, the model parameters of the sample 3D structure sequence generation model are adjusted to obtain the 3D structure sequence generation model.

10. The method according to claim 9, characterized in that, The modeling loss determined based on the sample modeling sequence and the standard modeling sequence is determined in the following way: Based on the sample modeling sequence and the standard modeling sequence, a geometric topology loss is determined, wherein the geometric topology loss represents the loss caused by the sample modeling sequence's incorrect prediction of the topological relationships between sample geometric feature sets compared to the standard modeling sequence; Based on the sample encoding rules applied when encoding the sample geometric feature set, determine the encoding accuracy loss; and The modeling loss is determined based on the geometric topology loss and the encoding accuracy loss.

Images