Three-dimensional model segmentation method, device, computing device and computer-readable storage medium

Abstract

The embodiments of the present invention relate to the field of computer vision technology, and disclose a three-dimensional model segmentation method, apparatus, computing device, and computer-readable storage medium. The method includes: obtaining mesh information of a three-dimensional model; dividing the three-dimensional model into K geometric units based on the mesh information, each geometric unit being formed by clustering N adjacent meshes that meet preset geometric conditions, and the preset geometric conditions are set according to the geometric features of the meshes; extracting the primary visual features of each geometric unit; and fusing adjacent geometric units with the same primary visual features into a collection unit. In the above manner, the embodiments of the present invention reduce the computational complexity of the processor and the amount of computation of the processor by dividing the geometric units and fusing adjacent geometric units with the same primary visual features into a collection unit, while not affecting the mapping effect of the three-dimensional model and maintaining an effect that is basically consistent with the real object.

Description

[0001] This application is a divisional application of the patent application with application number 202210428745.2, application date April 22, 2022, and name “Three-dimensional model segmentation method, device, computing device and readable storage medium”. Technical Field

[0002] The embodiments of the present invention relate to the field of computer vision technology, and specifically to a three-dimensional model segmentation method, apparatus, computing device, and computer-readable storage medium. Background Art

[0003] A digital twin is a virtual mirror of the real world created through computer technology to achieve the integration and interaction of humans, machines, and objects. It serves as a crucial bridge and infrastructure connecting the physical world with the virtual digital world. Digital twin technology digitizes physical entities to simulate, analyze, and emulate the real world. With the rise and continuous development of emerging technologies such as 5G, the Internet of Things, big data, and cloud computing, the establishment of digital twin systems has become a crucial scientific driving force for development. They can be widely applied in smart cities, security, smart transportation, surveying and mapping, smart homes, urban planning, and other fields, possessing immense economic and social value.

[0004] Building a 3D model of the real world is a fundamental element in constructing a digital twin system. With the advancement of 3D reconstruction technology, it's now possible to generate a 3D model of a real object from multiple images and apply the texture information from the images to each triangular mesh in the 3D model, achieving texture mapping. However, the basic geometric unit of such a 3D model is a triangular mesh, which lacks semantic or high-level geometric information. Furthermore, the 3D model segmentation process is complex and computationally intensive, making subsequent editing of the 3D model difficult for users.

[0005] Therefore, it is necessary to provide a three-dimensional model segmentation method, device and computer-readable storage medium to overcome the above problems. Summary of the Invention

[0006] In view of the above problems, embodiments of the present invention provide a three-dimensional model segmentation method, apparatus, computing device, and readable storage medium to solve the problem of high complexity in three-dimensional model segmentation in the prior art.

[0007] According to a first aspect of an embodiment of the present invention, a three-dimensional model segmentation method is provided, the method comprising:

[0008] Get the mesh information of the 3D model;

[0009] Dividing the three-dimensional model into K geometric units according to the grid information, each of the geometric units is formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set according to geometric features of the grids;

[0010] Extracting the main visual features of each of the geometric units;

[0011] Adjacent geometric units having the same main visual features are fused into a collection unit.

[0012] In some embodiments, dividing the three-dimensional model into K geometric units according to the grid information, each of the geometric units being formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set according to the geometric features of the grids, further includes:

[0013] Selecting one of the candidate grids as a seed grid;

[0014] Clustering the adjacent grids that meet the preset geometric conditions into the same geometric unit, wherein the preset geometric conditions include a first preset condition, wherein the first preset condition is that the distance between the grid and the seed grid is less than or equal to a preset distance, and at least one grid in the geometric unit is adjacent to the seed grid;

[0015] The grids that are not clustered into the geometric unit and are not used as the seed grids are taken as the candidate grids, and the step of selecting one of the grids from the candidate grids as the seed grid is performed until all the grids are clustered into the geometric unit.

[0016] In some embodiments, the adjacent grids that satisfy the preset geometric condition are clustered into the same geometric unit, and the preset geometric condition also includes a second preset condition, and the second preset condition is that the angle between the normal vector of the grid and the normal vector of the seed grid is less than or equal to a preset angle.

[0017] In some embodiments, extracting the primary visual feature of each of the geometric units further includes:

[0018] Taking the average visual feature vector of all the grids in each geometric unit as the visual feature vector of the geometric unit;

[0019] The main visual feature of the geometric unit is calculated according to the visual feature vector.

[0020] In some embodiments, the visual feature vector is a color vector, and calculating the main visual feature of the geometric unit according to the visual feature vector further includes:

[0021] The color vectors of the K geometric units are used as sample points x of the mean shift algorithm i , calculate the offset mean s of each sample point i , ,in, , is x i and x j The distance, x j is x i Neighbor point, a j The area of ​​the geometric unit corresponding to each sample point;

[0022] Cluster the sample points after the shift to obtain the main colors c of K geometric units k .

[0023] In some embodiments, fusing adjacent geometric units having the same primary visual feature into a collective unit further comprises:

[0024] Calculating the main visual feature probability of each of the geometric units;

[0025] Assigning a main visual feature label corresponding to each of the geometric units according to the main visual feature probability;

[0026] Adjacent geometric units with the same main visual feature label are fused into a collection unit.

[0027] In some embodiments, the primary visual feature is a primary color, and fusing adjacent geometric units having the same primary visual feature into a collective unit further comprises:

[0028] Optimize the geometric unit to obtain the Markov field model

[0029] ,in, , represents the probability that each geometric unit belongs to each primary color, c k is the main color of the geometric unit, , w ij is the inner product of the plane normal vectors corresponding to the two geometric units;

[0030] Solving the Markov field model using a graph segmentation algorithm, and assigning a primary color label corresponding to each of the geometric units;

[0031] Adjacent geometric units with the same primary color label are merged into a collective unit.

[0032] According to a second aspect of an embodiment of the present invention, a three-dimensional model segmentation device is provided, the device comprising:

[0033] An acquisition module is used to obtain the mesh information of the three-dimensional model;

[0034] a segmentation module, configured to divide the three-dimensional model into K geometric units based on the grid information, each of the geometric units being formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set based on the geometric features of the grids;

[0035] An extraction module, configured to extract the main visual features of each of the geometric units;

[0036] A fusion module is used to fuse adjacent geometric units with the same main visual features into a collection unit.

[0037] According to a third aspect of an embodiment of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0038] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform any one of the operations of the three-dimensional model segmentation method described above.

[0039] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, characterized in that the storage medium stores at least one executable instruction, and the executable instruction executes the operation of any one of the above-mentioned three-dimensional model segmentation methods when run.

[0040] The present invention divides a 3D model into K geometric units, thereby dividing the model into different parts represented by different geometric units. The geometric feature parameters represented by these parts have interpretable physical meanings and contain semantic information. In this case, the K geometric units are the initial segmentation results. Each geometric unit is formed by clustering N adjacent grids that meet preset geometric conditions, so that the same geometric unit is composed of at least two or more adjacent grids, that is, N ≥ 2, so that the number of grids M is greater than the number of geometric units K. Therefore, the number of geometric units is reduced compared to the number of grids, reducing computational complexity and the amount of operations.

[0041] Furthermore, by fusing adjacent geometric units with the same primary visual features into aggregate units, the 3D model is further divided into fewer parts, further reducing the computational complexity and workload of the processor while maintaining the mapping effect of the 3D model, essentially maintaining consistency with the real object. Furthermore, when visual features need to be edited or replaced, there's no need to re-operate on the initialized mesh; operations can be performed on the fused aggregate units. This reduces repetitive processor operations, facilitates user operations, and improves editing efficiency. Furthermore, this 3D modeling method can also be applied to other types of objects, demonstrating its versatility.

[0042] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0044] Figure 1 A schematic diagram showing a flow chart of a three-dimensional model segmentation method provided by an embodiment of the present invention is shown;

[0045] Figure 2 A schematic flow chart of step 120 of a three-dimensional model segmentation method provided by some embodiments of the present invention is shown;

[0046] Figure 3 A schematic flow chart of step 130 of a three-dimensional model segmentation method provided by some embodiments of the present invention is shown;

[0047] Figure 4 A schematic flow chart of step 140 of a three-dimensional model segmentation method provided by some embodiments of the present invention is shown;

[0048] Figure 5 A schematic structural diagram of a three-dimensional model segmentation device provided by an embodiment of the present invention is shown;

[0049] Figure 6 shows a schematic structural diagram of a segmentation module provided in some embodiments of the present invention;

[0050] Figure 7 shows a schematic structural diagram of an extraction module provided in some embodiments of the present invention;

[0051] Figure 8 shows a schematic structural diagram of a fusion module provided in some embodiments of the present invention;

[0052] Figure 9 A schematic structural diagram of a computing device provided by some embodiments of the present invention is shown. DETAILED DESCRIPTION

[0053] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0054] To address the high complexity and computational complexity of existing 3D model segmentation methods, the inventors discovered that existing methods typically perform texture mapping directly on each triangular mesh initially segmented from the original 3D model. This requires the processor to perform mapping calculations for each triangular mesh, resulting in high computational complexity and a high computational load on the processor. Taking sneakers as an example, if you want to replace the material of a sneaker, you first need to segment the corresponding 3D model into the initial triangular meshes. Then, for each triangular mesh, you need to perform the corresponding material replacement. The processor then needs to perform material parameter and boundary calculations for each triangular mesh, requiring a large amount of computation and being computationally complex. Furthermore, this segmentation method does not include semantic information about each part, such as the upper, sole, and vamp, which have physical meanings. This makes model editing operations extremely difficult and labor-intensive.

[0055] The inventors also found that the existing three-dimensional model segmentation methods are mainly divided into two categories: learning-based and non-learning-based frameworks. (1) The learning-based method extracts the feature vector of each triangular mesh through artificial or neural networks, and then trains a classifier to predict the semantic label of each triangular mesh. However, this type of method relies on a large amount of training data, and requires manual annotation of multiple three-dimensional models for training for each category of objects, which has extremely high annotation costs. In addition, this type of method has weak generalization ability and is not universal. When applied to new categories of objects, it is impossible to perform correct semantic segmentation of new categories of objects, resulting in duplication of work. (2) The non-learning-based unsupervised method extracts the geometric features of each triangular mesh and uses a clustering method to cluster triangular meshes belonging to the same part into one category. Although this type of method does not require data annotation, it requires manual setting of the number of clusters K. Since the number of parts segmented for each category of objects is different, this method is also not universal and is not conducive to segmenting all objects.

[0056] Therefore, how to divide the three-dimensional model into parts with physical meaning plays an important role in subsequent model editing.

[0057] The inventor provides a three-dimensional model segmentation method, which clusters an initial large number of grids into a relatively smaller number of geometric units according to geometric conditions, and then fuses the geometric units according to visual features such as color features or texture features of the original three-dimensional model to form collective units. At this time, the boundaries of the collective units are larger, and the number is further smaller than the number of geometric units. When subsequent color feature or texture feature mapping is performed, the processor can directly perform mapping calculations for the collective units. Since the number of collective units is small, the calculation complexity can be greatly reduced, and the amount of processor calculation can be reduced. At the same time, this three-dimensional model method can also be applied to other categories of objects and is universal.

[0058] Figure 1 A flowchart of a three-dimensional model segmentation method provided by an embodiment of the present invention is shown. The method is executed by a computing device, which may include one or more processors. The processors may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement an embodiment of the present invention, without limitation herein. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs, without limitation herein.

[0059] like Figure 1 As shown, the method includes the following steps:

[0060] Step 110: Obtain mesh information of the three-dimensional model.

[0061] Step 120: Divide into K geometric units, each geometric unit is formed by clustering N adjacent grids that meet preset geometric conditions, and K and N are both natural numbers greater than 0.

[0062] Step 130: Extract the main visual features of each geometric unit.

[0063] Step 140: Fuse adjacent geometric units with the same primary visual features into a collection unit.

[0064] In step 110, the 3D model can be a 3D model corresponding to a real object generated directly on a computing device based on multiple images, 3D data information, or other methods. In this case, the mesh information is generated on the computing device. Alternatively, 3D model data generated by another device or existing 3D model data can be input into the computing device.

[0065] The mesh information obtained by the computing device may be generated by the computing device, or the mesh information may be generated by the computing device after the three-dimensional model data is input into the computing device, or the mesh information may be included in the input three-dimensional model data, and the computing device may directly obtain it from the three-dimensional model data.

[0066] Among them, the mesh generated according to the three-dimensional model is a polygonal mesh, and each mesh is composed of vertices and edges according to the topological relationship. The mesh can be a triangular mesh, a quadrilateral mesh, or other polygonal mesh. The value of the number of meshes M is usually more than 10,000, depending on the size and shape of the three-dimensional model. Mesh information includes parameters such as points, lines, surfaces, curvature, normal vectors, angles, flatness, distances, and visual features. In this embodiment, taking sneakers as an example, the number of meshes in the three-dimensional model of the sneakers is about 100,000, then the M value corresponds to the value of the number of meshes, and the mesh is a triangular mesh. In other embodiments, if the number of meshes in the three-dimensional model is one million, ten million, or other values, then the M value corresponds to one million, ten million, or other values, and the mesh can also be a quadrilateral mesh, or other polygonal mesh, which is not limited here.

[0067] In step 120, the parameters of the geometric features of different parts of the three-dimensional model are usually different. The three-dimensional model is divided into K geometric units according to the geometric feature information of M grids, so as to divide the three-dimensional model into different parts represented by different geometric units. The geometric feature parameters represented by these parts have interpretable physical meanings and contain semantic information. At this time, the K geometric units are the preliminary segmentation results.

[0068] Each geometric unit is formed by clustering N adjacent grids that meet preset geometric conditions, so that the grids in the same geometric unit meet the preset geometric conditions, and the grids in different geometric units do not meet the preset geometric conditions with each other, such as the grids in one of the geometric units do not meet the preset geometric conditions with the grids in any other geometric units, or even if the grids in one of the geometric units meet the preset geometric conditions with the grids in any other geometric units, the grids in the different geometric units are not adjacent, and thus cannot be clustered to form the same geometric unit. Among them, the same geometric unit is composed of at least two or more adjacent grids, that is, N ≥ 2, so that M is greater than K. Therefore, compared with the number of grids, the number of geometric units is reduced, reducing the computational complexity of the processor and reducing the amount of computation of the processor. Among them, when N is greater than or equal to 10 on average, the number of K is correspondingly less than one order of magnitude of M. When N is greater than or equal to 100 on average, the corresponding number of K is less than two orders of magnitude of M, and so on, which will not be repeated here. Different geometric condition judgments will produce different geometric unit division effects, and the corresponding number of grids in the same geometric unit is also different. If only one grid meets the preset geometric conditions and no adjacent grids meet the preset geometric conditions, that is, when N=1, since the area of ​​the surface where the grid is located is very small, it can be ignored without affecting the overall segmentation result and can also reduce the amount of computation.

[0069] Among them, the preset geometric conditions are parameters set according to geometric features, which include curvature, normal vector, angle, flatness, distance, etc. Correspondingly, the geometric conditions are parameters for clustering the grid according to aspects such as curvature, normal vector, angle, flatness, distance, etc. The geometric conditions can be set according to one of the curvature, normal vector, angle, flatness, and distance, or according to two or more of the curvature, normal vector, angle, flatness, and distance. The more judgment conditions there are, the fewer the number of corresponding geometric units. The preset geometric conditions can be set as needed and are not limited here. In the process of geometric unit segmentation, the K geometric units are automatically determined by the preset geometric conditions without manual definition, thus achieving preliminary automatic segmentation.

[0070] In step 130, since real objects usually have visual features such as color, texture, or brightness, in subsequent editing, it is necessary to perform corresponding mapping operations on the three-dimensional model to achieve the mapping of the visual features of the real object. The geometric units that have been initially segmented may have multiple visual features, such as different colors and different textures. In order to further fuse the geometric units, it is necessary to unify the visual features and extract the main visual features of each geometric unit. The main visual features are based on most of the main visual features of each grid in the geometric unit to provide a fusion basis for the subsequent fusion of adjacent geometric units, so that the fused geometric units can achieve the same effect as the visual features of the real object in the subsequent visual feature editing process, so as to be robust.

[0071] For example, in a geometric unit, if the main visual feature of most grids is color, then the main color feature of the geometric unit is extracted. If the main color feature is red, then the extracted main color feature is red; if the main visual feature of most grids is texture feature, then the main texture feature of the geometric unit is extracted, where the texture can be a combination of different colors in a certain pattern, or a texture in the form of a picture, etc., then the main texture feature of the geometric unit is displayed as a texture feature of different color combinations or features in the form of a picture, and so on.

[0072] In step 140, after extracting the main visual features of the geometric units, the main visual feature labels corresponding to the geometric units are assigned to fuse adjacent geometric units with the same main visual features into a collection unit. In other words, adjacent geometric units with the same main visual features can be directly fused into a collection unit, or adjacent geometric units with similar main visual features but with the same main visual features after algorithm optimization processing can be fused into a collection unit. The final result is that adjacent geometric units with the same main visual features are fused into a collection unit. At this time, the three-dimensional model is further divided into fewer parts to further reduce the computational complexity of the processor and reduce the amount of calculation of the processor, while not affecting the mapping effect of the three-dimensional model and maintaining an effect that is basically consistent with the real object. In addition, when it is necessary to edit and replace the visual features, there is no need to re-operate the initialized grid. It is only necessary to operate on the multiple collection units after fusion, which reduces the repeated operations of the processor, facilitates user operations, and improves editing efficiency.

[0073] By dividing the 3D model into K geometric units, the model is divided into different parts represented by different geometric units. The geometric feature parameters represented by these parts have interpretable physical meaning and contain semantic information. At this time, the K geometric units are the initial segmentation results. Each geometric unit is composed of N adjacent grids that meet preset geometric conditions. The same geometric unit is composed of at least two or more adjacent grids, that is, N ≥ 2, making M greater than K. Therefore, the number of geometric units is reduced compared to the number of grids, reducing computational complexity and the amount of operations.

[0074] Furthermore, by fusing adjacent geometric units with the same primary visual features into aggregate units, the 3D model is further divided into smaller parts, further reducing the computational complexity and workload of the processor while maintaining the mapping effect of the 3D model, essentially maintaining consistency with the real object. Furthermore, when visual features need to be edited or replaced, there's no need to re-operate on the initialized mesh; operations can be performed on the fused aggregate units. This reduces repetitive processor operations, facilitates user operations, and improves editing efficiency. This 3D modeling method can also be applied to other types of objects, demonstrating its versatility.

[0075] In some embodiments, as Figure 2 As shown, step 120 further includes:

[0076] Step a01: Select one of the candidate grids as a seed grid.

[0077] Step a02: clustering adjacent grids that meet preset geometric conditions into the same geometric unit, where the preset geometric conditions include a first preset condition, where the distance from the grid to the seed grid is less than or equal to a preset distance, and at least one grid in the geometric unit is adjacent to the seed grid.

[0078] Step a03: Grids that are not clustered into geometric units and are not used as seed grids are considered as candidate grids, and the process proceeds to the step of selecting one of the candidate grids as a seed grid, until all grids are clustered into geometric units.

[0079] Steps a01 to a03 are implemented using a region growing algorithm. For example, all initial grids are considered as candidate grids for the first time. One of these candidate grids is selected as the first seed grid. The remaining grids outside the seed grid are then traversed sequentially to determine whether they meet preset geometric conditions. Adjacent grids around the seed grid that meet the preset geometric conditions are clustered around the seed grid until no grids meeting the preset geometric conditions are included. This forms the first geometric unit. Grids that are not clustered into the first geometric unit and not used as the first seed grid are then considered as candidate grids. One of these candidate grids is selected as the second seed grid. The previous traversal and determination of whether the preset geometric conditions are met are repeated to form the second geometric unit. This iterative calculation continues until the Kth geometric unit is formed, where each geometric unit is formed by the grids within it.

[0080] Among them, the area of ​​each grid that meets the above-mentioned first preset condition is very small, and the surface of each grid of the three-dimensional model can be considered to be a plane. Therefore, by judging whether the distance from each grid to the seed grid is less than or equal to the preset distance, the grids whose distance to the seed grid is less than or equal to the preset distance are clustered with the seed grid into a geometric unit, so that the planes of the grids in the same geometric unit do not deviate from the preset distance and are consistent within the limit range of the preset distance, so that the division of each geometric unit is more accurate and more physically meaningful.

[0081] Preferably, the preset distance ranges from 5-10 mm, and the preset angle ranges from 10-20°, so that the characteristics of each mesh within the same geometric unit are highly consistent. In this embodiment, the number of meshes M in the three-dimensional model of the sneaker is approximately 100,000. The preset distance is 5 mm, and the preset angle is 15°. This results in a relatively small number of geometric units, and good consistency among the meshes within the same geometric unit. Ultimately, based on the preset geometric conditions of this embodiment, the number of geometric units K of the sneaker is determined to be several hundred. This number of geometric units K is three orders of magnitude lower than the number of meshes M, significantly reducing the computational complexity and workload of the processor.

[0082] In some embodiments, the preset geometric condition further includes a second preset condition, where the second preset condition is that the angle between the normal vector of the grid and the normal vector of the seed grid is less than or equal to a preset angle.

[0083] By determining whether the angle between the normal vector of each grid and the normal vector of the corresponding surface of the seed grid is less than or equal to a preset angle, the grids whose normal vectors have an angle less than or equal to the preset angle with the normal vector of the corresponding surface of the seed grid and the seed grid are clustered into a geometric unit, so that the normal vectors of the planes where the grids in the same geometric unit are located are consistent within the limit range of the preset angle, making the division of each geometric unit more accurate and more physically meaningful, and through the restrictions of the first preset condition and the second preset condition, the number of geometric units is less than the number restricted by only the first preset condition, thereby further reducing the computational complexity of the processor and reducing the amount of computation of the processor.

[0084] In some embodiments, as Figure 3 As shown, step 130 further includes:

[0085] Step b01: taking the average visual feature vector of all grids in each geometric unit as the visual feature vector of the geometric unit.

[0086] Step b02: Calculate the main visual features of the geometric unit based on the visual feature vector.

[0087] Since different parts of an object may have similar visual features, each geometric unit is clustered according to the visual features of the initial 3D model. All meshes of each geometric unit have visual features. To calculate the main visual features of the geometric unit, it is necessary to calculate the visual feature vector of the geometric unit. This is calculated based on the average visual feature vector of all meshes of the geometric unit. Through vector calculation, the main visual features can be quantified to realize the main visual feature extraction of the geometric unit.

[0088] In some embodiments, the visual feature is a color feature, and step b02 further includes:

[0089] Step b021: Use the color vectors of K geometric units as sample points x for the mean shift algorithm i , calculate the offset mean s of each sample point i , ,in, , is x i and x j The distance, x j is x i Neighbor point, a j The area of ​​the geometric unit corresponding to each sample point.

[0090] Step b022: Cluster the sample points after the shift to obtain the main colors c of K geometric units k .

[0091] Usually, the visual features of objects are expressed as colors. The corresponding average color vector is calculated based on all the grids in each geometric unit as the color vector of the geometric unit, so as to facilitate the calculation of the mean shift algorithm through the color vector. Among them, the color vector is calculated using the RGB vector, which is expressed as [rgb], such as red: [1 0 0], green: [0 1 0], blue: [0 0 1]. At this time, the distance between sample points with similar color vectors becomes smaller after the offset, and the distance between sample points with different color vectors becomes larger after the offset. Repeat the above process until the offset distance of each sample point is small enough. Finally, the sample point set after the offset is clustered to obtain K main colors c k , the main color vector closest to each geometric unit is used as its corresponding main color c k .

[0092] In step b021, x j is x i The neighbor point of x is one of the geometric units. i , another geometric unit connected to the boundary of this geometric unit is x j ;w j is the weight of each neighborhood point. σ is the kernel band. In this embodiment, σ is 0.1. In other embodiments, σ ​​may also be 0.2 or 0.3, or other values, which are set as needed and are not limited here.

[0093] Since the number of geometric units that the object is finally clustered into according to the preset geometric conditions cannot be judged and the number of clusters K cannot be determined in advance, the mean shift algorithm is used to extract the main visual features appearing in the 3D model to maintain the automation of the main visual feature extraction process without manual intervention.

[0094] In some embodiments, as Figure 4 As shown, step 140 further includes:

[0095] Step c01: Calculate the main visual feature probability of each geometric unit.

[0096] Step c02: assigning a main visual feature label corresponding to each geometric unit according to the main visual feature probability.

[0097] Step c03: Fuse adjacent geometric units with the same main visual feature label into a collection unit.

[0098] Among them, the main visual feature probability of each geometric unit is calculated through step c01 to optimize the main visual features of the geometric units, so that the main visual features of different geometric units that meet the requirements can be subsequently optimized into the same main visual feature to prepare for fusion.

[0099] Step c02 is used to make a judgment based on the main visual feature probability in c01 to obtain the main visual feature of the optimized geometric unit, and to assign the main visual feature label corresponding to the geometric unit so as to perform a fusion operation based on the main visual feature label later.

[0100] Step c03 fuses adjacent geometric units with the same primary visual feature label into a single aggregate unit, which serves as the model segmentation result. For example, the number of aggregate units obtained by fusion before optimization in step c01 is X1, while the number of aggregate units obtained by optimization after steps c01 to c03 is X2. X2 is smaller than X1, further reducing the number of aggregate units, thereby further reducing the computational complexity and workload of the processor, and further increasing the processing speed of the processor.

[0101] In some embodiments, the primary visual feature is a primary color, and step 140 further includes:

[0102] Step c04: Optimize the geometric units to obtain the Markov field model

[0103] ,in,

[0104] , represents the probability that each geometric unit belongs to each primary color, c k is the main color of the geometric unit,

[0105] , w ij is the inner product of the plane normal vectors corresponding to the two geometric units.

[0106] Step c05: Solve the Markov field model using a graph segmentation algorithm and assign a corresponding primary color label to each geometric unit.

[0107] Step c06: Merge adjacent geometric units with the same primary color label into a collective unit.

[0108] In step c04, the main color of each geometric unit is optimized and calculated using a Markov field model to optimize the main colors of different geometric units. For example, the main colors of different geometric units and the plane normal vectors of different geometric units are calculated using the Markov field model, so that the main colors of different geometric units that meet the requirements can be subsequently optimized to the same main color.

[0109] l i and l jIndicates the primary color label to which the i-th and j-th geometric units belong. In this embodiment, λ is set to 0.5. In other embodiments, λ may also be set to other values ​​as needed and is not limited here.

[0110] Then, step c05 uses the graph segmentation algorithm to solve the Markov field model, obtain the main color corresponding to each geometric unit after optimization, and solve to obtain l i and l j , assigning each geometric unit a corresponding primary color label. Finally, step c06 fuses adjacent geometric units with the same primary color label into a single aggregate unit, which serves as the model segmentation result. At this point, the number of aggregate units after fusion is smaller than the K geometric units, further reducing the number of aggregate units, thereby further reducing the computational complexity and workload of the processor, and further improving the processing speed of the processor. In the sneaker model of this embodiment, after steps c04 to c06, the number of aggregate units obtained is approximately dozens, which is four orders of magnitude smaller than the initial number of grids. Accordingly, the computational complexity and workload of the processor are greatly reduced, and the processing speed of the processor is greatly improved.

[0111] Figure 5 FIG. 2 shows a schematic diagram of the structure of a 3D model segmentation device provided by an embodiment of the present invention. The device 200 includes:

[0112] The acquisition module 210 is used to acquire mesh information of the three-dimensional model.

[0113] The segmentation module 220 is used to divide the three-dimensional model into K geometric units according to the grid information. Each geometric unit is formed by clustering N adjacent grids that meet preset geometric conditions. K and N are both natural numbers greater than 0.

[0114] The extraction module 230 is used to extract the main visual features of each geometric unit.

[0115] The fusion module 240 is used to fuse adjacent geometric units with the same main visual features into a collection unit.

[0116] In some embodiments, as Figure 6 As shown, the segmentation module 220 includes:

[0117] A first selection module 221 is configured to select one of the candidate grids as a seed grid;

[0118] The first clustering module 222 is used to cluster adjacent grids that meet preset conditions into the same geometric unit. The preset conditions include a first preset condition, which is that the distance from the grid to the seed grid is less than or equal to the preset distance, and at least one grid in the geometric unit is adjacent to the seed grid.

[0119] The second selection module 223 is used to select meshes that are not clustered into geometric units and are not used as seed meshes as candidate meshes, and proceed to the step of selecting one of the candidate meshes as a seed mesh until all meshes are clustered into geometric units.

[0120] In some embodiments, as Figure 7 As shown, the extraction module 230 includes:

[0121] The first calculation module 231 is configured to use the average visual feature vector of all grids in each geometric unit as the visual feature vector of the geometric unit.

[0122] The second calculation module 232 is configured to calculate the main visual features of the geometric unit according to the visual feature vector.

[0123] In some embodiments, the second calculation module 232 further includes:

[0124] The offset module 2321 is used to use the color vectors of K geometric units as sample points x of the mean shift algorithm. i , calculate the offset mean s of each sample point i , ,in, , is x i and x j The distance, x j is x i Neighbor point, a j The area of ​​the geometric unit corresponding to each sample point.

[0125] The third clustering module 2322 is used to cluster the sample points after the shift to obtain the main color c of K geometric units. k .

[0126] In some embodiments, Figure 8 As shown, the fusion module 240 further includes:

[0127] The first optimization module 241 is used to calculate the probability of the main visual feature of each geometric unit.

[0128] The first assigning module 242 is configured to assign a main visual feature label corresponding to each geometric unit according to the main visual feature probability.

[0129] The fourth clustering module 243 is configured to fuse adjacent geometric units having the same main visual feature label into a cluster unit.

[0130] In some embodiments, the fusion module 240 further includes:

[0131] The second optimization module 244 is used to optimize the geometric unit to obtain the Markov field model

[0132] ,in,

[0133] , represents the probability that each geometric unit belongs to each primary color, c k is the main color of the geometric unit,

[0134] , w ij is the inner product of the plane normal vectors corresponding to the two geometric units.

[0135] The second assigning module 245 is used to solve the Markov field model using a graph segmentation algorithm and assign a corresponding primary color label to each geometric unit.

[0136] The fifth clustering module 246 is configured to fuse adjacent geometric units having the same primary color label into a cluster unit.

[0137] Figure 9 The schematic diagram of the structure of the computing device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0138] like Figure 9 As shown, the computing device may include: a processor 302 , a communications interface 304 , a memory 306 , and a communication bus 308 .

[0139] Processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 408. Communication interface 304 is used to communicate with other devices, such as clients or other server network elements. Processor 302 is used to execute program 310, which may specifically perform the steps described in the aforementioned embodiment of the method for 3D model segmentation.

[0140] Specifically, the program 310 may include program code including computer-executable instructions.

[0141] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a computing device may be of the same type, such as one or more CPUs, or different types, such as one or more CPUs and one or more ASICs.

[0142] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, or may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0143] An embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one executable instruction, and the executable instruction, when run, executes the operation of any one of the above-mentioned three-dimensional model segmentation methods.

[0144] The algorithm or demonstration provided herein are not inherently relevant to any particular computer, virtual system or other equipment. Various general-purpose systems may also be used together with the teachings based on this. According to the above description, it is apparent that the structure required for constructing this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0145] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0146] Similarly, it should be understood that in order to streamline the present invention and facilitate understanding of one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0147] Those skilled in the art will appreciate that the modules in the devices of the embodiments can be adaptively modified and installed in one or more devices different from the embodiments. The modules, units, or components in the embodiments can be combined into a single module, unit, or component, or they can be divided into multiple sub-modules, sub-units, or sub-components. All features disclosed in this specification (including the accompanying claims, abstract, and drawings), and all processes or units of any method or device disclosed therein, can be combined in any combination, unless at least some of such features and / or processes or units are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0148] It should be noted that the above embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by a person skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A three-dimensional model segmentation method, characterized in that: The method comprises: Acquire mesh information of a three-dimensional model, wherein the mesh information is geometric feature information, and the geometric feature information includes distance and normal vector; Dividing the three-dimensional model into K geometric units according to the grid information, each of the geometric units is formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set according to geometric features of the grids; Extracting the main visual features of each of the geometric units; Merging adjacent geometric units having the same main visual feature into a collective unit; The method further includes dividing the three-dimensional model into K geometric units according to the grid information, each of the geometric units being formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set according to geometric features of the grids. Selecting one of the candidate grids as a seed grid; Clustering the adjacent grids that meet the preset geometric conditions into the same geometric unit, wherein the preset geometric conditions include a first preset condition, wherein the first preset condition is that the distance between the grid and the seed grid is less than or equal to a preset distance, and at least one grid in the geometric unit is adjacent to the seed grid; The grids that are not clustered into the geometric unit and are not used as the seed grids are used as the candidate grids, and the process proceeds to the step of selecting one of the candidate grids as the seed grid, until all the grids are clustered into the geometric unit; The adjacent grids that meet the preset geometric conditions are clustered into the same geometric unit, and the preset geometric conditions also include a second preset condition, which is that the angle between the normal vector of the grid and the normal vector of the seed grid is less than or equal to a preset angle.

2. The three-dimensional model segmentation method according to claim 1, characterized in that: The extracting of the main visual features of each of the geometric units further includes: Taking the average visual feature vector of all the grids in each geometric unit as the visual feature vector of the geometric unit; The main visual feature of the geometric unit is calculated according to the visual feature vector.

3. The method according to claim 2, characterized in that The visual feature vector is a color vector, and calculating the main visual feature of the geometric unit according to the visual feature vector further includes: The color vectors of the K geometric units are used as sample points x of the mean shift algorithm i , calculate the offset mean s of each sample point i , ,in, , is x i and x j The distance, x j is x i Neighbor point, a j The area of ​​the geometric unit corresponding to each sample point; Cluster the sample points after the shift to obtain the main colors c of K geometric units k .

4. The three-dimensional model segmentation method according to claim 1, characterized in that: The step of fusing adjacent geometric units having the same primary visual feature into a collection unit further comprises: Calculating the main visual feature probability of each of the geometric units; Assigning a main visual feature label corresponding to each of the geometric units according to the main visual feature probability; Adjacent geometric units with the same main visual feature label are fused into a collection unit.

5. The 3D model segmentation method according to claim 1, wherein: The main visual feature is a main color, and the step of fusing adjacent geometric units having the same main visual feature into a collective unit further includes: Optimize the geometric unit to obtain the Markov field model ,in, , represents the probability that each geometric unit belongs to each primary color, c k is the main color of the geometric unit, , w ij is the inner product of the plane normal vectors corresponding to the two geometric units, l i and l j Indicates the primary color label to which the i-th and j-th geometric units belong; Solving the Markov field model using a graph segmentation algorithm, and assigning a primary color label corresponding to each of the geometric units; Adjacent geometric units with the same primary color label are merged into a collective unit.

6. A three-dimensional model segmentation device, characterized in that: The device comprises: An acquisition module, configured to acquire mesh information of a three-dimensional model, wherein the mesh information is geometric feature information including distance and normal vector; a segmentation module, configured to divide the three-dimensional model into K geometric units based on the grid information, each of the geometric units being formed by clustering N adjacent grids that meet preset geometric conditions, where K and N are both natural numbers greater than 0, and the preset geometric conditions are set based on the geometric features of the grids; An extraction module, configured to extract the main visual features of each of the geometric units; A fusion module, configured to fuse adjacent geometric units having the same primary visual feature into a collection unit; The segmentation module is further configured to: Selecting one of the candidate grids as a seed grid; Clustering the adjacent grids that meet the preset geometric conditions into the same geometric unit, wherein the preset geometric conditions include a first preset condition, wherein the first preset condition is that the distance between the grid and the seed grid is less than or equal to a preset distance, and at least one grid in the geometric unit is adjacent to the seed grid; The grids that are not clustered into the geometric unit and are not used as the seed grids are used as the candidate grids, and the process proceeds to the step of selecting one of the candidate grids as the seed grid, until all the grids are clustered into the geometric unit; The adjacent grids that meet the preset geometric conditions are clustered into the same geometric unit, and the preset geometric conditions also include a second preset condition, which is that the angle between the normal vector of the grid and the normal vector of the seed grid is less than or equal to a preset angle.

7. A computing device, characterized in that include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the three-dimensional model segmentation method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction, and the executable instruction executes the operation of the three-dimensional model segmentation method according to any one of claims 1 to 5 when running.

Images