A mesh model simplification method, device and related equipment

By using multiple iterations and visual difference calculations in the simplification process of mesh model, the problem of poor visual effects after simplification of mesh model in the prior art is solved, and a higher quality rendering effect is achieved.

CN117974939BActive Publication Date: 2024-11-12北京数原数字化城市研究中心
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410141066.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-31
Publication Date
2024-11-12
Estimated Expiration
2044-01-31

AI Technical Summary

Technical Problem

The prior art can easily lead to unnecessary topological errors in the simplified process of grid model, resulting in incoherent lines of the simplified model, holes appear or details are lost, affecting the visual effect after rendering.

Method used

By acquiring multiple rendered images of the initial mesh model under different camera poses, performing multiple iterations simplification, using pre-trained neural network models to calculate the visual differences between the rendered images, determining effective iteration simplification, avoiding invalid iterations, and ensuring simplification The visual effect of the model is close to the original model.

Benefits of technology

It effectively reduces unnecessary topological errors in the simplification process of mesh model, improves the visual effect of the rendered images of the simplified model, and ensures the retention of user visual preferences and the integrity of details.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117974939B_ABST
    Figure CN117974939B_ABST
Patent Text Reader

Abstract

The present disclosure provides a mesh model simplification method, device and related equipment, which relate to the technical fields of computer graphics and computer vision, wherein the method comprises: obtaining N first rendering images of an initial mesh model under N camera poses, where N is an integer greater than or equal to 1; performing multiple iterative simplifications on the initial mesh model according to the N first rendering images to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, and the visual difference between the N second rendering images of the effective iterative simplification and the N first rendering images is less than or equal to a first preset threshold, and the visual difference is obtained based on a pre-trained neural network model. The present invention can reduce or avoid unnecessary topological errors in the mesh model simplification process, and improve the visual effect of the final target mesh model after rendering.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of computer vision and computer graphics, and in particular to a grid model simplification method, device and related equipment. Background Art

[0002] For complex mesh models that include a large number of triangular facets, the storage space and rendering burden of the mesh model can be significantly reduced by reducing the number of vertices and polygons in the mesh model through model simplification technology.

[0003] It is found in the application that when the mesh model is simplified based on the relevant technology, unnecessary topological errors will occur in the mesh model. On the one hand, the above unnecessary topological errors may cause the lines of the simplified model to be incoherent, such as the appearance of holes or the breakage of the connection; on the other hand, the above unnecessary topological errors may also cause the loss of details of the simplified model, such as the disappearance of sharp edges or corners.

[0004] That is to say, the mesh model simplified based on the relevant technology has a poor visual effect after rendering. Summary of the invention

[0005] The purpose of the disclosed invention is to provide a mesh model simplification method, device and related equipment, which are used to solve the technical problem that the mesh model obtained after model simplification processing in the related technology has poor visual effect after model rendering.

[0006] In a first aspect, the present disclosure provides a mesh model simplification method, the method comprising:

[0007] Obtain N first rendered images of the initial mesh model at N camera poses, where N is an integer greater than or equal to 1;

[0008] According to the N first rendered images, the initial mesh model is iteratively simplified for multiple times to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, the visual difference between the N second rendered images obtained by effective iterative simplification and the N first rendered images is less than or equal to a first preset threshold, and the N second rendered images obtained by effective iterative simplification correspond one-to-one to the N camera poses.

[0009] In a second aspect, the present disclosure provides a mesh model simplification device, the device comprising:

[0010] An acquisition module, used to acquire N first rendered images of the initial mesh model under N camera poses, where N is an integer greater than or equal to 1;

[0011] A model simplification module is used to perform multiple iterative simplifications on the initial mesh model according to the N first rendered images to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, the visual difference between the N second rendered images obtained by the effective iterative simplifications and the N first rendered images is less than or equal to a first preset threshold, the visual difference is obtained based on a pre-trained neural network model, and the N second rendered images obtained by the effective iterative simplifications correspond one-to-one to the N camera poses.

[0012] In a third aspect, the present disclosure provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect.

[0013] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0014] In a fifth aspect, the present disclosure provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0015] In the present disclosure, the initial mesh model is iteratively simplified multiple times based on multiple rendered images of the initial mesh model under different camera poses, and the final target mesh model is obtained based on the effective iterative simplifications in the multiple iterative simplifications. At the same time, it is defined that the visual difference between the N second rendered images of an iteration and the N first rendered images is less than or equal to a first preset threshold as an effective iterative simplification, so as to reduce or avoid unnecessary topological errors in the mesh model simplification process by iteratively comparing the difference in visual effects of the mesh model before and after simplification, so as to improve the visual effect of the rendered image of the target mesh model finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a grid model simplification method provided by an embodiment of the present disclosure;

[0017] Figure 2 is a flowchart of another grid model simplification method provided by an embodiment of the present disclosure;

[0018] Figure 3 is a structural schematic diagram of a grid model simplification device provided by an embodiment of the present disclosure;

[0019] Figure 4 It is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0021] The present disclosure provides a model simplification method. Figure 1 As shown, the mesh model simplification method comprises:

[0022] Step 101: Obtain N first rendered images of the initial mesh model under N camera poses.

[0023] Wherein, N is an integer greater than or equal to 1.

[0024] The above initial mesh model is a three-dimensional model including a plurality of facets (also called triangular facets), for example, a mesh model composed of a plurality of triangular facets.

[0025] The N first rendered images correspond one-to-one to the N camera poses, wherein the first rendered image may be a two-dimensional image obtained by rendering the initial mesh model based on the pose parameters of the corresponding camera pose, and the first rendered image may also be a two-dimensional image of the target object captured by an image capturing device (such as a camera) at the corresponding camera pose, and the initial mesh model is a three-dimensional data representation of the target object. For example, the target object may be a church, a bedroom, a statue, etc.

[0026] Step 102: According to the N first rendered images, the initial mesh model is iteratively simplified multiple times to obtain a target mesh model.

[0027] Among them, the target mesh model is a mesh model obtained based on the effective iterative simplification in the multiple iterative simplifications, the visual difference between the N second rendered images of the effective iterative simplification and the N first rendered images is less than or equal to a first preset threshold, the visual difference is obtained based on a pre-trained neural network model, and the N second rendered images of the effective iterative simplification correspond one-to-one to the N camera poses.

[0028] In the multiple iterative simplifications, if the visual difference between the N second rendered images of an iteration and the N first rendered images is less than or equal to a first preset threshold, then the iteration is considered to be a valid iterative simplification; if the visual difference between the N second rendered images of an iteration and the N first rendered images is greater than the first preset threshold, then the iteration is considered to be an invalid iterative simplification.

[0029] In the present disclosure, an iteration action includes: performing model simplification processing on the mesh model to be simplified in this iteration to obtain a simplified model of this iteration, and performing image rendering on the simplified model of this iteration based on the aforementioned N camera poses to obtain N second rendered images of this iteration.

[0030] When an iteration is judged as a valid iterative simplification, the simplified model of this iteration will be used as the mesh model to be simplified or the target mesh model for the next iteration; and when an iteration is judged as an invalid iterative simplification, the iterative action will be rolled back, that is, the mesh model to be simplified of this iteration will be used as the mesh model to be simplified or the target mesh model for the next iteration.

[0031] In the present disclosure, the initial mesh model is iteratively simplified multiple times based on multiple rendered images of the initial mesh model under different camera poses, and the final target mesh model is obtained based on the effective iterative simplifications in the multiple iterative simplifications. At the same time, it is defined that the visual difference between the N second rendered images of an iteration and the N first rendered images is less than or equal to a first preset threshold as an effective iterative simplification, so as to reduce or avoid unnecessary topological errors in the mesh model simplification process by iteratively comparing the difference in visual effects of the mesh model before and after simplification, so as to improve the visual effect of the rendered image of the target mesh model finally obtained.

[0032] It should be understood that the specific value of the first preset threshold can be adaptively selected according to actual needs, and the present disclosure does not limit the specific value of the first preset threshold.

[0033] Exemplarily, the target mesh model obtained by the present disclosure can be used in a game scene or a virtual reality scene.

[0034] In one embodiment, the visual difference is used to indicate the difference in images perceived by human vision.

[0035] In this embodiment, whether to retain or delete the model simplification operation of the current iteration is determined by the image difference perceived by human vision, so that the image performance effects of the rendered image of the target mesh model after model simplification and the rendered image of the initial mesh model before model simplification in the user's vision are close.

[0036] In one embodiment, step 101 includes:

[0037] Get the initial mesh model;

[0038] The initial mesh model is respectively rendered based on the preset N camera pose parameters to obtain N first rendered images.

[0039] When the initial mesh model is obtained in advance, the image rendering of the initial mesh model is performed using preset camera pose parameters to generate a first rendered image corresponding to the camera pose parameters, thereby completing the rendering image acquisition operation for the existing three-dimensional model. In this embodiment, the initial mesh model can be a three-dimensional model independently modeled by the user, a three-dimensional model commissioned by a third party, or a three-dimensional model downloaded or imported through an open source channel.

[0040] Exemplarily, a renderer such as PyRender or Open3D may be used to perform the aforementioned image rendering operation.

[0041] In one embodiment, obtaining N first rendered images of the initial mesh model under N camera poses includes:

[0042] Obtain N first rendered images of the target object under N camera poses;

[0043] The N first rendered images are geometrically reconstructed to obtain an initial mesh model indicating the target object.

[0044] In this embodiment, the first rendered image should be understood as a real-life image of the target object at the corresponding camera pose captured by the image capture device. At this time, after obtaining N first rendered images, the three-dimensional model of the target object (i.e., the target mesh model) is obtained by using the two-dimensional image of the target object at each camera pose through geometric reconstruction, and the three-dimensional model acquisition operation of the existing two-dimensional image is completed to adapt to the model simplification process of the user in the physical modeling process. Exemplarily, the geometric reconstruction scheme can use a reconstruction scheme based on NeRF2Mesh, and obtain network training to achieve geometric reconstruction by inputting N first rendered images, and finally obtain the target mesh model.

[0045] In one embodiment, step 102 includes:

[0046] Simplifying the mesh model to be simplified in the nth iteration based on the nth first rendering image to obtain a simplified model in the nth iteration, wherein n is a positive integer less than or equal to N, and the mesh model to be simplified in the first iteration is the initial mesh model;

[0047] Comparing visual differences between the N first rendered images and the N second rendered images of the simplified model of the nth iteration to obtain a visual difference value of the nth iteration;

[0048] When the visual difference value of the nth iteration is less than or equal to the first preset threshold, the simplified model of the nth iteration is used as the mesh model to be simplified in the (n+1)th iteration, or the simplified model of the nth iteration is used as the target mesh model;

[0049] When the visual difference value of the nth iteration is greater than the first preset threshold, the mesh model to be simplified in the nth iteration is used as the mesh model to be simplified in the (n+1)th iteration, or the mesh model to be simplified in the nth iteration is used as the target mesh model.

[0050] The N second rendered images of the simplified model of the nth iteration correspond one to one with the N camera poses.

[0051] In one example, when N first rendered images are obtained by respectively rendering an initial mesh model based on N camera pose parameters, in order to ensure the consistency of camera pose between the first rendered image and the second rendered image, after obtaining the simplified model of the nth iteration, the simplified model of the nth iteration is further rendered using the aforementioned N camera pose parameters to obtain N second rendered images of the simplified model of the nth iteration.

[0052] In another example, when the N first rendered images are the aforementioned real scene images, in order to ensure the consistency of camera pose between the first rendered image and the second rendered image, after obtaining the simplified model of the nth iteration, the camera pose estimation can be performed on the N first rendered images respectively to obtain the camera pose parameters corresponding to each first rendered image, and then based on the camera pose parameters corresponding to each first rendered image, the simplified model of the nth iteration is rendered respectively to obtain N second rendered images of the simplified model of the nth iteration.

[0053] Exemplarily, the process of obtaining the visual difference value of the nth iteration may be:

[0054] Based on the correspondence between the first rendered image and the camera pose and the correspondence between the second rendered image and the camera pose, reorganize the N first rendered images and the N second rendered images of the simplified model of the nth iteration into N image groups, the N image groups and the N camera poses correspond one to one, and each image group includes a first rendered image and a second rendered image corresponding to the same camera pose;

[0055] Calculating the visual difference between the first rendered image and the second rendered image in each of the image groups to obtain a group difference value for each of the image groups;

[0056] The sum of the group difference values ​​of each image group in the multiple image groups is used as the visual difference value of the nth iteration.

[0057] It should be noted that, when n is less than N, and when the visual difference value of the nth iteration is less than or equal to the first preset threshold, the simplified model of the nth iteration is used as the mesh model to be simplified in the (n+1)th iteration;

[0058] When n=N, and when the visual difference value of the nth iteration is less than or equal to the first preset threshold, the simplified model of the nth iteration is used as the target mesh model;

[0059] Similarly, when n is less than N, and when the visual difference value of the nth iteration is greater than the first preset threshold, the mesh model to be simplified of the nth iteration is used as the mesh model to be simplified of the (n+1)th iteration;

[0060] When n=N, and when the visual difference value of the nth iteration is greater than the first preset threshold, the mesh model to be simplified of the nth iteration is used as the target mesh model.

[0061] In one embodiment, the step of performing model simplification on the mesh model to be simplified of the nth iteration based on the nth first rendered image to obtain the simplified model of the nth iteration includes:

[0062] Clustering multiple normal graphs of multiple facets included in the nth first rendered image to obtain M of the nth first rendered image n Group normal cluster, M n is a positive integer;

[0063] According to the M n The group normal cluster iteratively deletes edges of the mesh model to be simplified in the nth iteration to obtain a simplified model in the nth iteration.

[0064] In the nth first rendering image, one face patch corresponds to one normal map, and the normal map is a result of projecting the normal vector of the corresponding face patch onto the 2D camera plane.

[0065] In this embodiment, multiple normal graphs are clustered to obtain several groups of normal clusters, that is, multiple sets of similar faces are obtained. Then, through iterative edge deletion, at least two similar faces corresponding to each normal cluster are merged to adapt to the prior knowledge of the corresponding artificial plane (that is, multiple faces of the artificial plane can be represented by a smaller number of faces), thereby achieving efficient simplification of the artificial plane and further reducing the number of faces of the target mesh model.

[0066] Exemplarily, a mean shift method may be applied to complete a clustering operation on a plurality of normal maps of a plurality of facets included in the nth first rendered image.

[0067] In one embodiment, before performing multiple iterations of simplification on the initial mesh model according to the N first rendered images to obtain the target mesh model, the method further includes:

[0068] Based on N preset camera pose parameters, respectively performing normal map rendering on the initial mesh model to obtain a normal map corresponding to each of the first rendered images, wherein the N camera pose parameters correspond one-to-one to the N camera poses;

[0069] or,

[0070] Normal prediction is performed on each of the N first rendered images to obtain a normal map corresponding to each of the first rendered images.

[0071] When the N first rendered images are the aforementioned real-scene images, and the initial mesh model is obtained by geometrically reconstructing the N first rendered images, normal prediction is performed on each of the N first rendered images to skip the rendering operation of the renderer and directly obtain multiple normal maps of multiple facets included in each of the first rendered images, thereby accelerating the efficiency of model simplification.

[0072] Exemplarily, the above normal prediction operation can be completed based on a preset large vision model (LVM) Ominidata model.

[0073] When N first rendered images are obtained by respectively rendering the initial mesh model based on the preset N camera pose parameters, the renderer can be controlled to respectively render the normal map of the initial mesh model based on the preset N camera pose parameters to obtain multiple normal maps of the multiple facets included in each of the first rendered images.

[0074] In one embodiment, the M n The group normal cluster iteratively deletes edges of the mesh model to be simplified in the nth iteration to obtain a simplified model in the nth iteration, including:

[0075] Use the normal vector value of the cluster center of the mth group of normal clusters to replace the remaining normal vector values ​​of the mth group, and obtain the normal clustering graph of the mth group of normal clusters, where m is less than or equal to M. n integer;

[0076] Perform edge detection on the normal clustering graph of the m-th group of normal clusters to determine the edge contour of the m-th hyperplane;

[0077] In the mesh model to be deleted in the mth iteration, the face edge located in the edge contour of the mth hyperplane is deleted to obtain the deleted model for the mth iteration, and the mesh model to be deleted in the first deletion in the nth iteration is the mesh model to be simplified in the nth iteration;

[0078] Comparing visual differences between the N first rendered images and N second rendered images of the edge erasure model of the m-th edge erasure, to obtain a visual difference value of the m-th edge erasure;

[0079] When the visual difference value of the m-th edge deletion is less than or equal to the first preset threshold, the edge deletion model of the m-th edge deletion is used as the mesh model to be deleted for the (m+1)-th edge deletion, or the edge deletion model of the m-th edge deletion is used as the simplified model for the n-th iteration;

[0080] When the visual difference value of the m-th edge deletion is greater than the first preset threshold, the mesh model to be deleted of the m-th edge deletion is used as the mesh model to be deleted of the m+1-th edge deletion, or the mesh model to be deleted of the m-th edge deletion is used as the simplified model of the n-th iteration.

[0081] The N second rendered images of the edge-erased model of the m-th edge-erased process correspond one to one with the N camera poses.

[0082] It should be noted that, based on the normal vector value of the cluster center of the mth group of normal clusters, after replacing multiple normal vector values ​​of multiple normal graphs of the mth group of normal clusters, the mth group of normal clusters can be understood as a hyperplane, and the normal of the hyperplane is the cluster center of the mth group of normal clusters. At this time, edge detection is performed on the mth group of normal clusters using an edge detection operator (such as the Sobel operator) to determine the edge contour of the mth hyperplane.

[0083] The process of obtaining the visual difference value of the m-th border erasure may refer to the description of the process of obtaining the visual difference value of the n-th iteration in the above embodiment, which will not be described again to avoid repetition.

[0084] It should be noted that when m is less than M n , and when the visual difference value of the m-th edge deletion is less than or equal to the first preset threshold, the edge deletion model of the m-th edge deletion is used as the mesh model to be deleted for the (m+1)-th edge deletion;

[0085] When m=M n , and when the visual difference value of the m-th edge deletion is less than or equal to the first preset threshold, the edge deletion model of the m-th edge deletion is used as the simplified model of the n-th iteration;

[0086] When m is less than M n , and when the visual difference value of the m-th edge deletion is greater than the first preset threshold, the mesh model to be deleted of the m-th edge deletion is used as the mesh model to be deleted of the (m+1)-th edge deletion;

[0087] When m=M n , and when the visual difference value of the m-th edge deletion is greater than the first preset threshold, the mesh model to be deleted of the m-th edge deletion is used as the simplified model of the n-th iteration.

[0088] The visual difference between the N second rendered images simplified in each of the multiple iterative simplifications and the N first rendered images is obtained by calculating the Learned Perceptual Image Patch Similarity (LPIPS) based on a preset visual macro model.

[0089] The aforementioned group difference value or visual difference value is the LPIPS value output by the visual large model after the corresponding first rendered image and the corresponding second rendered image are input into the medium visual large model.

[0090] In some optional embodiments, after obtaining the visual large model, model training samples can be constructed in a targeted manner to enhance the LPIPS computing capability of the obtained visual large model in model simplification applications by fine-tuning the model, wherein the positive samples in the model training samples are: a rendered image before a simplified artificial plane, and a rendered image after a simplified artificial plane; the negative samples in the model training samples are: a rendered image before a simplified non-artificial plane, and a rendered image after a non-simplified artificial plane.

[0091] For ease of understanding, the following examples are provided:

[0092] Figure 2 is a flow chart of a model simplification method provided by the present disclosure, wherein the method is as follows Figure 2 As shown:

[0093] Step 1: Obtain an initial mesh model, and record the initial mesh model as M.

[0094] Step 2: Use the renderer to render M under N camera positions and obtain the rendering result of M with texture mapping (i.e. Figure 2, that is, to obtain N rendering images (which can be understood as the aforementioned N first rendering images), and the N rendering images form a set I = {I1, I2, I3, ..., I N}, the N camera pose parameters corresponding to the N camera poses form a set T = {T1, T2, T3, ..., T N}.

[0095] Step 3: Use the above renderer to render and obtain N normal map sets corresponding to N renderings. The normal map set of each rendering is recorded as K n , for K n Use the mean shift method to perform clustering and get K n The corresponding M n clusters, and based on K n The corresponding M n The normal graph of the cluster center of each cluster in the clusters determines K n The corresponding M n The hyperplane indicated by each cluster in the clusters (i.e. Figure 2 , which obtains the operation of the plane area.

[0096] Step 4: Delete K n The edge in the mth hyperplane (i.e. Figure 2 After deleting the edge, update K n The data structure of the mesh model to be deleted corresponding to the mth edge deletion is obtained. n The corresponding edge deletion model of the mth edge deletion.

[0097] Step 5: Use the same renderer and rendering parameters as step 2 to n The corresponding edge deletion model of the mth edge deletion is rendered under the camera parameter T, and K is obtained. n The corresponding m-th edge-deleted rendered image set (i.e. Figure 2 (Operation of obtaining the rendering result after the simplified model is obtained).

[0098] Step 6: Apply the visual model to calculate K n The LPIPS loss between the corresponding m-th edge-deleted rendered image set and I, if K n If the corresponding m-th LPIPS loss is less than or equal to the threshold, K n The corresponding edge-deleted model of the mth edge-deleted process is used as the mesh model to be deleted or the target mesh model for the next iteration; if K n If the corresponding m-th LPIPS loss is greater than the threshold, K nThe corresponding mesh model to be deleted in the mth time of edge deletion is used as the mesh model to be deleted or the target mesh model for the next iteration; each hyperplane in the N normal graph sets is traversed, and the actions of the aforementioned steps 3 to 5 are performed in each traversal until the traversal is completed.

[0099] It should be understood that Figure 2 The termination condition shown in should be understood as reaching a set number of traversals, where the set number of traversals is the sum of the numbers of hyperplanes in the N normal graph sets.

[0100] It should be noted that, in some implementations, the actions of step one and step two can be skipped, and multiple real-scene images of a certain object (i.e., the aforementioned target object) under multiple perspectives can be taken as I, and then geometric reconstruction is performed based on I to obtain an initial mesh model; by estimating the camera pose of the multiple real-scene images respectively, a set T can be obtained; at this time, the set of N normal maps in step three is obtained by the large visual model performing normal prediction on the multiple real-scene images respectively.

[0101] The mesh model simplification method provided by the present invention uses a large visual model to calculate the simplification loss of each modeling process during the model simplification process, and selects edges / points with smaller losses under LVM evaluation for model simplification and contraction, so as to adaptively skip the model simplification and contraction operations with excessive simplification losses, avoid unnecessary topological changes, and make the simplified model meet the user's visual preferences and priors; among them, by determining the hyperplane through normal clustering and deleting the edges, it is possible to better find artificial planes (for example: desktops, walls, and floors), so as to shrink the edges / points in specific areas in a targeted manner, and achieve efficient simplification of artificial planes.

[0102] See also Figure 3 , Figure 3 A grid model simplification device provided by an embodiment of the present disclosure is as follows: Figure 3 As shown, the grid model simplification device 300 includes:

[0103] An acquisition module 301 is used to acquire N first rendered images of the initial mesh model under N camera poses, where N is an integer greater than or equal to 1;

[0104] The model simplification module 302 is used to perform multiple iterative simplifications on the initial mesh model according to the N first rendered images to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, and the visual difference between the N second rendered images obtained by the effective iterative simplification and the N first rendered images is less than or equal to a first preset threshold, and the visual difference is obtained based on a pre-trained neural network model, and the N second rendered images obtained by the effective iterative simplification correspond one-to-one to the N camera poses.

[0105] In one embodiment, the model simplification module 302 includes:

[0106] a model simplification submodule, configured to simplify the mesh model to be simplified in the nth iteration based on the nth first rendering image to obtain a simplified model in the nth iteration, wherein n is a positive integer less than or equal to N, and the mesh model to be simplified in the first iteration is the initial mesh model;

[0107] a difference comparison submodule, configured to compare the visual differences between the N first rendered images and the N second rendered images of the simplified model of the nth iteration, to obtain a visual difference value of the nth iteration;

[0108] A first model determination submodule is used to use the simplified model of the nth iteration as the mesh model to be simplified for the (n+1)th iteration, or use the simplified model of the nth iteration as the target mesh model, if the visual difference value of the nth iteration is less than or equal to the first preset threshold;

[0109] The second model simplification submodule is used to use the mesh model to be simplified in the nth iteration as the mesh model to be simplified in the (n+1)th iteration, or to use the mesh model to be simplified in the nth iteration as the target mesh model, when the visual difference value of the nth iteration is greater than the first preset threshold.

[0110] In one embodiment, the model simplification submodule includes:

[0111] A normal clustering unit is used to cluster multiple normal graphs of multiple facets included in the nth first rendered image to obtain M of the nth first rendered image. n Group normal cluster, M n is a positive integer;

[0112] Iterative edge erasure unit, for n The group normal cluster iteratively deletes edges of the mesh model to be simplified in the nth iteration to obtain a simplified model in the nth iteration.

[0113] In one embodiment, the iterative edge erasing unit is specifically configured to:

[0114] Use the normal vector value of the cluster center of the mth group of normal clusters to replace the remaining normal vector values ​​of the mth group, and obtain the normal clustering graph of the mth group of normal clusters, where m is less than or equal to M. n integer;

[0115] Perform edge detection on the normal clustering graph of the m-th group of normal clusters to determine the edge contour of the m-th hyperplane;

[0116] In the mesh model to be deleted in the mth iteration, the face edge located in the edge contour of the mth hyperplane is deleted to obtain the deleted model for the mth iteration, and the mesh model to be deleted in the first deletion in the nth iteration is the mesh model to be simplified in the nth iteration;

[0117] Comparing visual differences between the N first rendered images and N second rendered images of the edge erasure model of the m-th edge erasure, to obtain a visual difference value of the m-th edge erasure;

[0118] When the visual difference value of the m-th edge deletion is less than or equal to the first preset threshold, the edge deletion model of the m-th edge deletion is used as the mesh model to be deleted for the (m+1)-th edge deletion, or the edge deletion model of the m-th edge deletion is used as the simplified model for the n-th iteration;

[0119] When the visual difference value of the m-th edge deletion is greater than the first preset threshold, the mesh model to be deleted of the m-th edge deletion is used as the mesh model to be deleted of the m+1-th edge deletion, or the mesh model to be deleted of the m-th edge deletion is used as the simplified model of the n-th iteration.

[0120] In one embodiment, the model simplification submodule includes a normal acquisition unit, and the normal acquisition unit is specifically used to:

[0121] Based on N preset camera pose parameters, respectively performing normal map rendering on the initial mesh model to obtain a normal map corresponding to each of the first rendered images, wherein the N camera pose parameters correspond one-to-one to the N camera poses;

[0122] or,

[0123] Normal prediction is performed on each of the N first rendered images to obtain a normal map corresponding to each of the first rendered images.

[0124] In one embodiment, the visual difference is used to indicate the difference in images perceived by human vision.

[0125] In one embodiment, the acquisition module 301 includes a first acquisition unit, and the first acquisition unit is specifically configured to:

[0126] Get the initial mesh model;

[0127] The initial mesh model is respectively rendered based on the preset N camera pose parameters to obtain N first rendered images.

[0128] In one embodiment, the acquisition module 301 includes a second acquisition unit, and the second acquisition unit is specifically configured to:

[0129] Obtain N first rendered images of the target object under N camera poses;

[0130] The N first rendered images are geometrically reconstructed to obtain an initial mesh model indicating the target object.

[0131] In one embodiment, the visual difference between the N second rendered images simplified in each of the multiple iterative simplifications and the N first rendered images is obtained by calculating the learnable perceptual image block similarity LPIPS based on a preset visual macro model.

[0132] The mesh model simplification device 300 provided in the embodiment of the present disclosure can implement each process in the above-mentioned mesh model simplification method embodiment, and will not be described again here to avoid repetition.

[0133] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0134] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0135] like Figure 4 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 to a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0136] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0137] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the mesh model simplification method. For example, in some embodiments, the mesh model simplification method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the mesh model simplification method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the mesh model simplification method in any other appropriate manner (eg, by means of firmware).

[0138] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] As used herein, the term "machine-readable medium" refers to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0143] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0144] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0145] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 or Figure 2The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A mesh model simplification method, characterized in that: The method comprises: Obtain N first rendered images of the initial mesh model at N camera poses, where N is an integer greater than or equal to 1; According to the N first rendered images, the initial mesh model is iteratively simplified multiple times to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, the visual difference between the N second rendered images obtained by effective iterative simplification and the N first rendered images is less than or equal to a first preset threshold, the visual difference is obtained based on a pre-trained neural network model, and the N second rendered images obtained by effective iterative simplification correspond one-to-one to the N camera poses; The step of performing multiple iterations of simplification on the initial mesh model according to the N first rendered images to obtain a target mesh model includes: Simplifying the mesh model to be simplified in the nth iteration based on the nth first rendering image to obtain a simplified model in the nth iteration, wherein n is a positive integer less than or equal to N, and the mesh model to be simplified in the first iteration is the initial mesh model; Comparing visual differences between the N first rendered images and the N second rendered images of the simplified model of the nth iteration to obtain a visual difference value of the nth iteration; When the visual difference value of the nth iteration is less than or equal to the first preset threshold, the simplified model of the nth iteration is used as the mesh model to be simplified in the (n+1)th iteration, or the simplified model of the nth iteration is used as the target mesh model; When the visual difference value of the nth iteration is greater than the first preset threshold, the mesh model to be simplified in the nth iteration is used as the mesh model to be simplified in the (n+1)th iteration, or the mesh model to be simplified in the nth iteration is used as the target mesh model.

2. The method according to claim 1, characterized in that The visual difference is used to indicate the difference in images perceived by human vision.

3. The method according to claim 2, characterized in that The step of performing model simplification on the mesh model to be simplified of the nth iteration based on the nth first rendered image to obtain the simplified model of the nth iteration includes: Clustering is performed on a plurality of normal graphs corresponding to the nth first rendered image to obtain an M corresponding to the nth first rendered image. n Group normal cluster, M n is a positive integer; According to the M n The group normal cluster iteratively deletes edges of the mesh model to be simplified in the nth iteration to obtain a simplified model in the nth iteration.

4. The method according to claim 3, characterized in that According to the M n The group normal cluster iteratively deletes edges of the mesh model to be simplified in the nth iteration to obtain a simplified model in the nth iteration, including: Use the normal vector value of the cluster center of the mth group of normal clusters to replace the remaining normal vector values ​​of the mth group, and obtain the normal clustering graph of the mth group of normal clusters, where m is less than or equal to M. n A positive integer of ; Perform edge detection on the normal clustering graph of the m-th group of normal clusters to determine the edge contour of the m-th hyperplane; In the mesh model to be deleted in the mth iteration, the face edge located in the edge contour of the mth hyperplane is deleted to obtain the deleted model for the mth iteration, and the mesh model to be deleted in the first deletion in the nth iteration is the mesh model to be simplified in the nth iteration; Comparing visual differences between the N first rendered images and N second rendered images of the edge erasure model of the m-th edge erasure, to obtain a visual difference value of the m-th edge erasure; When the visual difference value of the m-th edge deletion is less than or equal to the first preset threshold, the edge deletion model of the m-th edge deletion is used as the mesh model to be deleted for the (m+1)-th edge deletion, or the edge deletion model of the m-th edge deletion is used as the simplified model for the n-th iteration; When the visual difference value of the m-th edge deletion is greater than the first preset threshold, the mesh model to be deleted of the m-th edge deletion is used as the mesh model to be deleted of the m+1-th edge deletion, or the mesh model to be deleted of the m-th edge deletion is used as the simplified model of the n-th iteration.

5. The method according to claim 3, characterized in that: Before performing multiple iterations of simplification on the initial mesh model according to the N first rendered images to obtain the target mesh model, the method further includes: Based on N preset camera pose parameters, respectively performing normal map rendering on the initial mesh model to obtain a normal map corresponding to each of the first rendered images, wherein the N camera pose parameters correspond one-to-one to the N camera poses; or, Normal prediction is performed on each of the N first rendered images to obtain a normal map corresponding to each of the first rendered images.

6. The method according to any one of claims 1 to 5, characterized in that The step of obtaining N first rendered images of the initial mesh model under N camera poses includes: Get the initial mesh model; The initial mesh model is rendered respectively based on N preset camera pose parameters to obtain N first rendered images, and the N camera pose parameters correspond one to one to the N camera poses.

7. The method according to any one of claims 1 to 5, characterized in that The step of obtaining N first rendered images of the initial mesh model under N camera poses includes: Obtain N first rendered images of the target object under N camera poses; The N first rendered images are geometrically reconstructed to obtain an initial mesh model for indicating the target object.

8. A mesh model simplification device, characterized in that: The device comprises: An acquisition module, used to acquire N first rendered images of the initial mesh model under N camera poses, where N is an integer greater than or equal to 1; a model simplification module, configured to perform multiple iterative simplifications on the initial mesh model according to the N first rendered images to obtain a target mesh model, wherein the target mesh model is a mesh model obtained based on effective iterative simplifications in the multiple iterative simplifications, the visual difference between the N second rendered images obtained by effective iterative simplification and the N first rendered images is less than or equal to a first preset threshold, the visual difference is obtained based on a pre-trained neural network model, and the N second rendered images obtained by effective iterative simplification correspond one-to-one to the N camera poses; The model simplification module comprises: a model simplification submodule, configured to simplify the mesh model to be simplified in the nth iteration based on the nth first rendering image to obtain a simplified model in the nth iteration, wherein n is a positive integer less than or equal to N, and the mesh model to be simplified in the first iteration is the initial mesh model; a difference comparison submodule, configured to compare the visual differences between the N first rendered images and the N second rendered images of the simplified model of the nth iteration, to obtain a visual difference value of the nth iteration; A first model determination submodule is used to use the simplified model of the nth iteration as the mesh model to be simplified for the (n+1)th iteration, or use the simplified model of the nth iteration as the target mesh model, if the visual difference value of the nth iteration is less than or equal to the first preset threshold; The second model simplification submodule is used to use the mesh model to be simplified in the nth iteration as the mesh model to be simplified in the (n+1)th iteration, or to use the mesh model to be simplified in the nth iteration as the target mesh model, when the visual difference value of the nth iteration is greater than the first preset threshold.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.