Quality evaluation method and device of grid model, equipment, medium and product
By dividing the grid model into multiple regions and determining the contribution weight based on the region parameters, feature extraction is performed, which solves the problem of inaccurate evaluation caused by information redundancy in the existing technology and achieves more efficient quality assessment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-04-17
AI Technical Summary
In existing methods for assessing the quality of grid models, the consistency of global contribution leads to information redundancy, affecting the accuracy of the assessment.
The target mesh model is divided into multiple regional mesh models. The contribution weights are determined based on the regional parameters. Geometric and image features are extracted, and the quality assessment results are determined by combining the regional contribution weights.
This reduces information redundancy in the evaluation process and improves the accuracy of grid model quality assessment.
Smart Images
Figure CN121884082A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of model evaluation technology, and in particular to a method, apparatus, equipment, medium and product for quality evaluation of mesh models. Background Technology
[0002] A mesh model is a numerical model that uses discretized grid cells to approximate continuous geometry or physical fields. Mesh models serve as a bridge between geometric and physical simulations, and their quality determines the reliability of numerical simulations; therefore, evaluating the quality of mesh models is crucial.
[0003] With the development of machine learning, deep learning, computer vision, and graph neural networks, training a deep model using data has become the mainstream method for evaluating the quality of mesh models. Existing techniques, in addition to evaluating the quality of mesh models through deep models, also assess quality by comparing the mesh model under test with known high-quality mesh models. This means shifting the focus from directly evaluating the quality score to assessing how much it differs from the best-quality model.
[0004] However, both the mesh model and its corresponding rendered image suffer from uneven distribution of information and detail across different regions. Consequently, different regions contribute differently to the evaluation of mesh model quality. Existing quality assessment methods exhibit a uniform global contribution, introducing inefficient information and causing redundancy, ultimately affecting the accuracy of the quality assessment. Summary of the Invention
[0005] This application provides a method, apparatus, device, medium, and product for quality assessment of a mesh model, which addresses the technical problem that existing quality assessment methods suffer from inconsistent global contributions, leading to inefficient information being introduced into the quality assessment process, causing information redundancy, and ultimately affecting the accuracy of the quality assessment.
[0006] In a first aspect, embodiments of this application provide a method for quality assessment of a mesh model, including: The target mesh model is divided into multiple region mesh models; For each of the aforementioned regional grid models, the regional grid model is projected to obtain the regional projection image corresponding to the regional grid model; For each of the aforementioned regional grid models, the regional contribution weight of the regional grid model is determined based on the target parameters of the regional grid model; For each of the aforementioned regional grid models, geometric features are extracted from the regional grid models to obtain the regional geometric features of the regional grid models; For each of the region projection images, image features are extracted from the region projection images to obtain the region image features corresponding to the region projection images; The quality assessment result of the target mesh model is determined based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0007] In one embodiment, dividing the target mesh model into multiple region mesh models includes: The target mesh model is projected to obtain a target projection image corresponding to the target mesh model; the target projection image is divided into multiple image regions based on a preset number of regions and the extreme values of the mesh coordinates corresponding to the target mesh model; and the mesh model of each region is determined based on the projection coordinates of all the image regions and the mesh vertices of the target mesh model. Alternatively, the target grid model can be divided based on the preset number of regions and a preset clustering algorithm to obtain each region grid model; Alternatively, the grid vertices in the target grid model can be classified based on a point cloud segmentation algorithm, and the grid model of each region can be determined based on the classified grid vertices.
[0008] In one embodiment, the area projection image includes split-view projection images under multiple preset viewpoints; The step of projecting the region grid model to obtain the region projection image corresponding to the region grid model includes: For each preset viewpoint, the region grid model is projected onto the preset viewpoint using a projection tool to obtain a split-view projection image under the preset viewpoint.
[0009] In one embodiment, the target parameters include the region volume, number of grid vertices, grid vertex coordinates, center coordinates, number of faces, number of first-order adjacent faces of each face, and normal vector of the region mesh model. The determination of the regional contribution weight of the regional grid model based on the target parameters of the regional grid model includes: The regional contribution weight of the regional mesh model is determined based on one or more of the following: regional vertex density, regional average curvature, and regional average distance. Wherein, the region vertex density is determined based on the region volume and the number of grid vertices corresponding to the region mesh model; the region average curvature is determined based on the number of faces and the curvature of all faces corresponding to the region mesh model; for each face curvature, the face curvature is determined based on the number of first-order adjacent faces and their normal vectors, as well as the normal vector of another adjacent face; the region average distance is determined based on the number of grid vertices, grid vertex coordinates, and center coordinates corresponding to the region mesh model.
[0010] In one embodiment, the step of extracting geometric features from the region grid model to obtain the region geometric features of the region grid model includes: The graph structure corresponding to the region grid model is input into the geometric feature extraction network to obtain the regional geometric features of the region grid model output by the geometric feature extraction network; the geometric feature extraction network is trained based on the sample graph structure; the geometric feature extraction network is used to extract geometric features.
[0011] In one embodiment, the step of extracting image features from the region projection image to obtain the region image features corresponding to the region projection image includes: The region projection image is cropped and reconstructed to obtain the reconstructed image corresponding to the region projection image; The reconstructed image is input into an image feature extraction network to obtain the region image features of the region projection image output by the image feature extraction network; the image feature extraction network is trained based on sample images; the image feature extraction network is used to perform image feature extraction.
[0012] In one embodiment, determining the quality assessment result of the target mesh model based on all the region contribution weights, all the region geometric features, and all the region image features includes: For each of the aforementioned regional grid models, the regional geometric features and regional image features corresponding to the regional grid model are fused and stitched together to obtain the regional feature vector corresponding to the regional grid model. The contribution fine-tuning feature corresponding to the regional grid model is determined based on the product of the regional contribution weight and the regional feature vector. The quality assessment result of the target grid model is determined based on the contribution fine-tuning features corresponding to all the aforementioned regional grid models.
[0013] In one embodiment, determining the quality assessment result of the target grid model based on the contribution fine-tuning features corresponding to all the regional grid models includes: The contribution fine-tuning features corresponding to all the regional grid models are input into the quality assessment neural network to obtain the quality assessment result output by the quality assessment neural network; the quality assessment neural network is trained based on sample features and corresponding labels.
[0014] Secondly, embodiments of this application also provide a mesh model quality assessment device, comprising: The partitioning module is used to divide the target mesh model into multiple region mesh models; The projection module is used to project the region grid model onto each of the region grid models to obtain the region projection image corresponding to the region grid model. The contribution module is used to determine the regional contribution weight of each regional grid model based on the target parameters of the regional grid model. The geometry module is used to extract geometric features from each of the aforementioned regional mesh models to obtain the regional geometric features of the regional mesh models. The image module is used to extract image features from the projected images of each region to obtain the region image features corresponding to the projected images of the regions. The quality assessment module is used to determine the quality assessment result of the target mesh model based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0015] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the quality assessment method for the mesh model as described above.
[0016] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the quality assessment method for the mesh model as described above.
[0017] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements a quality assessment method for a mesh model as described above.
[0018] The mesh model quality assessment method, apparatus, device, medium, and product provided in this application divides the target mesh model into multiple regional mesh models; for each regional mesh model, a projection is performed to obtain a corresponding regional projection image; for each regional mesh model, a regional contribution weight is determined based on the target parameters of the regional mesh model; for each regional mesh model, geometric features are extracted to obtain regional geometric features; for each regional projection image, image features are extracted to obtain corresponding regional image features; and the quality assessment result of the target mesh model is determined based on all regional contribution weights, all regional geometric features, and all regional image features. The mesh model quality assessment provided in this application divides the target mesh model into multiple regions and determines the regional contribution weight for each region, reducing information redundancy in the assessment process, improving information utilization, and increasing the accuracy of mesh model quality assessment. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the quality assessment method for a mesh model provided in an embodiment of this application.
[0021] Figure 2 This is one of the flowcharts illustrating the adjustment of network fronthaul weights by regional contribution weights provided in the embodiments of this application.
[0022] Figure 3 This is the second schematic diagram of the process for adjusting the network fronthaul weight by regional contribution weight provided in the embodiments of this application.
[0023] Figure 4 This is a schematic diagram of the structure of the mesh model quality assessment device provided in the embodiments of this application.
[0024] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0026] To address the aforementioned problems in the prior art, embodiments of this application provide a method for quality assessment of mesh models. Figure 1 This is a flowchart illustrating the quality assessment method for mesh models provided in this application. Figure 1 As shown, the method includes the following steps 110 to 120.
[0027] Step 110: Divide the target mesh model into multiple region mesh models.
[0028] Specifically, to assess the contribution of different regions, this application requires prior division of the target mesh model into regions, that is, dividing all mesh vertices within the mesh model according to certain rules. When dividing the regional mesh model, it should be ensured that each vertex clearly belongs to only one specific regional mesh model after division.
[0029] In one embodiment, dividing the target mesh model into multiple region mesh models includes: The target mesh model is projected to obtain a target projection image corresponding to the target mesh model; the target projection image is divided into multiple image regions based on a preset number of regions and the extreme values of the mesh coordinates corresponding to the target mesh model; and the mesh model of each region is determined based on the projection coordinates of all the image regions and the mesh vertices of the target mesh model. Alternatively, the target grid model can be divided based on the preset number of regions and a preset clustering algorithm to obtain each region grid model; Alternatively, the grid vertices in the target grid model can be classified based on a point cloud segmentation algorithm, and the grid model of each region can be determined based on the classified grid vertices.
[0030] Specifically, the target mesh model can be divided using various methods. In this embodiment, three methods are listed below, and one method can be selected to divide the target mesh model.
[0031] (1) Project the target mesh model to obtain the target projection image corresponding to the target mesh model. Then, based on the preset number of regions and the extreme values of the mesh coordinates corresponding to the target mesh model, the target projection image is divided into multiple image regions. Then, the mesh model of each region can be determined based on the projection coordinates of all image regions and the mesh vertices of the target mesh model. That is, the mesh vertices whose projection coordinates fall within the same image region belong to the same region mesh model.
[0032] (2) The target mesh model is divided based on the preset number of regions and the preset clustering algorithm to obtain mesh models for each region. The preset clustering algorithm may include K-means clustering and Farthest Point Sampling (FPS) algorithms, etc., and is not specifically limited in this embodiment. Then, clustering can be performed within the vertex space coordinates of the target mesh model according to the preset number of regions and the hyperparameters of the clustering algorithm used. After clustering, each cluster corresponds to a mesh model for one region.
[0033] (3) The grid vertices in the target grid model can be classified based on the point cloud segmentation algorithm. After the segmentation is completed, the grid vertices of the same category can be regarded as the same region, and the grid model of each region can be determined based on the classified grid vertices. The point cloud segmentation algorithm may include algorithms such as point cloud neural network (PointNet), and the embodiments of this application do not make specific limitations here.
[0034] In the above embodiments, the target mesh model can be divided into multiple regional mesh models through various methods, laying the foundation for determining the contribution of each region in the subsequent process.
[0035] Step 120: For each of the aforementioned regional mesh models, project the regional mesh model to obtain the regional projection image corresponding to the regional mesh model.
[0036] Specifically, in the history of mesh model quality assessment, the earliest mesh models were colorless, and quality assessment only focused on identifying structural problems within the mesh model itself, such as holes, burrs, and clipping. However, with the development of display devices, mesh models have evolved into geometric structures with added textures (or mapping). At this stage, in addition to evaluating the geometric structure of the mesh model itself, it is also necessary to assess the quality of the content displayed to the viewer. A common practice for evaluating the quality of displayed content is to project the mesh model from a given projection angle and render it based on the texture image to obtain a color image. This color image is the content ultimately observed by the viewer on the display device, and its quality can be considered the quality of the displayed content. Projection methods are purely mathematical geometric mapping methods, and many methods are available; this application does not impose any particular restrictions on the projection method.
[0037] Therefore, for each region's mesh model, a projection can be performed on the region's mesh model to obtain the corresponding region's projected image. It is easy to understand that when projecting the region's mesh model, a single projection angle or multiple projection angles can be selected; this embodiment does not impose any specific limitations.
[0038] In one embodiment, the area projection image includes split-view projection images under multiple preset viewpoints; The step of projecting the region grid model to obtain the region projection image corresponding to the region grid model includes: For each preset viewpoint, the region grid model is projected onto the preset viewpoint using a projection tool to obtain a split-view projection image under the preset viewpoint.
[0039] Specifically, during the projection of a region mesh model, if multiple projection angles are selected, the region projection image includes segmented projection images from multiple preset viewpoints. When selecting projection angles, 3D space is projected onto 3D space, resulting in occlusion. The actual color of the surrounding area of an occluded vertex does not match the color of the projected rendered image. Furthermore, in practical business scenarios where mesh models are applied, their display is not limited to a single viewpoint. To avoid the aforementioned occlusion and to comprehensively evaluate the presentation quality from each viewpoint, multiple preset projection viewpoints are necessary. The selection of viewpoints ensures that each vertex is not occluded at at least one projection angle; therefore, six preset viewpoints can be considered. These preset viewpoints can include, for example, a front view, a back view, a left view, a right view, a top view, and an elevation view.
[0040] For each preset viewpoint, the region mesh model can be projected using a projection tool under that preset viewpoint (i.e., the coordinates of the corresponding facets, facet textures, and mesh vertex textures of the region mesh model can be projected), thus obtaining a split-view projection image under that preset viewpoint.
[0041] In the above embodiments, the regional mesh model is projected under multiple preset viewpoints, so that each mesh vertex corresponds to a split-view projection image without occlusion, thereby improving the reliability of projection.
[0042] Step 130: For each of the aforementioned regional grid models, determine the regional contribution weight of the regional grid model based on the target parameters of the regional grid model.
[0043] Specifically, after dividing the target mesh model into several regional mesh models, each regional mesh model has different values in visual, physical, and other quality-related dimensions, naturally resulting in significant differences in the mesh model's representation and visual rendering. For example, the denser the mesh vertices in a region, the more accurately it fits the transition surfaces of an object, resulting in a more detailed fit and a closer resemblance to the real object, thus achieving better rendering quality. Conversely, the sparser the mesh vertices in a region, the weaker its expressive power and the less precise the fit, also leading to poorer rendering quality. That is, the higher the vertex density of a region, the lower its contribution to the quality assessment; conversely, the lower the vertex density of a region, the higher its contribution to the quality assessment. Therefore, it is necessary to combine parameters that can evaluate the quality of the mesh model to calculate the contribution. For each regional mesh model, the regional contribution weight can be determined based on the target parameters of the regional mesh model.
[0044] In one embodiment, the target parameters include the region volume, number of grid vertices, grid vertex coordinates, center coordinates, number of faces, number of first-order adjacent faces of each face, and normal vector of the region mesh model. The determination of the regional contribution weight of the regional grid model based on the target parameters of the regional grid model includes: The regional contribution weight of the regional mesh model is determined based on one or more of the following: regional vertex density, regional average curvature, and regional average distance. Wherein, the region vertex density is determined based on the region volume and the number of grid vertices corresponding to the region mesh model; the region average curvature is determined based on the number of faces and the curvature of all faces corresponding to the region mesh model; for each face curvature, the face curvature is determined based on the number of first-order adjacent faces and their normal vectors, as well as the normal vector of another adjacent face; the region average distance is determined based on the number of grid vertices, grid vertex coordinates, and center coordinates corresponding to the region mesh model.
[0045] Specifically, the regional contribution weight of the regional mesh model can be determined based on one or more of the regional vertex density, regional average curvature, and regional average distance. In addition, the regional contribution weight of the regional mesh model can also be determined through other parameters that can evaluate the quality of the mesh model. This embodiment does not make specific limitations here.
[0046] For a given region-specific mesh model, higher vertex density generally results in stronger expressive power and naturally higher rendering quality. Conversely, lower vertex density leads to insufficient expressive power and naturally lower rendering quality. Therefore, in quality assessment tasks, more attention should be paid to low-density regions, where the contribution is higher and higher-density regions contribute less. (Regional vertex density) This can be expressed by the following formula: in, This indicates the number of grid vertices corresponding to the grid model of this region. This indicates the volume of the region corresponding to the mesh model of that region.
[0047] For a given mesh model, the smoother the transitions between facets, the higher the rendering quality. Conversely, the sharper the facet connections and the more pronounced the transitions, the lower the rendering quality. Therefore, in quality assessment tasks, more attention should be paid to sharp regions, i.e., low-smoothness regions contribute significantly, while high-smoothness regions contribute little. The smoothness of a facet and its adjacent faces can be approximated by the facet curvature. The curvature of each surface It can be done using the following formula: in, Indicates the first The number of first-order adjacent faces of a face. Indicates the first The normal vectors of adjacent faces, Indicates the first The normal vector of each facet. This represents finding the cosine of the angle between two normal vectors.
[0048] Furthermore, the region average curvature is the average of the curvatures of all patches corresponding to the mesh model of that region. It can be determined using the following formula: in, This indicates the number of facets corresponding to the mesh model of this region.
[0049] For a given region's mesh model, the closer each mesh vertex is to the center of the target mesh model, the closer it is to the visual center in the final display, and the higher its visual attention during human viewing. Conversely, the farther away from the center of the target mesh model, the further away it is from the visual center in the final display, and the lower its visual attention during human viewing. Therefore, in the quality assessment process, more attention should be paid to areas closer to the center of the target mesh model; that is, the central area contributes more, while the peripheral areas contribute less. Thus, the distance of the mesh model from the center in a given region can be determined by the average distance of that region. It can be expressed by the following formula in, , , Indicates the first The coordinates of each grid vertex , , This represents the center coordinates of the target mesh model.
[0050] It should be noted that when selecting multiple evaluation parameters (i.e., regional vertex density, regional average curvature, and regional average distance, etc.) to determine the regional contribution weight of the regional grid model, it is necessary to summarize the multiple evaluation parameters, such as taking the average value of all evaluation parameters. This embodiment does not restrict the summarization method.
[0051] In the above embodiments, the regional contribution weight of the regional mesh model is determined based on one or more of the following: regional vertex density, regional average curvature, and regional average distance, ensuring that different regional mesh models correspond to different regional contribution weights. Step 140: For each of the aforementioned regional mesh models, perform geometric feature extraction on the regional mesh model to obtain the regional geometric features of the regional mesh model.
[0052] In one embodiment, the step of extracting geometric features from the region grid model to obtain the region geometric features of the region grid model includes: The graph structure corresponding to the region grid model is input into the geometric feature extraction network to obtain the regional geometric features of the region grid model output by the geometric feature extraction network; the geometric feature extraction network is trained based on the sample graph structure; the geometric feature extraction network is used to extract geometric features.
[0053] Specifically, a geometric feature extraction network can be pre-trained based on the sample graph structure and the corresponding labels. In this embodiment, the graph structure corresponding to the regional grid model can be input into the geometric feature extraction network, which is used to extract geometric features, thereby obtaining the regional geometric features of the regional grid model output by the geometric feature extraction network.
[0054] In the above embodiments, inputting the graph structure corresponding to the regional grid model into the geometric feature extraction network can accurately extract the geometric features of the regional grid model, providing an evaluation dimension of the geometric structure for subsequent quality assessment.
[0055] Step 150: For each of the region projection images, perform image feature extraction on the region projection image to obtain the region image features corresponding to the region projection image.
[0056] In one embodiment, the step of extracting image features from the region projection image to obtain the region image features corresponding to the region projection image includes: The region projection image is cropped and reconstructed to obtain the reconstructed image corresponding to the region projection image; The reconstructed image is input into an image feature extraction network to obtain the region image features of the region projection image output by the image feature extraction network; the image feature extraction network is trained based on sample images; the image feature extraction network is used to perform image feature extraction.
[0057] Specifically, when multiple projection angles exist, calculating the quality score requires combining features from multiple preset viewpoints, thus necessitating a multi-view feature fusion method. Many methods exist for feature fusion; in this embodiment, the region projection image can be cropped and reconstructed to obtain a reconstructed image corresponding to the region projection image. For example, if the region projection image includes six preset viewpoint projection images, these six sub-viewpoint projection images can be cropped and reconstructed to obtain a single reconstructed image. Feature extraction from this reconstructed image completes the feature fusion process.
[0058] Furthermore, the reconstructed image can be input into an image feature extraction network, which extracts image features to obtain the region image features of the projected image of that region. It is easy to understand that the image feature extraction network is pre-trained based on sample images and their corresponding labels.
[0059] Alternatively, in another multi-view feature fusion method, multiple split-view projection images from the region projection image can be directly input into an image feature extraction network to obtain the split-view image features corresponding to each split-view projection image. Furthermore, the obtained split-view image features can be short-circuited in a preset order to obtain the region image features corresponding to the region projection image.
[0060] In the above embodiments, multi-view image fusion is achieved by cropping and reconstructing the regional projection image. Furthermore, the reconstructed image is input into the image feature extraction network, which can accurately extract the regional image features of the regional projection image, providing a dimension for human visual perception quality assessment for subsequent quality evaluation.
[0061] Step 160: Determine the quality assessment result of the target mesh model based on all the region contribution weights, all the region geometric features, and all the region image features.
[0062] Specifically, after the regional contribution weights are calculated, they can be introduced into the subsequent neural network. By adjusting the weights of some parameters in the network, the quality assessment process can focus more on the high-contribution parts, thereby reducing information redundancy and enhancing information expressiveness. Several examples of adjusting network forward propagation weights using regional contribution weights are provided below: Figure 2This is one of the flowcharts illustrating the adjustment of network fronthaul weights using regional contribution weights provided in this application embodiment, such as... Figure 2 As shown, the parameters related to regional contribution are the evaluation parameters mentioned earlier. Parameter n represents the nth evaluation parameter. The selected evaluation parameters are concatenated into a one-dimensional feature vector, which is the regional contribution weight. A multi-layer perceptron (MLP) is introduced, and the final output is a value regularized to between 0 and 1. In the subsequent forward propagation process of the network, at least one forward propagation calculation multiplies each regional feature vector by its regional weight coefficient. Figure 2 In Indicates the first Features of the layer This represents the weight calculated based on the region's contribution. This represents the weights of the first layer in a multi-layer neural network. This represents the characteristics of the first layer.
[0063] Figure 3 This is the second schematic diagram of the process for adjusting the network forward propagation weights by regional contribution weights provided in the embodiments of this application. Let c be the number of feature channels to be introduced into the regional contribution weight layer in the subsequent process. The selected evaluation parameters are concatenated into a one-dimensional feature vector, and a multilayer neural network (MLP) is introduced. The number of output channels of the last layer is also c, and the values are regularized to between 0 and 1. Similar to the attention mechanism, each regional feature vector is multiplied by its regional weight coefficient.
[0064] After adjusting the network forward weights by adjusting the regional contribution weights, the geometric features and image features of all regions can be input into the subsequent neural network to determine the quality assessment result of the target mesh model.
[0065] In one embodiment, determining the quality assessment result of the target mesh model based on all the region contribution weights, all the region geometric features, and all the region image features includes: For each of the aforementioned regional grid models, the regional geometric features and regional image features corresponding to the regional grid model are fused and stitched together to obtain the regional feature vector corresponding to the regional grid model. The contribution fine-tuning feature corresponding to the regional grid model is determined based on the product of the regional contribution weight and the regional feature vector. The quality assessment result of the target grid model is determined based on the contribution fine-tuning features corresponding to all the aforementioned regional grid models.
[0066] Specifically, for each region's grid model, the region's geometric features and image features can be fused and stitched together to obtain the region's feature vector corresponding to the region's grid model.
[0067] During feature fusion, a vertex in a region grid model corresponds to a specific coordinate in the region projection image, and its direct relation is only to the region within a certain range surrounding that coordinate (pixel). Existing techniques that extract features globally from the image introduce irrelevant region projection image features to each vertex, still resulting in feature redundancy.
[0068] In this application, the feature fusion step is performed with region as a constraint, rather than with a global dimension. The specific feature fusion method is not limited. The following is an embodiment based on the stitching fusion process: after extracting features from slices of the region projection image, they are shorted to the features of the corresponding grid vertices within the region, and then a graph neural network is used for quality score regression.
[0069] Furthermore, the contribution fine-tuning features of the regional grid model can be determined based on the product of the regional contribution weights and the regional feature vectors. After obtaining the contribution fine-tuning features, the quality assessment result of the target grid model can be determined based on the contribution fine-tuning features of all regional grid models. The industry standard for grid model quality assessment is the Mean Opinion Score (MOS): In subjective experiments, multiple participants can freely change their viewing direction or observe the object through different devices and give a score. Any of the following scales can be used: 1-5, 1-7, and 1-10. In practice, it is necessary to normalize the scores. For example, in a 1-5 scale (which is basically an integer score), the average of the scores given by multiple participants is the mean opinion score of the object (the average score is a decimal).
[0070] Existing technologies mostly assess quality from a single dimension, lacking an effective method that integrates two dimensions. In the above embodiment, by stitching and fusing regional geometric features and regional image features, a method is provided that can comprehensively evaluate the quality of a grid model using two dimensions. The contribution fine-tuning features corresponding to the regional grid model are determined based on the product of the regional contribution weight corresponding to the regional grid model and the regional feature vector, which further improves information utilization and enhances the accuracy of quality assessment.
[0071] In one embodiment, determining the quality assessment result of the target grid model based on the contribution fine-tuning features corresponding to all the regional grid models includes: The contribution fine-tuning features corresponding to all the regional grid models are input into the quality assessment neural network to obtain the quality assessment result output by the quality assessment neural network; the quality assessment neural network is trained based on sample features and corresponding labels.
[0072] Specifically, a quality assessment neural network can be pre-trained based on sample features and corresponding labels. The loss function of the quality assessment neural network can be a regression loss function such as Mean Squared Error (MSE), L1 norm loss, or L2 norm loss. During training, the network parameters of the quality assessment neural network can be updated using the backpropagation algorithm (BP algorithm).
[0073] By inputting the contribution fine-tuning features corresponding to all regional grid models into the quality assessment neural network, the quality assessment result output by the quality assessment neural network can be obtained, which can be a specific quality score.
[0074] In the above embodiments, the accuracy of mesh model quality assessment is further improved by using a quality assessment neural network.
[0075] By way of example, the present application is illustrated below through a complete embodiment: In mainstream mesh model data formats (such as .obj), each mesh vertex and face has a corresponding serial number when storing data. Mesh model data includes the following information: (1) Vertex coordinates: (2) Patch: A patch consists of at least 3 mesh vertices. For example, a patch (3) Patch mapping: Texture image of each patch (4) Patch normal: the normal vector of each patch (5) Edge: An edge is formed between two vertices. The edge is obtained from the face information, such as the edge. (6) Vertex map coordinates: The coordinates of the mesh vertices that make up the face within the face map. .
[0076] After obtaining the target mesh model, the mesh model is divided into m regions using the farthest point sampling algorithm and the mesh vertex coordinates. The result of the region division is the list of vertices contained in each region and the facets formed by these vertices.
[0077] in, Represents a region or region grid model. This represents the farthest point sampling algorithm. The farthest point sampling algorithm is an adaptive algorithm, meaning it can divide the mesh model into m regions regardless of its size.
[0078] Within each region, the coordinates of the face patches, face patches, and vertex patches within that region are used to perform a six-view projection using a projection tool (such as Open3D) to obtain six [images / descriptions]. The region projection image. Randomly crop out from all region projection images. indivual The image blocks are reconstructed into a The reconstructed image. Among them, Indicates the image height. Indicates the image width. This represents the hyperparameters, i.e., the reconstructed image is composed of... It is composed of randomly cropped image blocks, typically 32 or 16. The above process can be represented by the following formula: in, This indicates a projection operation. This represents a region after each projected image has been segmented. Indicates the first The first projected image , The segmented , This indicates a splitting operation. This indicates random sampling.
[0079] Furthermore, the reconstructed image is input into the image feature extraction network. (e.g., a sliding window transformer (ShiftedWindows Transformer, Swin-Transformer)) yields a length of... Regional image features : in, It represents the set of real numbers.
[0080] The graph structure consisting of vertices and edges in the region mesh model is input into the geometric feature extraction network. (e.g., GATv2) Extract features to obtain a set of lengths. Regional geometric features : Both regional image features and regional geometric features are one-dimensional features, and the two can be fused and concatenated into a regional feature vector (two-dimensional features). After performing this operation on all m regions, the total region mesh model can be obtained as an m×2d region feature vector matrix. .
[0081] This embodiment uses two evaluation parameters, region vertex density and region average curvature, to calculate the region contribution weight. The region vertex density is calculated using the coordinates of all grid vertices within the region. This can be achieved by first obtaining the coordinates of all points within the region. , , , , , Calculate the volume of the region. Thus, the vertex density of the region is determined. .
[0082] Region Vertex Density The values are floating-point numbers; the mean curvature of the region is then calculated. It is also a floating-point number.
[0083] The region vertex density and region average curvature are constructed into a two-dimensional vector, and a value between 0 and 1 is obtained through the MLP network. This value represents the region contribution weight of the mesh model for that region. : After performing this operation on all m regions, the grid model of all regions can obtain an m-by-1 contribution weight matrix. : Multiplying the aforementioned regional feature vectors by the regional contribution weights completes the weight fine-tuning, yielding the contribution-fine-tuned features. Contribution fine-tuning features It can be expressed by the following formula: After obtaining the contribution fine-tuning features, these features can be input into the quality assessment neural network. Quality assessment results .
[0084] When the training batch size is greater than 1, the loss function of the quality assessment neural network can be expressed by the following formula. : in, and All of these represent hyperparameters. Indicates L2 norm loss, Indicates the mean square error loss. Indicates the truth value.
[0085] The mesh model quality assessment method provided in this application divides the target mesh model into multiple regional mesh models; for each regional mesh model, a projection is performed to obtain the corresponding regional projection image; for each regional mesh model, a regional contribution weight is determined based on the target parameters of the regional mesh model; for each regional mesh model, geometric features are extracted to obtain the regional geometric features; for each regional projection image, image features are extracted to obtain the corresponding regional image features; and the quality assessment result of the target mesh model is determined based on all regional contribution weights, all regional geometric features, and all regional image features. The mesh model quality assessment method provided in this application divides the target mesh model into multiple regions and determines the regional contribution weight for each region, reducing information redundancy in the assessment process, improving information utilization, and increasing the accuracy of mesh model quality assessment.
[0086] The quality assessment device for the mesh model provided in the embodiments of this application is described below. The quality assessment device for the mesh model described below can be referred to in correspondence with the quality assessment method for the mesh model described above.
[0087] Figure 4 This is a schematic diagram of the structure of the mesh model quality assessment device provided in the embodiments of this application, as shown below. Figure 4 As shown, the quality assessment device 400 for the mesh model includes the following modules: The partitioning module 410 is used to divide the target mesh model into multiple region mesh models; The projection module 420 is used to project the region grid model onto each of the region grid models to obtain the region projection image corresponding to the region grid model. Contribution module 430 is used to determine the regional contribution weight of each regional grid model based on the target parameters of the regional grid model. The geometry module 440 is used to extract geometric features from each of the said regional mesh models to obtain the regional geometric features of the regional mesh models. Image module 450 is used to extract image features from the projected images of each region to obtain the region image features corresponding to the projected images of the regions. The quality assessment module 460 is used to determine the quality assessment result of the target mesh model based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0088] In one embodiment, the partitioning module 410 is specifically used for: The target mesh model is projected to obtain a target projection image corresponding to the target mesh model; the target projection image is divided into multiple image regions based on a preset number of regions and the extreme values of the mesh coordinates corresponding to the target mesh model; and the mesh model of each region is determined based on the projection coordinates of all the image regions and the mesh vertices of the target mesh model. Alternatively, the target grid model can be divided based on the preset number of regions and a preset clustering algorithm to obtain each region grid model; Alternatively, the grid vertices in the target grid model can be classified based on a point cloud segmentation algorithm, and the grid model of each region can be determined based on the classified grid vertices.
[0089] In one embodiment, the area projection image includes multiple preset viewing angle projection images; the projection module 420 is specifically used for: For each preset viewpoint, the region grid model is projected onto the preset viewpoint using a projection tool to obtain a split-view projection image under the preset viewpoint.
[0090] In one embodiment, the target parameters include the region volume, number of mesh vertices, mesh vertex coordinates, center coordinates, number of faces, number of first-order adjacent faces of each face, and normal vector of the region mesh model; the contribution module 430 is specifically used for: The regional contribution weight of the regional mesh model is determined based on one or more of the following: regional vertex density, regional average curvature, and regional average distance. Wherein, the region vertex density is determined based on the region volume and the number of grid vertices corresponding to the region mesh model; the region average curvature is determined based on the number of faces and the curvature of all faces corresponding to the region mesh model; for each face curvature, the face curvature is determined based on the number of first-order adjacent faces and their normal vectors, as well as the normal vector of another adjacent face; the region average distance is determined based on the number of grid vertices, grid vertex coordinates, and center coordinates corresponding to the region mesh model.
[0091] In one embodiment, the geometry module 440 is specifically used for: The graph structure corresponding to the region grid model is input into the geometric feature extraction network to obtain the regional geometric features of the region grid model output by the geometric feature extraction network; the geometric feature extraction network is trained based on the sample graph structure; the geometric feature extraction network is used to extract geometric features.
[0092] In one embodiment, the image module 450 is specifically used for: The region projection image is cropped and reconstructed to obtain the reconstructed image corresponding to the region projection image; The reconstructed image is input into an image feature extraction network to obtain the region image features of the region projection image output by the image feature extraction network; the image feature extraction network is trained based on sample images; the image feature extraction network is used to perform image feature extraction.
[0093] In one embodiment, the quality assessment module 460 is specifically used for: For each of the aforementioned regional grid models, the regional geometric features and regional image features corresponding to the regional grid model are fused and stitched together to obtain the regional feature vector corresponding to the regional grid model. The contribution fine-tuning feature corresponding to the regional grid model is determined based on the product of the regional contribution weight and the regional feature vector. The quality assessment result of the target grid model is determined based on the contribution fine-tuning features corresponding to all the aforementioned regional grid models.
[0094] In one embodiment, the quality assessment module 460 is further configured to: The contribution fine-tuning features corresponding to all the regional grid models are input into the quality assessment neural network to obtain the quality assessment result output by the quality assessment neural network; the quality assessment neural network is trained based on sample features and corresponding labels.
[0095] The mesh model quality assessment device provided in this application divides the target mesh model into multiple regional mesh models; for each regional mesh model, it projects the regional mesh model to obtain the corresponding regional projection image; for each regional mesh model, it determines the regional contribution weight of the regional mesh model based on the target parameters of the regional mesh model; for each regional mesh model, it extracts geometric features to obtain the regional geometric features of the regional mesh model; for each regional projection image, it extracts image features to obtain the corresponding regional image features; and it determines the quality assessment result of the target mesh model based on all regional contribution weights, all regional geometric features, and all regional image features. The mesh model quality assessment provided in this application divides the target mesh model into multiple regions and determines the regional contribution weight for each region, reducing information redundancy in the assessment process, improving information utilization, and increasing the accuracy of mesh model quality assessment.
[0096] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute steps of the mesh model quality assessment method, such as: The target mesh model is divided into multiple region mesh models; For each of the aforementioned regional grid models, the regional grid model is projected to obtain the regional projection image corresponding to the regional grid model; For each of the aforementioned regional grid models, the regional contribution weight of the regional grid model is determined based on the target parameters of the regional grid model; For each of the aforementioned regional grid models, geometric features are extracted from the regional grid models to obtain the regional geometric features of the regional grid models; For each of the region projection images, image features are extracted from the region projection images to obtain the region image features corresponding to the region projection images; The quality assessment result of the target mesh model is determined based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0097] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0098] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the mesh model quality assessment method provided by the above methods, the method including: The target mesh model is divided into multiple region mesh models; For each of the aforementioned regional grid models, the regional grid model is projected to obtain the regional projection image corresponding to the regional grid model; For each of the aforementioned regional grid models, the regional contribution weight of the regional grid model is determined based on the target parameters of the regional grid model; For each of the aforementioned regional grid models, geometric features are extracted from the regional grid models to obtain the regional geometric features of the regional grid models; For each of the region projection images, image features are extracted from the region projection images to obtain the region image features corresponding to the region projection images; The quality assessment result of the target mesh model is determined based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0099] In another aspect, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a method for evaluating the quality of a mesh model provided by the methods described above, the method comprising: The target mesh model is divided into multiple region mesh models; For each of the aforementioned regional grid models, the regional grid model is projected to obtain the regional projection image corresponding to the regional grid model; For each of the aforementioned regional grid models, the regional contribution weight of the regional grid model is determined based on the target parameters of the regional grid model; For each of the aforementioned regional grid models, geometric features are extracted from the regional grid models to obtain the regional geometric features of the regional grid models; For each of the region projection images, image features are extracted from the region projection images to obtain the region image features corresponding to the region projection images; The quality assessment result of the target mesh model is determined based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
[0100] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).
[0101] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0102] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for quality assessment of a mesh model, characterized in that, include: The target mesh model is divided into multiple region mesh models; For each of the aforementioned regional grid models, the regional grid model is projected to obtain the regional projection image corresponding to the regional grid model; For each of the aforementioned regional grid models, the regional contribution weight of the regional grid model is determined based on the target parameters of the regional grid model; For each of the aforementioned regional grid models, geometric features are extracted from the regional grid models to obtain the regional geometric features of the regional grid models; For each of the region projection images, image features are extracted from the region projection images to obtain the region image features corresponding to the region projection images; The quality assessment result of the target mesh model is determined based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
2. The method for quality assessment of mesh models according to claim 1, characterized in that, The process of dividing the target mesh model into multiple region mesh models includes: The target mesh model is projected to obtain a target projection image corresponding to the target mesh model; the target projection image is divided into multiple image regions based on a preset number of regions and the extreme values of the mesh coordinates corresponding to the target mesh model; and the mesh model of each region is determined based on the projection coordinates of all the image regions and the mesh vertices of the target mesh model. Alternatively, the target grid model can be divided based on the preset number of regions and a preset clustering algorithm to obtain each region grid model; Alternatively, the grid vertices in the target grid model can be classified based on a point cloud segmentation algorithm, and the grid model of each region can be determined based on the classified grid vertices.
3. The method for quality assessment of mesh models according to claim 1, characterized in that, The regional projection image includes multiple preset viewpoint projection images; The step of projecting the region grid model to obtain the region projection image corresponding to the region grid model includes: For each preset viewpoint, the region grid model is projected onto the preset viewpoint using a projection tool to obtain a split-view projection image under the preset viewpoint.
4. The method for quality assessment of mesh models according to claim 1, characterized in that, The target parameters include the region volume, number of grid vertices, grid vertex coordinates, center coordinates, number of faces, number of first-order adjacent faces of each face, and normal vector of the region mesh model. The determination of the regional contribution weight of the regional grid model based on the target parameters of the regional grid model includes: The regional contribution weight of the regional mesh model is determined based on one or more of the following: regional vertex density, regional average curvature, and regional average distance. Wherein, the region vertex density is determined based on the region volume and the number of grid vertices corresponding to the region mesh model; the region average curvature is determined based on the number of faces and the curvature of all faces corresponding to the region mesh model; for each face curvature, the face curvature is determined based on the number of first-order adjacent faces and their normal vectors, as well as the normal vector of another adjacent face; the region average distance is determined based on the number of grid vertices, grid vertex coordinates, and center coordinates corresponding to the region mesh model.
5. The method for quality assessment of a mesh model according to claim 1, characterized in that, The step of extracting geometric features from the region grid model to obtain the region geometric features of the region grid model includes: The graph structure corresponding to the region grid model is input into the geometric feature extraction network to obtain the regional geometric features of the region grid model output by the geometric feature extraction network; the geometric feature extraction network is trained based on the sample graph structure; the geometric feature extraction network is used to extract geometric features.
6. The method for quality assessment of a mesh model according to claim 1, characterized in that, The step of extracting image features from the projected image of the region to obtain the region image features corresponding to the projected image includes: The region projection image is cropped and reconstructed to obtain the reconstructed image corresponding to the region projection image; The reconstructed image is input into an image feature extraction network to obtain the region image features of the region projection image output by the image feature extraction network; the image feature extraction network is trained based on sample images; the image feature extraction network is used to perform image feature extraction.
7. The method for quality assessment of a mesh model according to any one of claims 1 to 6, characterized in that, The process of determining the quality assessment result of the target mesh model based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions includes: For each of the aforementioned regional grid models, the regional geometric features and regional image features corresponding to the regional grid model are fused and stitched together to obtain the regional feature vector corresponding to the regional grid model. The contribution fine-tuning feature corresponding to the regional grid model is determined based on the product of the regional contribution weight and the regional feature vector. The quality assessment result of the target grid model is determined based on the contribution fine-tuning features corresponding to all the aforementioned regional grid models.
8. The method for quality assessment of a mesh model according to claim 7, characterized in that, The process of determining the quality assessment result of the target grid model based on the contribution fine-tuning features corresponding to all the regional grid models includes: The contribution fine-tuning features corresponding to all the regional grid models are input into the quality assessment neural network to obtain the quality assessment result output by the quality assessment neural network; the quality assessment neural network is trained based on sample features and corresponding labels.
9. A quality assessment device for a mesh model, characterized in that, include: The partitioning module is used to divide the target mesh model into multiple region mesh models; The projection module is used to project the region grid model onto each of the region grid models to obtain the region projection image corresponding to the region grid model. The contribution module is used to determine the regional contribution weight of each regional grid model based on the target parameters of the regional grid model. The geometry module is used to extract geometric features from each of the aforementioned regional mesh models to obtain the regional geometric features of the regional mesh models. The image module is used to extract image features from the projected images of each region to obtain the region image features corresponding to the projected images of the regions. The quality assessment module is used to determine the quality assessment result of the target mesh model based on the contribution weights of all the regions, the geometric features of all the regions, and the image features of all the regions.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the quality assessment method for the mesh model as described in any one of claims 1 to 8.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the quality assessment method for the mesh model as described in any one of claims 1 to 8.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the quality assessment method for the mesh model as described in any one of claims 1 to 8.