A method and apparatus for simplifying a three-dimensional model attribute image based on rate-distortion optimization

Abstract

This application discloses a method and apparatus for simplifying 3D model attribute images based on rate-distortion optimization. The method includes: acquiring a 3D model attribute image; dividing the entire attribute image into multiple image blocks; determining the rate-distortion curve of each image block; optimizing the downsampling rate of each image block using the Lagrange multiplier method, under a given pixel count constraint, based on the rate-distortion curve of each image block; and downsampling the corresponding image block using bicubic interpolation based on the downsampling rate of each image block to obtain a simplified attribute image. This application optimizes the downsampling rate of different regions of the attribute image to make the simplified attributed 3D model as close as possible to the original model in appearance. This optimization process is based not only on the features of the 2D attribute image but also considers the geometric features of the original and simplified 3D models, as well as the parameterization of the model surface to the attribute image. The attribute simplification method proposed in this application has universality.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for simplifying three-dimensional model attribute images based on rate-distortion optimization. Background Technology

[0002] Three-dimensional (3D) models are increasingly widely used in numerous fields such as computer-aided engineering (CAE), 3D games, and digital cities. Besides geometric and topological data, a 3D model often contains other attribute data. For example, in CAE computer simulation, attributes such as displacement, stress, temperature, and pressure may exist; in 3D rendering, attributes such as normals, materials, and textures may exist. In computer simulation, attribute data such as displacement, stress, temperature, and pressure are often presented as color-coded shading maps on the surface of the 3D model. This is similar to the texture images used in 3D model rendering. Texture images are also presented as color maps on the model surface. Therefore, they all belong to attribute data defined on the surface of a 3D model. The surface of a 3D model can be parameterized to a two-dimensional (2D) plane; correspondingly, the attribute data of the 3D model surface is parameterized into a two-dimensional image, which this application refers to as a 3D model attribute image, or simply attribute image.

[0003] To achieve high 3D rendering quality or computer simulation accuracy, many applications increasingly demand higher resolutions for 3D geometric models (hereinafter referred to as 3D models) and their attribute images. High-resolution 3D models and attribute images lead to a surge in data volume, placing enormous pressure on storage, transmission, and processing. On some lower-configuration platforms, such as mobile terminals, related applications can only store, transmit, and display relatively lightweight 3D models. Therefore, to meet the needs of lower-configuration platforms, it is often necessary to simplify the original 3D model and its attribute images, reducing the number of vertices in the 3D model and the number of pixels in the attribute images while maintaining model quality as much as possible.

[0004] Numerous papers have been published in the field of 3D model simplification. Initially, 3D model simplification algorithms aimed to maintain geometric fidelity while reducing the number of vertices; later, they also focused on reducing the offset of attribute images on the model surface to better preserve the appearance of attributed 3D models. In contrast, simplification of the attribute images themselves (or simply attribute simplification) has received little attention in the past. Despite the limited research, attribute simplification is crucial for reducing the memory footprint of attributes, especially GPU memory. One of the most direct methods for attribute image simplification is image retargeting. However, these methods only optimize the appearance of the adjusted 2D image, not its appearance on the 3D model surface.

[0005] Research in the field of 3D model simplification has been ongoing for over two decades, resulting in numerous published algorithms. Early 3D model simplification algorithms largely focused on maintaining the geometric fidelity of the simplified model. Subsequent algorithms have also considered properties beyond geometry, particularly texture coordinates, to better preserve the appearance of textured 3D models. However, none of these 3D model simplification algorithms have made any simplification to the texture image itself.

[0006] Texture warping was originally proposed to optimize texture space, but it can also be used to simplify texture images. Specifically, it can distort and uniformly downsample texture images to achieve a simplification effect.

[0007] Texture warping techniques optimize the surface parameterization of 3D models by adaptively deforming different regions of a texture image, thereby allocating more texture space to surface patches with more shading detail and / or higher parametric distortion. Existing techniques include: 1. Multi-layer free deformation of texture images based on metrics that consider user-specified importance and parametric distortion. 2. Building texture warping algorithms on Fourier analysis, balancing the spatial frequency distribution in the image through piecewise linear warping. 3. Integrating metrics of image frequency and parametric distortion into a single mapping, guiding the texture warping process through Laplacian smoothing-based relaxation. 4. Using a signal mapping metric tensor to capture the directionality of the signal and locally squeezing parameterization perpendicular to the signal gradient. This metric was later improved to provide higher sensitivity to signal detail.

[0008] A closely related but distinct field to texture simplification is texture compression. The former reduces the number of pixels in an image, while the latter compresses the bit representation of a given textured image.

[0009] Texture images can be compressed using general-purpose 2D image codecs, such as the JPEG codec. This compression method requires decompressing the texture image to its original RGB or RGBA arrays and sending the raw data arrays to the GPU for rendering. Furthermore, specific algorithms have been designed for GPU texture compression, facilitating efficient memory usage, fast decompression, and direct access to pixel data. Several standard methods have been developed for various platforms, including DXTC (DirectX Texture Compression), ETC (Ericsson Texture Compression), ASTC (Adaptive Scalable Texture Compression), PVRTC (PowerVR Texture Compression), and their variants. All of these methods are based on regular block subdivision of the texture image, using a fixed number of bits for each block.

[0010] When compressing texture images, the above method does not consider their presentation on a 3D surface, nor does it consider their interaction with other texture images mapped on the same surface.

[0011] There is currently no effective solution to the technical problem of simplifying 3D model attribute images in the existing technology. Summary of the Invention

[0012] The embodiments of this disclosure provide a method and apparatus for simplifying three-dimensional model attribute images based on rate distortion optimization, so as to at least solve the technical problems of simplifying three-dimensional model attribute images in the prior art.

[0013] According to one aspect of the present disclosure, a method for simplifying three-dimensional model attribute images based on rate-distortion optimization is provided, comprising:

[0014] Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block;

[0015] Based on the rate-distortion curve of each image patch, and given the total number of pixels, the downsampling rate of each image patch is optimized using the Lagrange multiplier method; and

[0016] Based on the downsampling rate of each image block, bicubic interpolation is used to downsample the corresponding image block to obtain a simplified attribute image.

[0017] According to another aspect of the present disclosure, a storage medium is also provided, the storage medium including a stored program, wherein, when the program is executed, a processor performs any of the methods described above.

[0018] According to another aspect of the present disclosure, a three-dimensional model attribute image simplification device based on rate-distortion optimization is also provided, comprising:

[0019] The rate-distortion module is used to acquire the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block.

[0020] The downsampling rate calculation module is used to optimize and solve for the downsampling rate of each image patch using the Lagrange multiplier method, given a total pixel count limit, based on the rate-distortion curve of each patch.

[0021] The interpolation module is used to downsample the corresponding image block using bicubic interpolation based on the downsampling rate of each image block to obtain a simplified attribute image.

[0022] According to another aspect of the present disclosure, a three-dimensional model attribute image simplification device based on rate-distortion optimization is also provided, comprising:

[0023] First processor; and

[0024] A first memory, connected to the first processor, is used to provide the first processor with instructions to perform the following processing steps:

[0025] Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block;

[0026] Based on the rate-distortion curve of each image patch, and given the total number of pixels, the downsampling rate of each image patch is optimized using the Lagrange multiplier method; and

[0027] Based on the downsampling rate of each image block, bicubic interpolation is used to downsample the corresponding image block to obtain a simplified attribute image.

[0028] This disclosure innovatively proposes an appearance-driven attribute image simplification method. This method works in conjunction with a 3D model simplification algorithm, assuming the 3D model has already been simplified, and then further simplifying the attribute image. Under the constraint of an overall pixel budget, the proposed method optimizes the downsampling rate of different regions of the attribute image to make the simplified attributed 3D model as close as possible in appearance to the original model. This optimization process is based not only on the features of the 2D attribute image but also considers the geometric features of the original and simplified 3D models, as well as the parameterization of the model surface to the attribute image. Furthermore, the attribute simplification method proposed in this application is universal and can work in conjunction with different 3D model simplification algorithms (such as those based on edge shrinkage, vertex deletion, vertex clustering, etc.). Attached Figure Description

[0029] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0030] Figure 1 This is a hardware structure block diagram of a computing device for implementing the method described in Embodiment 1 of this disclosure;

[0031] Figure 2 This is a schematic diagram of a three-dimensional model attribute image simplification system based on rate distortion optimization according to Embodiment 1 of this disclosure;

[0032] Figure 3 This is a flowchart illustrating the method for simplifying three-dimensional model attribute images based on rate-distortion optimization according to the first aspect of Embodiment 1 of this disclosure;

[0033] Figure 4This is a schematic diagram of the three-dimensional mesh model (shown in the top row) and the corresponding texture image (shown in the bottom row) used in Embodiment 1 of this disclosure;

[0034] Figure 5 This is a visual illustration of the results of texture simplification of Panda, Cartoon-man, and Earth using UNIFORM, WARP, and OURS according to Embodiment 1 of this disclosure; the algorithm name and downsampling rate are labeled below each model image. The original model is displayed in the last column for reference;

[0035] Figure 6 This is a simplified rate-distortion curve of the textures of Panda, House, Cartoon-man, Earth, Dragon, and Spiderman using UNIFORM, WARP, and OURS according to Embodiment 1 of this disclosure.

[0036] Figure 7 This is a visual illustration of the simplification results of 3D models and texture images of Panda, House, and Spiderman using SADO+UNIFORM and SADO+OURS according to Embodiment 1 of this disclosure; the algorithm name, model simplification rate, and texture image downsampling rate are marked below each model image. The original model is displayed in the last column for reference;

[0037] Figure 8 This is a schematic diagram of a three-dimensional model attribute image simplification device based on rate distortion optimization according to the first aspect of Embodiment 2 of this disclosure;

[0038] Figure 9 This is a schematic diagram of a three-dimensional model attribute image simplification device based on rate-distortion optimization according to the second aspect of Embodiment 2 of this disclosure. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] Example 1

[0042] According to this embodiment, a method embodiment of a three-dimensional model attribute image simplification method based on rate distortion optimization is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] The method embodiments provided in this example can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Figure 1 A hardware block diagram of a computing device for implementing a rate-distortion optimization-based method for simplifying 3D model attribute images is shown. Figure 1 As shown, a computing device may include one or more processors (processors may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), memory for storing data, transmission devices for communication functions, and input / output interfaces. The memory, transmission devices, and input / output interfaces are connected to the processor via a bus. In addition, it may also include a display, keyboard, and cursor control device connected to the input / output interfaces. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, a computing device may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0044] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to as "data processing circuits" in this application. The data processing circuit can be embodied, in whole or in part, as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuit can be a single, independent processing module, or it can be integrated, in whole or in part, into any other element in a computing device. As involved in the embodiments of this disclosure, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0045] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the rate-distortion-optimized 3D model attribute image simplification method in this embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the rate-distortion-optimized 3D model attribute image simplification method of the aforementioned application. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the computing device's communications provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0047] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of the computing device.

[0048] It should be noted here that, in some optional embodiments, the above... Figure 1 The computing device shown may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that... Figure 1 This is only one instance of a specific particular instance, and is intended to illustrate the types of components that may exist in the aforementioned computing devices.

[0049] This application innovatively proposes a method and apparatus for simplifying 3D model attribute images based on rate-distortion optimization. A 3D model attribute image is an image formed by parameterizing the surface attributes of a 3D model into a 2D space. This application works in conjunction with the geometric simplification of the 3D model to optimally maintain the quality of the original model with surface attributes while reducing the pixel budget. The overall strategy of this application is to divide the attribute image into multiple image blocks, and for a given total pixel budget, to optimally determine the downsampling rate of each image block. Specifically, this application first proposes a novel attribute distortion metric to measure the difference in quality between the 3D model before and after simplification. This application comprehensively considers features such as pixel values, 3D surface geometry, and attribute mapping. Based on this attribute distortion metric, this application determines the downsampling rate of each image block through rate-distortion optimization. Experimental results qualitatively and quantitatively demonstrate the superior performance of the proposed attribute image simplification algorithm. This application uses texture images as a specific attribute image for algorithm illustration and experimental verification.

[0050] Regardless of whether a general image codec or a specific GPU texture codec is used to compress texture images, the size of the texture data transmitted to the GPU is always linearly proportional to the number of pixels in the original image, with the specific ratio depending on the codec used. Therefore, to reduce the GPU memory footprint of texture images, it is necessary to first reduce the number of pixels in the texture image (i.e., simplify the texture). The texture image simplification algorithm and the 3D model simplification algorithm proposed in this application work together. Assuming that the 3D model has already been simplified, the texture image is further simplified on this basis, preserving the appearance of the model to the maximum extent given a pixel budget.

[0051] For a texture image of size W×H, the goal of this application is to reduce the number of pixels while preserving significant texture features on the surface of a 3D model. A simple approach is to uniformly downsample the entire image. However, this cannot distinguish image regions with different degrees of feature saliency. Texture warping techniques were originally proposed for parametric optimization and can also be used for texture image simplification tasks. That is, this application can warp a texture image and then uniformly downsample it. However, it is not fully adaptive because the texture image is warped only once, and then uniform downsampling is performed for any given pixel budget. Moreover, texture warping modifies the texture coordinate properties of vertices and deforms the texture image, which may be inconvenient for some applications (e.g., texture redesign). In contrast, this application proposes a general texture image simplification scheme. It takes into account the characteristics of both the texture image and the 3D model, adapts to any given pixel budget, does not deform the texture image, and does not modify the texture coordinate properties of vertices.

[0052] Figure 2This is a schematic diagram of the 3D model attribute image simplification system based on rate-distortion optimization as described in this embodiment. (Refer to...) Figure 2 As shown, the system includes: a front-end portable electronic terminal 100 (e.g., a laptop computer), a computing device 200 based on a rate-distortion optimization-based method for simplifying 3D model attribute images, and a cloud server 300. It should be noted that the computing device 200 based on a rate-distortion optimization-based method for simplifying 3D model attribute images in the system can utilize the hardware structure described above.

[0053] Under the aforementioned operating environment, according to the first aspect of this embodiment, a method for simplifying 3D model attribute images based on rate-distortion optimization is provided. This method consists of... Figure 2 The computing device 200 shown implements the rate-distortion optimization-based method for simplifying 3D model attribute images. Figure 3 A flowchart illustrating the method is shown below. (Refer to...) Figure 3 As shown, the method includes:

[0054] S302: Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block;

[0055] S304: Based on the rate-distortion curve of each image patch, and within a given total pixel count constraint, optimize the downsampling rate of each image patch using the Lagrange multiplier method; and

[0056] S306: Based on the downsampling rate of each image block, use bicubic interpolation to downsample the corresponding image block to obtain a simplified attribute image.

[0057] This application divides a texture image into nb×nb equal-sized rectangular blocks and downsamples each block. The application optimally determines the downsampling rate for each image block to minimize overall texture distortion within the total pixel budget. Specifically, the application first proposes a texture distortion metric; then, based on this metric, it proposes a rate-distortion optimization process to determine the downsampling rate for each block. Finally, the application uses bicubic interpolation to downsample each block at a determined ratio. Therefore, the simplified texture image is actually an nb×nb array, where each element is a downsampled image block. Different elements may have different sizes because their corresponding downsampling rates are not necessarily the same.

[0058] It is worth noting that although the method proposed in this application aims to reduce pixel counts, it can be easily extended for texture image compression with adaptive block quality control. For example, based on a general JPEG codec, this application can determine the quality parameters for each block, but replace the feature point count of each block with its downsampled pixel count.

[0059] Texture distortion should not be measured by directly comparing two texture images in two-dimensional space, because texture images are ultimately presented on a three-dimensional surface, not as separate two-dimensional images. Measuring texture distortion requires considering the content of the texture images, the texture mapping of the two 3D models (before and after simplification), and their geometry. One method is to compare rendered images of two texture-mapped models on a sampled set of views. However, this is not always accurate because: 1) the degree of distortion may be uneven across different parts of the 3D model surface during view projection; and 2) it is difficult to perform a fair, concise, and sufficient view sampling. Therefore, this application aims to find a more universally applicable method for measuring texture distortion. Inspired by the Metro tool for comparing the geometry of 3D surfaces, this application samples two texture-mapped 3D surfaces and calculates the shading difference between the two sample point sets, explained in detail below.

[0060] Given two textured models, the proposed metric integrates the shading differences between spatially close point pairs on the surfaces of the two models. The two textured models are represented as S =<M,I> and S' =<M’,I’> Here, M and M' are two 3D triangular mesh models, and I and I' are two corresponding texture images. The function F(·) parameterizes M to I, and the function F'(·) parameterizes M' to I'. The appearance distance d(S, S') from S to S' is defined as:

[0061]

[0062] in, And n(·) represents the normal vector at a point on the surface, meaning that c(p) is the point in M' that is closest to p and whose normal angle is within 90 degrees. Finally, this application defines the difference in appearance between S and S' as a symmetric distance:

[0063]

[0064] Generally speaking, M, M', F(·), and F'(·) do not have closed-form mathematical expressions. Therefore, the implementation of this application is based on discrete sampling of M and M' and piecewise linear approximation of F(·) and F'(·). Specifically, this application performs the same barycentric coordinate sampling on all triangles on M and M', using a barycentric coordinate set L = {l1, l2, ..., l...} m}. Among them, l i =(λ 1，i， λ 2，i , λ 3，i), i = 1, 2, ..., m. That is to say, in each triangular facet Δ of M M In (v1, v2, v3), m points p are sampled. i =l i ·Δ M Let i = 1, ..., m. Assume Δ M The triangle Δ parameterized in texture space I = (t1, t2, t3), this application obtains p by linearly interpolating the texture coordinates of the three corner points. i The texture coordinates of the point, i.e.: F(p) i )=l i ·Δ I Let i = 1, ..., m. For M', this application uses the same method for surface sampling and texture coordinate calculation of the sampling points. In the following description, this application also uses M and M' to represent the discrete sampling point sets on the surfaces of the two models.

[0065] Furthermore, image regions of the same size can be mapped to surface patches of different sizes. Image regions that map to larger surface patches tend to have a greater impact on the final model appearance. Therefore, this application introduces a weight w(p) for each point p, resulting in the updated d(S, S') defined as follows:

[0066]

[0067] Assuming ΔMp is the triangular facet containing p on M, and ΔTp is the corresponding triangular facet in the texture domain, and A(·) gives the area of ​​the triangle, this application defines w(p) = A(ΔMp) / A(ΔpT). Thus, if a small triangle in the texture image maps to a large triangle on the 3D surface, thus significantly contributing to the model's appearance, its texture content will still be preferentially preserved.

[0068] If M and I are the original model and texture, and M' and I' are the simplified model and texture, then Equation 2 (combined with Equation 3) gives the simplified texture distortion metric.

[0069] To determine the optimal downsampling rate for all texture image patches, this application proposes using the Lagrange multiplier technique.

[0070] Assume this application has K = nb × nb image blocks. For the i-th (i = 1, ..., K) image block, it is arranged according to... The ratio of downsampling is used to obtain the number of pixels and texture distortion, respectively denoted as . and The goal of this application is to find a set In meeting pixel budget constraints Minimize overall texture distortion under the given conditions This problem can be solved by minimizing the following Lagrange cost function.

[0071]

[0072] Therefore, this application needs to calculate the corresponding values ​​for a series of ni values. and Value. For a specific ni value, this application has In this application, the i-th (1≤i≤K) image patch before and after downsampling is denoted as B. i，0 and Assume B i，0 Patch P mapped onto the surface of the original model i Mapped onto the patch P′ on the simplified model surface i This application uses equations 2 and 3 to calculate the distortion.

[0073] Regarding the fineness of texture block division, taking into account both rate-distortion performance and computational efficiency, this application adopts nb=8 based on experience.

[0074] Experiments and Analysis

[0075] This application used six textured models in its experiments. These are Panda, House, Cartoon-man, Earth, Spider-man, and Dragon, and their geometric meshes and texture images are as follows: Figure 4 As shown. The original Cartoon-Man model is from Intel, the original House model is from http: / / www.turbosquid.com, the original Dragon model is from https: / / graphics.stanford.edu / data / 3Dscanrep / , and other original models are from http: / / www.cgjoy.com. Note that for ease of use, this application preprocessed the original geometric models and original texture images, and used software tools to design textures for the Dragon. Table 1 provides more information about these processed models and textures.

[0076] Table 1. Relevant information about the models used in the experiments of this application.

[0077]

[0078] To measure the distortion caused by simplification, this application renders the original and simplified textured 3D models from a randomly selected set of viewpoints using the same lighting. At each viewpoint, this application compares the rendered images of the original and simplified models using the well-known SSIM (Structural Similarity) index. This application uses the average SSIM index of all viewpoint samples to measure the quality of the simplified model.

[0079] 1. Simplified results of texture image alone

[0080] In this experiment, this application does not simplify the 3D model in any way, but only simplifies the texture image. This application is compared experimentally with two benchmark algorithms: the first is Uniform Image Downsampling (UNIFORM), and the second is Texture Distortion Plus Uniform Image Downsampling (WARP). The method of this application is referred to as OURS. For all UNIFORM, WARP, and OURS algorithms, this application upsamples the simplified image (image patch) to restore a texture image of the same size as the original image for visual and quantitative comparison. In these algorithms, both upsampling and downsampling are performed using bicubic interpolation.

[0081] This application runs UNIFORM, WARP, and OURS on the test model to simplify the texture image while keeping the original mesh unchanged. Partial visual results are shown... Figure 2 In the figures, the algorithm name and downsampling rate are labeled below each model image. For Panda and Cartoon-man, magnified local regions are also shown in the figures. Figure 5 As can be seen, for the selected downsampling rate, OURS outperforms other methods in preserving the model appearance.

[0082] Furthermore, this application measured the SSIM metrics of six models at different texture downsampling rates, and presented the results. Figure 6 In the middle. For example Figure 6 As shown, OURS has a significant rate-distortion advantage over UNIFORM and WARP.

[0083] The experiment was conducted on a desktop computer with 12GB of RAM and a 3.4GHz Intel(R)Xeon(R) CPU. Table 2 shows the runtimes of Earth, Panda, Cartoon-man, House, Spiderman, and Dragon at downsampling rates of 1 / 100, 1 / 196, 1 / 196, 1 / 100, 1 / 196, and 1 / 196, respectively. This table shows that UNIFORM is the most efficient, while WARP is the least efficient. WARP's low efficiency mainly stems from the texture distortion process before uniform downsampling.

[0084] Table 2. Runtime statistics for texture image simplification (unit: seconds)

[0085] Model UNIFORM WARP OURS Earth 0.068 391.262 4.918 Panda 0.150 742.674 18.333 Cartoon-man 0.138 750.183 54.945 House 0.245 747.732 105.485 Spiderman 0.137 148.129 4.654 Dragon 0.348 52600.110 2238.844

[0086] 2. 3D model simplification + texture image simplification results

[0087] In this experiment, this application first simplifies the 3D model, and then simplifies the texture image. For the 3D model simplification, this application uses the SADO algorithm (Mesh Simplification with Appearance-Driven Optimizations). For the texture image simplification, this application uses UNIFORM and OURS respectively. Experiments were conducted on three textured models (Panda, House, and Spiderman), and some results are shown in [the table / document / etc.]. Figure 7 In the image, the algorithm name, 3D model simplification rate, and texture image downsampling rate are labeled below each model's image. The original model is displayed in the last column for reference. Figure 7 As shown, SADO+OURS performs better than SADO+UNIFORM in preserving the appearance of the model. This illustrates two points: 1) The OURS texture image simplification method is superior to UNIFORM in preserving important texture details; 2) OURS texture image simplification can work well with 3D model simplification, reducing the amount of model and texture data while preserving the appearance of the original textured model as much as possible.

[0088] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0089] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0090] Example 2

[0091] Figure 8 A rate-distortion optimized 3D model attribute image simplification apparatus 500 according to a first aspect of this embodiment is shown, which corresponds to the method described according to the first aspect of Embodiment 1. Reference Figure 8 As shown, the device 500 includes:

[0092] Rate distortion module 510 is used to acquire the attribute image of the three-dimensional model, divide the entire attribute image into multiple image blocks, and determine the rate distortion curve of each image block.

[0093] The downsampling rate calculation module 520 is used to optimize and solve for the downsampling rate of each image patch using the Lagrange multiplier method, given a total pixel count limit, based on the rate-distortion curve of each image patch; and

[0094] The interpolation module 530 is used to downsample the corresponding image block using bicubic interpolation based on the downsampling rate of each image block to obtain a simplified attribute image.

[0095] According to another aspect of the embodiments of this disclosure, such as Figure 9 As shown, a 3D model attribute image simplification device 700 based on rate-distortion optimization is also provided, comprising:

[0096] First processor 710; and

[0097] A first memory 720, connected to the first processor, is used to provide the first processor with instructions to perform the following processing steps:

[0098] Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block;

[0099] Based on the rate-distortion curve of each image patch, and given the total number of pixels, the downsampling rate of each image patch is optimized using the Lagrange multiplier method; and

[0100] Based on the downsampling rate of each image block, bicubic interpolation is used to downsample the corresponding image block to obtain a simplified attribute image.

[0101] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0102] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0107] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A rate-distortion optimization based method for simplifying a three-dimensional model attribute image, characterized in that, include: Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block; Based on the rate-distortion curve of each image patch, and given the total number of pixels, the downsampling rate of each image patch is optimized using the Lagrange multiplier method. as well as Based on the downsampling rate of each image block, bicubic interpolation is used to downsample the corresponding image block to obtain a simplified attribute image; Determining the rate-distortion curve for each image patch includes: For a given two textured 3D models, the texture distortion metric is calculated based on the shading difference between spatially close point pairs on the surfaces of the two models, specifically as follows: The two textured mapping models are represented as S =<M,I> and S'=<M’,I’> Where M and M' are two 3D triangular mesh models, and I and I' are two corresponding texture images. The function F(·) parameterizes M' to I'. The appearance distance d(S, S') from S to S' is defined as: ； （1） in, And n(·) represents the normal vector at a point on the surface, that is, c(p) is the point in M' that is closest to p and whose normal angle is within 90 degrees; The difference in appearance between S and S' is a symmetrical distance: ； （2） Assigning a weight w(p) to each point p, we obtain the updated d(S, S') defined as follows: ；（3） in, Let p be the triangular facet on M. It is the corresponding triangular facet in the texture domain, and A(·) gives the area of ​​the triangle. ; Combining equation (2) with equation (3) generates a simplified texture distortion metric: ； in, and These are the i-th image blocks before and after downsampling, respectively. and They are respectively Patches mapped onto the surfaces of the original and simplified models.

2. The method according to claim 1, characterized in that, The three-dimensional model attribute image is an image formed by parameterizing the surface attributes of the three-dimensional model into a two-dimensional space.

3. The method according to claim 2, characterized in that, The attribute image of the three-dimensional model is a distorted texture image.

4. The method according to claim 2 or 3, characterized in that, Before obtaining the 3D model attribute image, the following steps are included: The 3D model is simplified using an appearance-driven optimization simplification algorithm.

5. The method according to claim 1, characterized in that, After obtaining the simplified attribute image, it includes: The simplified attribute image is upsampled to restore a texture image of the same size as the original image for visual and quantitative comparison.

6. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, the method described in any one of claims 1 to 5 is performed by a processor.

7. A rate-distortion optimization based three-dimensional model attribute image simplification apparatus, characterized by, include: The rate-distortion module is used to acquire the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block. The downsampling rate calculation module is used to optimize and solve the downsampling rate of each image patch using the Lagrange multiplier method, based on the rate-distortion curve of each image patch and within a given limit on the total number of pixels. Determining the rate-distortion curve for each image patch includes: for two given textured 3D models, calculating a texture distortion metric based on the shading difference between spatially close point pairs on the surfaces of the two models, specifically: Represent the two textured mapping models as S =<M,I> and S'=<M’,I’> Where M and M' are two 3D triangular mesh models, and I and I' are two corresponding texture images. The function F(·) parameterizes M' to I'. The appearance distance d(S, S') from S to S' is defined as: ； （1） in, And n(·) represents the normal vector at a point on the surface, that is, c(p) is the point in M' that is closest to p and whose normal angle is within 90 degrees; The difference in appearance between S and S' is a symmetrical distance: ； （2） Assigning a weight w(p) to each point p, we obtain the updated d(S, S') defined as follows: ；（3） in, Let p be the triangular facet on M. It is the corresponding triangular facet in the texture domain, and A(·) gives the area of ​​the triangle. ; Combining equation (2) with equation (3) generates a simplified texture distortion metric: ； in, and These are the i-th image blocks before and after downsampling, respectively. and They are respectively Patches mapped onto the surfaces of the original and simplified models; and The interpolation module is used to downsample the corresponding image block using bicubic interpolation based on the downsampling rate of each image block to obtain a simplified attribute image.

8. The apparatus according to claim 7, characterized in that, The three-dimensional model attribute image is an image formed by parameterizing the surface attributes of the three-dimensional model into a two-dimensional space.

9. A rate-distortion optimization based three-dimensional model attribute image simplification apparatus, characterized by, include: First processor; as well as A first memory, connected to the first processor, is used to provide the first processor with instructions to perform the following processing steps: Obtain the attribute image of the 3D model, divide the entire attribute image into multiple image blocks, and determine the rate-distortion curve of each image block; Based on the rate-distortion curve of each image patch, and given the total number of pixels, the downsampling rate of each image patch is optimized using the Lagrange multiplier method. as well as Based on the downsampling rate of each image block, bicubic interpolation is used to downsample the corresponding image block to obtain a simplified attribute image; Determining the rate-distortion curve for each image patch includes: For a given two textured 3D models, the texture distortion metric is calculated based on the shading difference between spatially close point pairs on the surfaces of the two models, specifically as follows: Represent the two textured mapping models as S =<M,I> and S'=<M’,I’> Where M and M' are two 3D triangular mesh models, and I and I' are two corresponding texture images. The function F(·) parameterizes M' to I'. The appearance distance d(S, S') from S to S' is defined as: ； （1） in, And n(·) represents the normal vector at a point on the surface, that is, c(p) is the point in M' that is closest to p and whose normal angle is within 90 degrees; The difference in appearance between S and S' is a symmetrical distance: ； （2） Assigning a weight w(p) to each point p, we obtain the updated d(S, S') defined as follows: ；（3） in, Let p be the triangular facet on M. It is the corresponding triangular facet in the texture domain, and A(·) gives the area of ​​the triangle. ; Combining equation (2) with equation (3) generates a simplified texture distortion metric: ； in, and These are the i-th image blocks before and after downsampling, respectively. and They are respectively Patches mapped onto the surfaces of the original and simplified models.

Images