A sampling-based method for objective quality assessment of meshes.

The sampling-based method subdivides polygonal meshes into triangular faces, samples points, and calculates geometric and attribute distortions to address the limitations of existing point-based methods, offering a more general and efficient quality assessment for meshes.

JP2024545008AActive Publication Date: 2024-12-05TENCENT AMERICA LLC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024529237
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-03-27
Filing Date
2023-03-28
Publication Date
2024-12-05
Estimated Expiration
2043-03-28

AI Technical Summary

Technical Problem

Current point-based quality assessment methods for meshes are limited in their generality, applicable only to triangular meshes, and lack efficiency and accuracy in calculating quality metrics for polygonal meshes, particularly those with quadrilateral faces, and are computationally complex.

Method used

A sampling-based objective quality assessment method that subdivides polygonal meshes into triangular faces, samples points on these faces, and calculates geometric and attribute distortions using point clouds to evaluate quality metrics such as MSE and PSNR.

Benefits of technology

The method provides a more general and efficient assessment of mesh quality, applicable to various polygonal meshes, with improved accuracy and reduced computational complexity, enabling effective quality evaluation for both static and dynamic meshes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024545008000020
    Figure 2024545008000020
  • Figure 2024545008000021
    Figure 2024545008000021
  • Figure 2024545008000022
    Figure 2024545008000022
Patent Text Reader

Abstract

The method includes receiving, at an encoder, a first original polygonal mesh and a second polygonal mesh, where the first original polygonal mesh is an original polygonal mesh and the second polygonal mesh is a distorted polygonal mesh; converting the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing a plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces; sampling a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes; generating at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled plurality of points; and calculating a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on the at least first sampled point cloud.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 338,342, filed May 4, 2022, and U.S. Patent Application No. 18 / 190,610, filed March 27, 2023, the disclosures of which are incorporated by reference herein in their entireties.

[0002] FIELD OF THE DISCLOSURE This disclosure relates to a set of advanced video coding techniques, and more particularly, to a sampling-based objective quality assessment method for meshes. [Background technology]

[0003] In the MPEG CFP for dynamic mesh compression, a point-based sampling metric is proposed to assess the quality of distorted triangular meshes.

[0004] Current point-based quality assessment methods have several limitations. First, they lack the generality to handle various polygonal meshes and are only applicable to triangular meshes, where each face of a mesh can have more than four edges, e.g., meshes with quadrilateral faces. Second, the sampling method from triangular meshes to point clouds may be suboptimal. Third, computing a quality metric from two transformed point clouds may be less accurate in terms of correlation with subjective quality and less efficient in terms of computational complexity. Summary of the Invention [Problem to be solved by the invention]

[0005] The following presents a simplified summary of one or more embodiments of the present disclosure in order to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments, and is not intended to identify key or critical elements of all embodiments or to delineate the scope of any or all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0006] The present disclosure provides a sampling-based objective quality assessment method for meshes. [Means for solving the problem]

[0007] According to some embodiments, a method is provided that is executed by at least one processor. The method includes receiving, at an encoder, a first polygonal mesh and a second polygonal mesh, the first polygonal mesh being an original polygonal mesh and the second polygonal mesh being a distorted polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces. The method further includes converting the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective triangular mesh of the two or more triangular meshes. The method further includes sampling a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes. The method further includes generating at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled plurality of points. The method further includes calculating a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on at least the first sampled point cloud.

[0008] According to some embodiments, an apparatus includes at least one memory configured to store program code and at least one processor configured to read the program code and to operate as directed by the program code. The program code includes receiving code configured to cause the at least one processor to receive, at an encoder, a first polygonal mesh and a second polygonal mesh, where the first polygonal mesh is an original polygonal mesh and the second polygonal mesh is a distorted polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces. The program code further includes transforming code configured to cause the at least one processor to transform the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective triangular mesh of the two or more triangular meshes. The program code further includes sampling code configured to cause the at least one processor to sample a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes. The program code further includes generation code configured to cause the at least one processor to generate at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled plurality of points. The program code further includes calculation code configured to cause the at least one processor to calculate a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on at least the first sampled point cloud.

[0009] According to some embodiments, a non-transitory computer-readable storage medium stores instructions that, when executed by at least one processor, cause the at least one processor to receive, at an encoder, a first polygonal mesh and a second polygonal mesh, where the first polygonal mesh is an original polygonal mesh and the second polygonal mesh is a distorted polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces. The instructions further cause the at least one processor to convert the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective triangular mesh of the two or more triangular meshes. The instructions further cause the at least one processor to sample a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes. The instructions further cause the at least one processor to generate at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled points. The instructions further cause the at least one processor to calculate a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on the at least first sampled point cloud.

[0010] Further embodiments will be set forth in the description which follows, and in part will be obvious from the description, and / or may be learned by practice of presented embodiments of the present disclosure.

[0011] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to some embodiments. [Diagram 2]FIG. 1 is a schematic diagram of a simplified block diagram of a streaming system, according to some embodiments. [Diagram 3] 1 is a framework for objective quality assessment for triangle meshes, according to some embodiments. [Figure 4A] 1 illustrates various frameworks for objective quality assessment for polygonal meshes, according to some embodiments. [Figure 4B] 1 illustrates various frameworks for objective quality assessment for polygonal meshes, according to some embodiments. [Figure 4C] 1 illustrates various frameworks for objective quality assessment for polygonal meshes, according to some embodiments. [Diagram 5] 1 is a diagram of point-to-point distance (D1) and point-to-surface distance (D2) according to some embodiments. [Figure 6] FIG. 2 is a diagram illustrating subdividing a polygon into triangles according to some embodiments. [Figure 7] FIG. 13 is an illustration of uniform sampling within a triangular plane, according to some embodiments. [Figure 8] 1 is an operational flowchart illustrating steps performed by a program for a sampling-based objective quality assessment method for meshes, according to some embodiments. [Figure 9] FIG. 1 illustrates a computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following detailed description of the exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0014] A mesh may contain a number of polygons that describe the surface of a volumetric object. Information about its vertices in 3D space and how the vertices are connected may define each polygon, referred to as connectivity information. Vertex attributes such as color, normal, etc. may be associated with the mesh vertices. Attributes may also be associated with the mesh's surface by utilizing mapping information that parameterizes the mesh with 2D attribute maps. Such mappings may be defined using a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. 2D attribute maps may be used to store high-resolution attribute information such as texture, normal, displacement, etc. The high-resolution attribute information may be used for various purposes such as texture mapping and shading.

[0015] This disclosure provides a sampling-based objective quality assessment method for meshes. The quality metric used is based on MSE (Mean Squared Error) or PSNR (Peak Signal-to-Noise Ratio). Similar methods can be extended to other quality metrics such as SSIM, MS-SSIM, etc.

[0016] With reference to FIGS. 1-2, an embodiment of the present disclosure for implementing the encoding and decoding structures of the present disclosure will be described.

[0017] 1 shows a simplified block diagram of a communication system 100 according to one embodiment of the present disclosure. The system 100 may include at least two terminals 110, 120 interconnected via a network 150. In the case of a unidirectional transmission of data, the first terminal 110 may code video data, which may include mesh data, at a local location for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the coded video data of the other terminal from the network 150, decode the coded data, and display the recovered video data. The unidirectional data transmission may be common in media provision applications, etc.

[0018] 1 shows a second pair of terminals 130, 140 provided to support two-way transmission of coded video, such as might occur during a video conference. For two-way transmission of data, each terminal 130, 140 can code video data captured at a local location for transmission to the other terminal over network 150. Each terminal 130, 140 can also receive coded video data transmitted by the other terminal, can decode the coded data, and can display the recovered video data on a local display device.

[0019] In FIG. 1, the terminals 110-140 may be, for example, servers, personal computers, and smartphones, and / or any other type of terminal. For example, the terminals (110-140) may be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network 150 represents any number of networks that convey coded video data between the terminals 110-140, including, for example, wired and / or wireless communication networks. The communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of the network 150 may not be important to the operation of the present disclosure unless described below herein.

[0020] 2 shows an arrangement of video encoders and decoders in a streaming environment as an example of an application for the disclosed subject matter. The disclosed subject matter may be used in other video-enabled applications including, for example, video conferencing, digital television, storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0021] 2, the streaming system 200 may include a capture subsystem 213 that includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0022] The video source 201 may create a stream 202 including, for example, a 3D mesh and metadata associated with the 3D mesh. The video source 201 may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., a digital camera) and a computing device configured to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. The sample stream 202, which may have a high amount of data compared to an encoded video bitstream, may be processed by an encoder 203 coupled to the video source 201. The encoder 203 may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoder 203 may also further generate an encoded video bitstream 204. The encoded video bitstream 204, which may have a low amount of data compared to the uncompressed stream 202, may be stored on a streaming server 205 for future use. One or more streaming clients 206 can access the streaming server 205 to retrieve a video bitstream 209 , which may be a copy of the encoded video bitstream 204 .

[0023] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 may, for example, decode a video bitstream 209, which is an incoming copy of the encoded video bitstream 204, and create an outgoing video sample stream 211 that may be rendered on a display 212 or another rendering device (not shown). In some streaming systems, the video bitstreams 204, 209 may be encoded according to a particular video coding / compression standard.

[0024] In the MPEG CFP for dynamic mesh compression, a point-based sampling metric is proposed to assess the quality of distorted triangular meshes.

[0025] As shown in Fig. 3, the objective quality assessment framework can be described as follows: First, both the original mesh 301 and the distorted mesh 303 may be transformed into point clouds 302 and 304 by sampling points on the mesh triangles. Then, an objective quality metric on the point clouds 302 and 304 may be applied to calculate the geometric and attribute distortions 305 of the transformed point clouds.

[0026] A point cloud may be created by performing ray casting in the axial (x,y,z) directions according to the triangle normal. A hit test determines if the cast ray hits a triangle, then the color is obtained by barycentric interpolation (to determine the UV coordinates of the point) and then by bilinear interpolation (to get the RGB values ​​from a texture map).

[0027] The normal of a triangle is computed as the cross product of its two edges, possibly normalized to have unit length: all points obtained by sampling the triangle inherit that normal vector.

[0028] A geometric distortion metric may be used to assess the objective quality of a point cloud sampled from a distorted mesh given a point cloud sampled from an original mesh.

[0029] Let A and B be the original and compressed point clouds obtained from the sampling procedure described above, respectively. B,A Consider evaluating the compression error, denoted as: The steps to calculate both the point-to-point error (D1) and the point-to-face error (D2) of the geometric error are summarized below and shown in Figure 5.

[0030] Point b in point group B i , i.e., for each black point in Fig. 5, the corresponding point a in point set A j , i.e., the red points in Fig. 5. Nearest neighbors are used to locate the corresponding points. In detail, to reduce the computational complexity, a KD tree search is used to perform the nearest neighbor search.

[0031] Identified point a in the reference point group A j Let b be the point in point group B. i The length of the error vector is the point-to-point error, i.e.

number

[0032] The distance between any two points i∈B

number

number

[0033] Then, the D1 PSNR value is

number

number

number

number

[0034] Normal direction N j Project the error vector E(i,j) along the

number

number

[0035] The point-to-surface error (D2) of the entire point cloud is then calculated as:

number

[0036] Then, the D2 PSNR value is

number

number

number

number

[0037] The attribute PSNR can be calculated as follows:

number

number

number

number

number

[0038] For color attributes, PSNR is calculated in the YUV domain. The conversion from RGB space to YUV space is done using ITU-R BT.709. The peak value p for PSNR calculation is 255.

[0039] The proposed methods may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), the encoder, and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0040] In this disclosure, several methods are proposed. The methods may be applied individually or in any form of combination. It is noted that the methods may be applied to both static and dynamic meshes, where dynamic meshes may have time-varying geometry and attributes.

[0041] First, the method applied to triangular meshes may be extended to polygonal meshes. Polygonal meshes may have faces with four or more edges. A framework according to some embodiments may be shown in FIG. 4A. In this case, an input original polygonal mesh 401 may be converted to a triangular mesh 406 by first subdividing each of the polygons into triangles while maintaining the orientation of the original faces. The same operation may be performed to convert a distorted polygonal mesh 403 to a distorted triangular mesh 407. Two sampled point clouds 402 and 404 are then created, and quality metrics for the sampled point clouds 402 and 404 are used to calculate the peak signal-to-noise ratio 405 of the geometry and attributes.

[0042] Specifically, a polygonal face is a set of vertices (V1, V2, …, V n ), where n>=3 is the degree of the polygon (number of edges). For example, for triangular faces, n=3, and for quadrilateral faces, n=4. A polygonal face of degree n may be subdivided into n-2 triangles. The subdivision of a polygonal face into triangular faces may be performed in different ways.

[0043] In some embodiments, the triangles subdivided from the polygon are (V1, V2, V3), (V1, V3, V4), ..., (V1, V n-1 , V n ). For example, a quadrilateral face with (A, B, C, D) may be divided into two triangles (A, B, C) and (A, C, D). A pentagon with (A, B, C, D, E) may be divided into three triangles (A, B, C), (A, C, D), and (A, D, E) as shown in FIG. 6.

[0044] In some embodiments, the triangles subdivided from a polygon (where n is an even number) are (V1, V2, V3), (V3, V4, V5), ..., (V n-1 , V n , V1), and a method of further subdividing the interior polygon (V1, V3, V5, …) into triangles can be applied. Alternatively, the triangles subdivided from the polygon (where n is an odd number) can be written as (V1, V2, V3), (V3, V4, V5), …, (V n-2 , V n-1 , V n ), which can further apply methods to sub-divide the interior polygons (V1, V3, V5, ...) into triangles. For example, a quadrilateral face with (A, B, C, D) may be divided into two triangles (A, B, C) and (C, D, A). A pentagon with (A, B, C, D, E) may be divided into three triangles (A, B, C), (C, D, E), and (E, A, C).

[0045] To convert the triangle mesh into a point cloud, different sampling strategies may be applied.

[0046] In some embodiments, for each triangle on the mesh, uniform sampling (given the sampling step as an input parameter) may be applied to the 2D plane of the triangle.

[0047] As shown in Figure 7, triangle ABC is in 3D space, and each vertex is associated with xyz coordinates. N is the normal vector of the triangle surface. Uniform sampling is applied to the 2D uv plane of the triangle surface (orthogonal to the normal N). Specifically, the uniform sampling can start from the origin and spread along the uv axis on the 2D uv plane with a given sampling step. All 2D sampling points inside the triangle are collected. These 2D sampling points may be transformed back to 3D world coordinates by an inverse transformation. Attribute values ​​of the sampled points may be derived in different ways, for example, by centroid-based interpolation.

[0048] The origin position may be chosen differently.

[0049] In one embodiment, the origin position is chosen as the centroid of the triangle (as shown in FIG. 7), which ensures that even if the sampling step is too large, there is at least one sampling point that is the centroid.

[0050] In another embodiment, the origin position is selected as one of the vertices.

[0051] In another embodiment, the origin location is selected as a point on the edge of a triangle.

[0052] The uv axis directions may be set differently.

[0053] In one embodiment, the direction of the u axis is chosen to be parallel to one of the edges of the triangle, for example in Figure 7 the u axis is parallel to edge BC.

[0054] Figure 4B and Figure 4C show two examples of point-plane based framework. The main difference with Figure 4A is that one of the meshes, either the original mesh 401 or the distorted mesh 403, is not sampled into the point cloud. In these cases, only the point-plane based D2 PSNR is calculated. By taking a combination of Figure 4B and Figure 4C, the symmetric point-plane D2 PSNR can also be used.

[0055] Taking the framework of FIG. 4B as an example, for each point on the sampled point cloud 404 (transformed from the distorted mesh), we can find the nearest triangle in the original mesh.

[0056] The geometric distortion may be calculated by the distance from the sampled point to the nearest triangle. The distance from a point to a triangle may be defined in different ways. In some embodiments, the distance is measured by the minimum distance, i.e., the length of the vector from the point to its orthogonal projection on the triangle.

[0057] The attribute distortion may be calculated by the difference between the attribute value of the point and the attribute value of the orthogonal projection on the triangle. The attribute value of the orthogonal projection on the triangle may be estimated in different ways. In some embodiments, a barycenter-based interpolation may be used.

[0058] 8 is a flowchart of an example process 800 for a sampling-based objective quality assessment method for a mesh. In some implementations, one or more process blocks in FIG. 8 may be performed by any of the elements described above.

[0059] As shown in FIG. 8, process 800 may include receiving, at an encoder, a first original polygonal mesh and a second distorted polygonal mesh, the first and second polygonal meshes each comprising a plurality of polygonal faces (block 810).

[0060] As further shown in FIG. 8, the process 800 may include converting the first and second polygonal meshes into two or more triangular meshes by subdividing a plurality of polygonal faces of each of the first and second polygonal meshes into a plurality of triangular faces (block 820).

[0061] As further shown in FIG. 8, the process 800 may include sampling a plurality of points on each of a plurality of triangular faces from both the first original polygonal mesh and the second distorted polygonal mesh (block 830).

[0062] As further shown in FIG. 8, process 800 may include creating at least a first sampled point cloud for one of the first original polygonal mesh or the second distorted polygonal mesh using the plurality of sampled points (block 840).

[0063] As further shown in FIG. 8, the process 800 may include calculating a geometric and attribute distortion profile between the first original polygonal mesh and the second distorted polygonal mesh based on at least the first sampled point cloud (block 850).

[0064] 8 illustrates example blocks of process 800, in some implementations, process 800 may include additional, fewer, different, or differently arranged blocks than those depicted in FIG 8. Additionally or alternatively, two or more blocks of process 800 may be performed in parallel.

[0065] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, FIG. 9 illustrates a computer system 900 suitable for implementing certain embodiments of the present disclosure.

[0066] The computer software may be coded using any suitable machine code or computer language that may be assembled, compiled, linked, or similar mechanisms to create code containing instructions that may be executed by a computer central processing unit (CPU), graphics processing unit (GPU), or the like, directly, or via interpretation, microcode execution, or the like.

[0067] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smart phones, gaming devices, Internet of Things devices, and the like.

[0068] 9 for computer system 900 are examples and are not intended to suggest any limitations on the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the non-limiting embodiment of computer system 900.

[0069] The computer system 900 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, clapping), visual input (such as gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as sound (such as speech, music, ambient sounds, etc.), images (such as scanned images, photographic images obtained from a still image camera, etc.), and video (such as two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0070] The input human interface devices may include one or more of a keyboard 901, a mouse 902, a trackpad 903, a touch screen 910, a data glove, a joystick 905, a microphone 906, a scanner 907, and a camera 908 (only one of each is depicted).

[0071] The computer system 900 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen 910, data gloves, or joystick 905, although there may also be haptic feedback devices that do not function as input devices). For example, such devices may be audio output devices (such as speakers 909, headphones (not shown)), visual output devices (such as screens 910 including CRT screens, LCD screens, plasma screens, OLED screens, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or output in more than three dimensions, such as through means such as stereoscopic output), and printers (not shown).

[0072] The computer system 900 may also include human accessible storage devices and their associated media, such as optical media, including a CD / DVD ROM / RW 920 with CD / DVD or similar media 921, thumb drives 922, removable hard drives or solid state drives 923, legacy magnetic media such as tapes and floppy disks (not shown), and specialized ROM / ASIC / PLD based devices (not shown) such as security dongles.

[0073] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0074] The computer system 900 may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicular and industrial including CANBus, etc. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port or peripheral bus 949, such as, for example, a USB port of the computer system 900, while other networks are generally integrated into the core of the computer system 900 by connection to a system bus as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system 900 can communicate with other entities. Such communications may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a particular CANbus device), or two-way to other computer systems using, for example, local or wide area digital networks. Such communications may include communications to cloud computing environments 955. Particular protocols and protocol stacks may be used in each of those networks and network interfaces, as described above.

[0075] The aforementioned human interface devices, human accessible storage devices, and network interface 954 may be attached to the core 940 of the computer system 900 .

[0076] The core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, dedicated programmable processing units in the form of field programmable gate areas (FPGAs) 943, hardware accelerators for specific tasks 944, etc. These devices may be connected via a system bus 948, along with read only memory (ROM) 945, random access memory 946, internal mass storage 947 such as an internal hard drive, SSD, etc. that is not accessible to the user. In some computer systems, the system bus 948 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be attached directly to the core's system bus 948 or via a peripheral bus 949. Architectures for peripheral buses include PCI, USB, etc. A graphics adapter 950 may be included in the core 940.

[0077] The CPU 941, GPU 942, FPGA 943, and accelerator 944 can execute certain instructions that can combine to make up the aforementioned computer code. That computer code may be stored in ROM 945 or RAM 946. Persistent data may be stored, for example, in internal mass storage device 947, while temporary data may also be stored in RAM 946. Rapid storage and retrieval in any of the memory devices may be enabled by the use of cache memory, which may be closely associated with one or more of the CPU 941, GPU 942, mass storage device 947, ROM 945, RAM 946, etc.

[0078] The computer-readable media can have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0079] By way of example and not limitation, computer system 900 having the architecture, and specifically core 940, may provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage devices as introduced above, as well as media associated with specific storage devices of core 940 that are non-transitory in nature, such as mass storage device 947 internal to the core or ROM 945. Software implementing various embodiments of the present disclosure may be stored in such devices and executed by core 940. Computer-readable media may include one or more memory devices or chips, depending on specific needs. The software may cause core 940, and specifically the processor therein (including a CPU, GPU, FPGA, etc.) to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM 946 and modifying such data structures according to processes defined by the software. Additionally or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator 944) that can operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. References to software may encompass logic, and vice versa, as appropriate. References to computer-readable media encompass circuitry (such as integrated circuits (ICs)) that stores software for execution, circuitry that embodies logic for execution, or both, as appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0080] While this disclosure describes several non-limiting embodiments, there are modifications, substitutions, and various alternative equivalents that are within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure. [Explanation of symbols]

[0081] 100 Communication Systems 110 Terminal 120 terminals 130 terminals 140 terminals 150 Network 200 Streaming System 201 Video Sources 202 Sample Stream 203 Encoder 204 encoded video bitstream 205 Streaming Server 206 Streaming Client 209 Video Bitstream 210 Video Decoder 211 Video Sample Stream 212 Display 213 Capture Subsystem 301 Original Mesh 302 point cloud 303 Distorted Mesh 304 point cloud 305 Geometry and Attribute Distortion 401 original polygon mesh 402 point cloud 403 Distorted Polygon Mesh 404 point cloud 405 Geometry and Attribute Peak Signal-to-Noise Ratio 406 Triangle Mesh 407 Distorted Triangle Mesh 800 processes 810 Block 820 Block 830 Block 840 Blocks 850 Block 900 Computer Systems 901 Keyboard 902 Mouse 903 Trackpad 905 Joystick 906 Microphone 907 Scanner 908 Camera 909 Speaker 910 Touchscreen 920 CD / DVD ROM / RW 921 CD / DVD or similar media 922 Thumb Drive 923 Removable Hard Drive or Solid State Drive 940 cores 941 Central Processing Unit (CPU) 942 Graphics Processing Unit (GPU) 943 Field Programmable Gate Area (FPGA) 944 Hardware Accelerator 945 Read-Only Memory (ROM) 946 Random Access Memory (RAM) 947 Internal mass storage 948 System Bus 949 Surrounding Bus 950 Graphics Adapter 954 Network Interface 955 Network

Claims

1. 1. A method executed by at least one processor, the method comprising: receiving, at an encoder, a first polygonal mesh and a second polygonal mesh, the first polygonal mesh being an original polygonal mesh and the second polygonal mesh being a warped polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces; converting the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective one of the two or more triangular meshes; sampling a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes; generating at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled points; calculating a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on at least the first sampled point cloud; A method comprising:

2. The method of claim 1 , wherein the step of sampling the plurality of points on each triangular face from the plurality of triangular faces begins at an origin and spreads along a UV axis on a 2D UV plane.

3. The method of claim 2 , wherein the origin is selected as a centroid of the respective triangular face from the plurality of triangular faces.

4. The method of claim 2 , wherein the origin is selected as one of a plurality of vertices on the respective triangular face from the plurality of triangular faces.

5. The method of claim 2 , wherein the origin is selected as a point on an edge of the respective triangular face from the plurality of triangular faces.

6. The method of claim 2 , wherein a direction of a U axis is selected to be parallel to one of a plurality of edges of the respective triangular face from the plurality of triangular faces.

7. The method of claim 1 , wherein the distortion profile of the geometry and attributes is based on one of a mean squared error or a peak signal to noise ratio.

8. Calculating a distortion profile of the geometry and attributes comprises: using the sampled points to generate a second sampled point cloud for the other of the first polygonal mesh or the second polygonal mesh; determining one or more characteristics of the second sampled point cloud based on the first sampled point cloud; The method of claim 1, further comprising:

9. The method of claim 8 , wherein the step of calculating a distortion profile of the geometry and attributes between the first polygonal mesh and the second polygonal mesh is further based on the determined one or more characteristics.

10. at least one memory configured to store program code; at least one processor configured to read said program code and to act as directed by said program code; 11. An apparatus comprising: receiving code configured to cause the at least one processor to receive, in an encoder, a first polygonal mesh and a second polygonal mesh, the first polygonal mesh being an original polygonal mesh and the second polygonal mesh being a distorted polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces; conversion code configured to cause the at least one processor to convert the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective triangular mesh of the two or more triangular meshes; sampling code configured to cause the at least one processor to sample a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes; generation code configured to cause the at least one processor to generate at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled points; and computing code configured to cause the at least one processor to compute a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on at least the first sampled point cloud.

13. An apparatus comprising:

11. The apparatus of claim 10 , wherein the sampling of the plurality of points on each triangular face from the plurality of triangular faces begins at an origin and spreads along a UV axis on a 2D UV plane.

12. The apparatus of claim 11 , wherein the origin is selected as a centroid of the respective triangular face from the plurality of triangular faces.

13. The apparatus of claim 11 , wherein the origin is selected as one of a plurality of vertices on the respective triangular face from the plurality of triangular faces.

14. The apparatus of claim 11 , wherein the origin is selected as a point on an edge of the respective triangular face from the plurality of triangular faces.

15. The apparatus of claim 11 , wherein a direction of the U axis is selected to be parallel to one of a plurality of edges of the respective triangular face from the plurality of triangular faces.

16. The apparatus of claim 10 , wherein the distortion profile of the geometry and attributes is based on one of a mean squared error or a peak signal to noise ratio.

17. The calculation code comprises: generating a second sampled point cloud for the other of the first polygonal mesh and the second polygonal mesh using the sampled points; determining one or more characteristics of the second sampled point cloud based on the first sampled point cloud; The apparatus of claim 10 , further causing the at least one processor to:

18. 20. The apparatus of claim 17, wherein the calculating a distortion profile of the geometry and attributes between the first polygonal mesh and the second polygonal mesh is further based on the determined one or more characteristics.

19. When executed by at least one processor, receiving, at an encoder, a first polygonal mesh and a second polygonal mesh, the first polygonal mesh being an original polygonal mesh and the second polygonal mesh being a warped polygonal mesh, each of the first and second polygonal meshes comprising a plurality of polygonal faces; converting the first polygonal mesh and the second polygonal mesh into two or more triangular meshes by subdividing the plurality of polygonal faces of each of the first polygonal mesh and the second polygonal mesh into a plurality of triangular faces, each triangular face corresponding to a respective one of the two or more triangular meshes; sampling a plurality of points on each of the plurality of triangular faces from both the first and second polygonal meshes; generating at least a first sampled point cloud for one of the first or second polygonal meshes using the sampled points; calculating a distortion profile of geometry and attributes between the first polygonal mesh and the second polygonal mesh based on at least the first sampled point cloud; A non-transitory computer-readable storage medium storing instructions that cause the at least one processor to:

20. 20. The non-transitory computer-readable storage medium of claim 19, wherein the sampling of the plurality of points on each triangular face from the plurality of triangular faces begins at an origin and spreads along a UV axis on a 2D UV plane.

Citation Information

Patent Citations

  • Point cloud coding standard conformance definition in computing environments

    US20200120347A1

  • Cost-driven framework for progressive compression of textured meshes

    US20200265552A1

  • Three-Dimensional Mesh Compression Using a Video Encoder

    US20210090301A1