Method for determining an objective quality of a reference representation of at least one 3D object

The method and system refine bounding boxes and compute 2D metric scores to enhance the precision and efficiency of 3D model quality assessment, addressing the limitations of existing subjective and objective methods.

WO2025177222A1PCT designated stage Publication Date: 2025-08-28TENCENT AMERICA LLC
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
PCT/IB2025/051868
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-02-23
Filing Date
2025-02-21
Publication Date
2025-08-28

AI Technical Summary

Technical Problem

Current methods for assessing the quality of 3D models face challenges such as subjective tests being costly and time-consuming, model-based objective methods lacking reliability, and projection-based methods lacking precision and universality, particularly in real-time applications.

Method used

A method and system for determining the objective quality of 3D objects by projecting them into 2D images, refining bounding boxes, and computing 2D metric scores, which includes selecting qualified viewpoints, adjusting bounding box sizes, and cropping images to enhance accuracy.

Benefits of technology

Improves the precision and efficiency of quality assessment by focusing on meaningful regions and reducing biases, enabling more accurate evaluation of 3D model distortions and visual fidelity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure IB2025051868_28082025_PF_FP_ABST
    Figure IB2025051868_28082025_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a method for determining an objective quality of a reference representation of at least one 3D object, comprising: - projecting (E30) the reference representation into a 2D reference image and a distorted representation of said 3D object into a 2D distorted image; - generating (E40) at least one reference bounding box in the 2D reference image and at least one corresponding distorted bounding box in the 2D distorted image; - for at least one confirmed viewpoint: i. refining (E60) at least one said reference bounding box and its corresponding distorted bounding box; ii. cropping (E70) the at least one 2D reference image and the at least one 2D distorted image; iii. comparing (E80) said cropped images and obtaining (E80) at least one 2D metric score; - obtaining (E90) an overall quality score for said reference representation using the 2D metric.
Need to check novelty before this filing date? Find Prior Art

Description

[0001]METHOD FOR DETERMINING AN OBJECTIVE QUALITY OF A REFERENCE REPRESENTATION OF AT LEAST ONE 3D OBJECT ▪ Technical field The invention relates to the general field of computer graphics andmultimedia processing. It concerns more particularly a method of predicting the objective quality of 3D content representation. ▪ Background In the context of this disclosure, we consider a video scene comprising 3Dobjects for which a 3D model is known. In a known way, the 3D scene can be projected to generate a stream comprising a sequence of 2D images, this stream being then compressed for transmission and later decompressed. This disclosure applies to any type of representation of 3D objects and inparticular to discrete representations such as point clouds or continuousrepresentations such as meshes. As the model of the 3D objects is known, for each pixel in the original 3D scene or 2D decompressed image, it is possible to determine whether this pixel belongs to a given object or to the background for example. A point cloud is a collection of data points in a three-dimensional coordinatesystem. Each point represents the spatial location and sometimes additionalattributes (such as colour or intensity) of a surface or object captured by 3D scanning technologies like LiDAR or photogrammetry. These points form a representation of the surface geometry of the scanned object or environment. While the point cloud consists of individual points in a 3D coordinate system,where each point represents a specific location in space along with optionalattributes, a mesh on the other hand represents geometry using interconnected vertices, edges, and faces. It consists of a network of vertices (points), edges (lines connecting vertices), and faces (polygons formed by connecting edges) that define the surface geometry of an object. Meshes provide a more structured andinterconnected representation of geometry compared to point clouds.Quality assessment of 3D content representations, such as point clouds or meshes, can be carried out at various stages. For example, quality assessment may be used to detect imperfections or anomalies introduced during the 3D data acquisition process. It can also be used to estimate the extent to which visual quality may be degraded, for example in the form of geometric distortions and colour distortions,when compression techniques are applied particularly for transmission or storage,especially at reduced transmission rates. After decompression, the quality of the reconstruction may be assessed, by checking whether the compressed data can be accurately reconstructed, without visible artifacts or noticeable loss of detail. Quality assessment is generally based on two main methods. The firstmethod, subjective testing, involves human assessors evaluating various meshes encoded at different bitrates. In these tests, participants assign scores based on visual perception, assessing factors such as geometric accuracy, texture quality and overall visual fidelity. Subjective tests provide valuable information on how thehuman eye perceives quality, which is important for applications where visual appealis a priority, but they are time-consuming and can be inconsistent. The second method is objective measurements, which use algorithms to automatically assess the quality of 3D models or meshes. These measurements fall into two main categories: model-based and projection-based. Model-based measurements directly measure the geometric accuracy of the3D representation. These include well-established metrics such as point-to-point, point-to-plane and luma peak signal-to-noise ratio (PSNR), which are commonly used in the evaluation of compression and distortion in 3D models. In addition, more specialized measures, such as the Point Cloud Quality Metric (PCQM), have beenintroduced in academic research to better capture the perceptual quality of pointclouds. These model-based measures provide a quantitative measure of geometric fidelity and are useful for assessing the extent to which the compressed or transmitted mesh matches the original model. On the other hand, projection-based measures assess quality by comparingthe rendered or projected version of a 3D model with a 2D image. These measuresare generally used in contexts where the final result is seen as a 2D projection, as in VR or live streaming applications. They are used in standards such as MPEG (Moving Picture Experts Group), which incorporates projection-based quality measures to assess the visual quality of compressed 3D content in real-time applications. These measures assess the visual quality of the 3D model whenrendered and viewed from different angles or distances, taking into accountdistortions that may be introduced during compression or transmission. Current methods for assessing the quality of 3D models face a number of difficulties. Subjective methods are costly, time-consuming and unsuitable for real- time monitoring. Model-based objective methods often lack a reliable correlationwith the mean opinion score (MOS) and are complex to implement. They also lackrobustness across different encoders and content types. Projection-based methods lack precision and introduce biases. Cropping can introduce blank areas into images, distorting quality scores, and current solutions such as masking or bounding boxes are not universally applicable. Therefore, there is need for more effective quality assessment methods.▪ Summary Therefore, the disclosure concerns a method for determining an objective quality of a reference representation of at least one 3D object, comprising:- projecting, from at least one qualified viewpoint, the referencerepresentation of said at least one 3D object into a 2D reference image and a distorted representation of said 3D object into a 2D distorted image; - generating, for said at least one qualified viewpoint, at least one reference bounding box in the 2D reference image and at least one correspondingdistorted bounding box in the 2D distorted image;- for at least one confirmed viewpoint among said at least one qualified viewpoint: i. refining at least one said reference bounding box generated for said confirmed viewpoint and its corresponding distorted bounding box;ii. cropping the at least one 2D reference image and the one 2D distortedimage obtained for said confirmed viewpoint respectively into a 2D cropped reference image and into a 2D cropped distorted image, the cropping of an image being based on the at least one bounding box of this image; iii. comparing said cropped images and obtaining at least one 2D metric score for said confirmed viewpoint using the result of said comparison; and- obtaining an overall quality score for said reference representation of the3D object using the 2D metric scores obtained for said at least one confirmed viewpoints. Correlatively, the disclosure concerns a system for determining an objective quality of a reference representation of at least one 3D object, comprising:- a rendering module configured to project, from at least one qualifiedviewpoint, the reference representation of said at least one 3D object into a 2D reference image and a distorted representation of said 3D object into a 2D distorted image; - at least one bounding box computing module configured to generate, forsaid at least one qualified viewpoint, at least one reference bounding box in the 2Dreference image and at least one distorted bounding box in the 2D distorted image, - said at least one bounding box computing module being configured to refine, for at least one confirmed viewpoint among said at least one qualified viewpoint, at least one said reference bounding box generated for said confirmedviewpoint and its corresponding distorted bounding box;- a cropping module configured to crop the at least one 2D reference image and the at least one 2D distorted image obtained for said confirmed viewpoint respectively into a 2D cropped reference image and into a 2D cropped distorted image based, the cropping of an image being based on the at least one referencebounding box of this image;- a first scoring module configured to compute at least one 2D metric score for said confirmed viewpoint at least by comparing said cropped images; and - a second scoring module configured to compute an overall quality score for said reference representation of the 3D object using the 2D metric scoresobtained for said at least one confirmed viewpoints. In one embodiment, the reference representation of the 3D object is a representation in the form of point clouds or meshes but the invention applies to other types of representations. The method and the system are notably remarkable in that theycomprises a step or a module for refining at least a bounding box generated in the2D reference image and the corresponding bounding box in the 2D distorted image before cropping these images and computing a 2D metric score by comparing the cropped images. Such a refinement increases the quality of the determination. In one embodiment, the step of refining one said bounding box comprisesshrinking said bounding box to exclude portions thereof which do not contain sufficient information related to said 3D object. For example, shrinking one said bounding box comprises removing a group of lines on at least one border of the bounding box if a percentage of said object contained in said group of lines is lessthan a predetermined threshold.This method enhances the count of valid points (i.e. points corresponding to the 3D object) contained within the bounding box, facilitating more accurate 2D metric computation. In one embodiment, the step of refining one said bounding box comprisesat least one operation among:- deleting a fixed number of rows and columns at boarders of the bounding box; - setting said bounding box as the largest rectangle encompassing said 3D object and not including any empty pixel;- compactly arranging a plurality of patches formed by the object.In one embodiment, the step of generating the bounding boxes comprise setting the size of the reference bounding box and the size of the distorted bounding box as a function of each other. This step advantageously applies notably when the reference and thedistorted bounding boxes have different sizes, for example due to the influence of compression. In one embodiment, at least one of these bounding boxes is resized so that they have the same size. For example: - In one embodiment, the common size of the bounding boxes is computed solely based on the 2D distorted image IMDIS,j.- In another embodiment, the largest bounding box is reduced to match the size of the smallest one. - In another embodiment, the smallest bounding box is expanded to match the size of the largest one. In one embodiment, the method comprises a step of selecting said atleast one qualified viewpoint among a set of candidate viewpoints. In one embodiment, the reference representation of the 3D object is a representation in the form of point clouds or meshes. In one embodiment, at least one candidate viewpoint is selected as aqualified viewpoint based on a knowledge of the reference representation of the 3Dobject, for example based on a knowledge of the object representation in the form of point clouds or meshes. For examples, qualified viewpoints may be chosen to focus on areas: i / of high geometric complexity, such as sharp edges, fine details, or regions withsignificant curvature, as these are often more susceptible to distortions.ii / with texture and / or colour variations to emphasize regions with significant colorimetric details, where compression or processing could degrade visual fidelity. iii / with uneven point distribution or sparsity, as these can highlight issues such as compression artifacts or missing data. In one embodiment, at least one said candidate viewpoint is randomlyselected. In one embodiment, the method comprises a step for determining whether at least one said candidate viewpoint should be rejected or selected as one said qualified viewpoint. In one embodiment, a candidate viewpoint is rejected if a distancebetween said candidate viewpoint and one qualified viewpoint is smaller than a determined threshold. The distance may be an Euclidian distance or a Minkovski distance based on rendering parameters characterizing the viewpoints. In one embodiment, the method takes into account the reference bounding box and / or the distorted bounding box generated for one said qualifiedviewpoint to decide whether said qualified viewpoint should be confirmed as a saidconfirmed viewpoint. In one embodiment, a qualified viewpoint should be confirmed if: - a size of said reference bounding box and a size of said distorted bounding box both exceed a predetermined size ; and if- said reference bounding box and said distorted bounding box both exhibita percentage of empty area which is lower than a predetermined threshold. Thanks to this feature, only the viewpoints that generate meaningful bounding boxes are considered for computing the 2D metric score. In one embodiment, a plurality of reference bounding boxes and aplurality of corresponding distorted bounding boxes are generated for at least oneconfirmed viewpoint and one said 2D metric score is obtained for each pair or bounding boxes comprising one said reference bounding box and its of corresponding distorted bounding box. In one embodiment, the step of obtaining the 2D metric score comprisesmerging the result of the above mentioned comparison with at least one featureamong: - a resolution of one said bounding box; - a percentage of empty area in at least one said bounding box; - a difference of vertices between the reference representation and thedistorted representation of said at least one 3D object;- a type of said reference representation. In one embodiment the disclosed method is implemented in the form of a computer program. The disclosure also concerns a computer program comprising instructionsconfigured to implement the steps of the method for determining an objective quality of a reference representation of a 3D object when this computer program is executed by a computer. This program can use any programming language, and be in the form of source code, object code, or intermediate code between source code and objectcode, such as in a partially compiled form, or in any other desirable form.The invention also concerns a readable medium comprising this computer program of this set of computer programs. The recording medium can be any entity or device capable of storing the program. For example, the support may include a storage means, such as a ROM,for example a CD ROM or a microelectronic circuit ROM, or also a magnetic recordingmeans, for example a hard disk. On the other hand, the recording medium can be a transmissible medium such as an electrical or optical signal, which can be carried via an electrical or optical cable, by radio or by other means. The program according to the invention can inparticular be downloaded on an Internet-type network.Alternatively, the recording medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method in question.▪ DrawingsFurther features and advantages of the present invention will become apparent from the description below, with reference to the accompanying drawings, which illustrate a non-limiting example: - Figure 1 represents in flowchart form the main steps of a method for predicting the objective quality of the representation of 3D objects, forexample in the form of point clouds or meshes. - Figure 2 shows schematically a system for predicting the objective quality of the representation of 3D objects, for example in the form of point clouds or meshes. -Figures 3A and 3B respectively illustrate a correct bounding box and aninappropriate bounding box for a same mesh. - Figures 4, 5, 6 and 7 illustrate examples of bounding box refinement. - Figure 8 represent the architecture of the system of figure 2 in one specific embodiment.Detailed descriptionFigure 1 represents in flowchart form the main steps of a method for predicting the objective quality of the representation of 3D objects, for example in the form of point clouds or meshes. In the following description, and for the sake of simplicity, we focus on asingle object, but the method can be applied in the same way to determine theobjective quality of the representation of several objects. For a given object, the method involves rendering a reference representation OBJREFof the object into so called 2D reference images from a set of viewpoints and rendering a distorted representation OBJDISof the object into so called 2D distortedimages under the same viewpoints.These 2D projections represent how the 3D object appears from each angle. Viewpoints are selected camera angles or positions around the object. Each viewpoint produces a 2D image of the 3D object projected from that angle. The general aim is to assess the visual quality of the model from different perspectives,enabling the identification of any distortions, visual artifacts or loss of detail thatmay appear depending on the angle of observation. Besides, in interactive applications such as virtual reality or video games, users may change their point of view, hence the need to assess quality from several angles. In one embodiment, the method comprises a step E10 of selecting thequalified viewpoints.Each qualified viewpoint may be characterized by one or several rendering parameters. These parameters may for example comprise at least one parameter among: - rotation angles which determine the orientation of the virtual camera relative to the 3D model.- scaling which adjusts the size of the object within the rendered image - lighting conditions,… In one embodiment, at least one qualified viewpoint if selected among a set of candidate viewpoints. In one embodiment, at least one candidate viewpoint is selected as aqualified viewpoint based on a knowledge of the reference representation of the 3Dobject. In one embodiment, at least some of the qualified viewpoints are randomly chosen. In one embodiment, the method comprises a first qualification step E20 fordetermining whether at least one said candidate viewpoint VPk should be rejectedor retained as qualified viewpoints. For example, a randomly selected candidate viewpoint may be rejected if a distance between this candidate viewpoint and a qualified viewpoint is too small, for example smaller than a determined threshold. This distance may be an Euclidiandistance or a Minkowski distance based on the above mentioned renderingparameters. We consider that NVPqualified viewpoints VPk, k=1…NVPhave been selected at step E10. In one embodiment the number NVPof qualified viewpoints is predeterminedand the method comprises a step of controlling that NVP qualified viewpoints areactually selected. In the embodiment of Figure 1, the method comprises a loop for processing at least part of the NVPqualified viewpoints VPj, j=1…NVP. In one embodiment, the loop comprises a step E30 of projecting, from thequalified viewpoint VPj:- the reference representation OBJREFof the 3D object into a so called 2D reference image IMREF,j, and - a distorted representation OBJDISof the 3D object into a so called 2D distorted image IMDIS,j. In one embodiment, the loop comprises a step E40 of generating, for thequalified viewpoint VPj, at least one reference bounding box BBREF,jin the 2D reference image and at least one distorted bounding box BBDIS,jin the 2D distorted image. In one embodiment, a qualified viewpoint should be confirmed as a said confirmed viewpoint if: -a size of said reference bounding box BBREF,j and a size of said distortedbounding box BBDIS,jboth exceed a predetermined size ; and if - said reference bounding box and said distorted bounding box both exhibit a percentage of empty area which is lower than a predetermined threshold. In other words, selected viewpoints generating bounding boxes that areexcessively small (with one of the dimensions, height, or width too small) or those that inadequately represent the 3D object (exhibiting a high percentage of empty area) may be rejected. Figure 3A and 3B respectively illustrate a correct bounding box and aninappropriate bounding box for a same mesh. In thus example, the selectedviewpoint corresponding to Figure 3a should be confirmed and the selected viewpoint corresponding to Figure 3b should be rejected. Figure 4 illustrates a first example of bounding box refinement. This refinement process comprises shrinking one or several of the four borders of thebounding box successively. For example, starting with the top border, followed bythe bottom, left, and right borders, the refinement is carried out iteratively. For each border, a group of n lines, typically 8, is analysed to determine the percentage of the object contained within. If this percentage falls below a predefined threshold, typically 20%, the bounding box is further shrunk by removing the corresponding nlines. This iterative process continues for all four directions until convergence isachieved. Advantageously, this method enhances the count of valid points contained within the bounding box, facilitating more accurate 2D metric computation. Another example of bounding box refinement presented at figure 5, comprises deleting a fixed number of rows and columns at boarders of the boundingbox. Another example of bounding box refinement presented at figure 6 comprises setting said bounding box as the largest rectangle encompassing said 3D object and not including any empty pixel. Another example of bounding box refinement presented at figure 7 comprisescompactly arranging a plurality of patches formed by the object.In this embodiment, in the step E40 of generating the reference bounding boxes BBREF,jand the distorted bounding box BBDIS,j, if the sizes of the reference bounding box and of the distorted bounding differ, they are adjusted to a common size which is computed solely based on the 2D distorted image IMDIS,j. The method comprises a step E70 of cropping:- the 2D reference image IMREF,jobtained for the confirmed viewpoint VPjinto a 2D cropped reference image CIMREF,jbased on its at least one reference bounding box BBREF,jand - the 2D distorted image IMDIS,jobtained this viewpoint VPjinto a 2D cropped distorted image CIMDIS,j based on its at least one distortedbounding box BBDIS,j. As known by the man skilled in the art, one purpose of cropping the projected images is to retain only the area within the bounding boxes, thus eliminating unnecessary parts of the 2D images. This enables the evaluation to be focused onthe object or regions of interest, thus improving the accuracy of the quality metric.The method comprises a step E80 of obtaining at least one 2D metric score SC2Djfor the confirmed viewpoint VPjat least by comparing said at least one 2D cropped reference image CIMREF,jand said at least one 2D cropped distorted image CIMDIS,j. In one embodiment, the step E80 of obtaining the 2D metric score SC2Djcomprises merging the result of this comparison with at least one feature f among: - a resolution of one the reference bounding box BBREF,jor of the distorted bounding box BBDIS,j; - a percentage of empty area in at least one of these bounding boxes BBREF,j, BBDIS,j; - a difference of vertices between the reference representation OBJREFand the distorted representation (OBJDIS) of said at least one 3D object; - a type of said reference representation. The merging process can be implemented using conditional statements,where the 2D metric score SC2Dj is adjust with the above-mentioned features byapplying predefined conditions. Alternatively, regression models may be employed to compute the final score. These models can take various forms, including linear or non-linear regression functions, and machine learning models, such as gradient boosting, random forests,support vector machines (SVMs), multilayer perceptrons (MLPs), convolutionalnetworks, or deep networks. It is important to note that several bounding boxes may be generated for a given viewpoint in the 2D reference image and in the 2D distorted image. For example, each bounding box represents a specific region of interest, forexample a face, a text portion, or other critical element.In one embodiment, one 2D metric score may be obtained for each pair or bounding boxes. The scores may be weighted. In one embodiment, all the confirmed viewpoints are not actually processed. More precisely, in one embodiment, at the end of the iteration executed fora confirmed viewpoint, at step E100, a test is made to decide whether to proceedwith another confirmed viewpoint or to terminate the process, based on a predefined criterion representative of whether the accuracy of the score prediction is good enough. For this reason, in one embodiment, the method comprises a step E90 ofcalculating an overall quality score OQSj for the j first viewpoints.In one embodiment, at step E100, it is decided to terminate the loop when a difference Δ^^between: - the overall score OSCjcalculated for the j first viewpoints and - the overall score OSCj-1calculated for the j-1 first viewpointsis lower than with a predefined threshold TH. The threshold TH may be defined as a function of a metric of the 2D model. For example, when utilizing the VMAF (Video Multi-Method Assessment Fusion) objective video quality assessment metric, the threshold TH may be fixed equal to 1, a difference Δ^^below 5 being not perceptible to humans. At step E90, several methods may be used for calculating the overall qualityscore OQSjfrom the j individual 2D metric scores SC2Dk, k=1…jcalculated for the j first confirmed viewpoints VPj. For example, the overall quality score OQSjis computed as: - the median of the j 2D metric scores; or as -the overall quality score OQSj is the mean of individual 2D metric scoresbetween two quantiles, for example between the first and the third quartiles or between the first and the ninth deciles. In some embodiments, the loop is not interrupted until a minimum number of confirmed viewpoints has been considered. For example, in one embodiment, the loop is interrupted when (i) thepercentage of valid viewpoints among the first j viewpoints is above a predetermined threshold, e.g. 90%, and (ii) the number j of examined confirmed viewpoints is at least equal to a minimum number of viewpoints. For example, in one embodiment, a confirmed viewpoint VPkis consideredvalid if and only if the 2D metric score SC2Dk calculated for this viewpoint lies withina specific quality range. In another embodiment, the loop is terminated when a confidence interval calculated from the SC2D metric scores of all valid viewpoints is below a predefined threshold. Figure 2 shows schematically a system SYS for predicting the objective qualityof the representation of 3D objects, for example in the form of point clouds or meshes. The system SYS may be configured to implement the method of Figure 1. The system comprises a rendering module RM configured to project, from at least one qualified viewpoint VPj, the reference representation OBJREFof a 3D objectinto a 2D reference image IMREF,j and a distorted representation OBJDIS of this 3Dobject into a 2D distorted image IMDIS,j. In the example of figure 2, the system S comprises two bounding box computing modules BBCM. A first bounding box computing module BBCM is configured to generate, for said at least one qualified viewpoint VPj, at least one reference bounding box BBREF,jin the 2D reference image. This first bounding box computing module is configuredto refine at least one said reference bounding box BBREF,j. A second bounding box computing module BBCM is configured to generate, for said at least one qualified viewpoint VPj, at least one distorted bounding box BBDIS,jin the 2D distorted image. This second bounding box computing module isconfigured to refine at least one distorted bounding box BBDIS,j.Alternately, only one BBCM module could be used. The system comprises a cropping module CM configured to crop the at least one 2D reference image IMREF,jbased on its at least one reference bounding box BBREF,jand to crop the at least one 2D distorted image IMDIS,jbased on its at leastone distorted bounding box BBDIS,j.The system S comprises a first scoring module 2DMSM configured to compute at least one 2D metric score for a confirmed viewpoint VPjat least by comparing the cropped images. This module may be configured to use features f to modify the score obtained from the 2D metric. The system S comprise a control module CTRLM configured to make a listVPL of selected viewpoints. The control module CTRLM is configured to confirm selected viewpoints a confirmed viewpoint and to terminate the process when the accuracy of the score prediction is good enough. The system S comprises a second scoring module configured to compute anoverall quality score OQSj for the reference representation of the 3D object usingthe 2D metric scores SC2Djobtained for at least one confirmed viewpoints VPj. In one embodiment, the system SYS of figure 2 has the hardware architecture of a computer. As shown on Figure 8, it comprises at least one processor PROC; and at least one memory MEM. The memory MEM constitutes a storage medium readable by processor PROCand storing a computer program PROG comprising instructions for implementing all or part of the steps of a method conforming to this disclosure, when the computer program PROG is executed by the processor PROC.

Claims

CLAIMS 1. A method for determining an objective quality of a reference representation of at least one 3D object, comprising:- projecting (E30), from at least one qualified viewpoint (VPj), the reference representation (OBJREF) of said at least one 3D object into a 2D reference image (IMREF,j) and a distorted representation (OBJDIS) of said 3D object into a 2D distorted image (IMDIS,j); -generating (E40), for said at least one qualified viewpoint (VPj), at leastone reference bounding box (BBREF,j) in the 2D reference image and at least one corresponding distorted bounding box (BBDIS,j) in the 2D distorted image; - for at least one confirmed viewpoint (VPj) among said at least one qualified viewpoint (VPj):i. refining (E60) at least one said reference bounding box (BBREF,j) generated for said confirmed viewpoint and its corresponding distorted bounding box (BBDIS,j); ii. cropping (E70) the at least one 2D reference image (IMREF,j) and the at least one 2D distorted image (IMDIS,j) obtained for said confirmedviewpoint (VPj) respectively into a 2D cropped reference image (CIMREF,j) and into a 2D cropped distorted image (CIMDIS,j), the cropping of an image being based on its at least one bounding box (BBREF,j, BBDIS,j) of this image; iii. comparing (E80) said cropped images (CIMREF,j, CIMDIS,j) and obtaining(E80) at least one 2D metric score (SC2Dj) for said confirmed viewpoint (VPj) using the result of said comparison ; - obtaining (E90) an overall quality score (OQSj) for said reference representation of the 3D object using the 2D metric scores (SC2Dj) obtained for said at least one confirmed viewpoints (VPj).

2. The method of claim 1, wherein refining (E60) one said bounding box (BBREF,j, BBDIS,j) comprises at least one operation among: - shrinking said bounding box to exclude portions thereof which do not contain sufficient information related to said 3D object; -deleting a fixed number of rows and columns at boarders of the boundingbox; - setting said bounding box as the largest rectangle encompassing said 3D object and not including any empty pixel; - compactly arranging a plurality of patches formed by the object.

3. The method of claim 2, wherein shrinking one said bounding box (BBREF,j, BBDIS,j) comprises removing a group of lines on at least one border of the bounding box if a percentage of said object contained in said group of lines is less than a predetermined threshold.

4. The method of any one of claims 1 to 3, wherein the step (E40) of generating said bounding boxes (BBREF,j, BBDIS,j) comprise setting the size of the reference bounding box (BBREF,j) and the size of the distorted bounding box (BBDIS,j) as a function of each other.

5. The method of any one of claim 1 to 4, comprising a step (E10) of selecting said at least one qualified viewpoint (VPk) among a set of candidate viewpoints.

6. The method of claim 5, wherein at least one said candidate viewpoint is selected as a qualified viewpoint based on a knowledge of the reference representation of said object.

7. The method of claim 5 or 6, wherein at least one said candidate viewpoint israndomly selected.

8. The method of any one of claims 5 to 7, comprising a step (E20) for determining whether at least one said candidate viewpoint should be rejected or selected as one said qualified viewpoint.

9. The method of claim 8, wherein one said candidate viewpoint is rejected if adistance between said candidate viewpoint and one said qualified viewpoint is smaller than a determined threshold.

10. The method of any one of claim 1 to 9, comprising taking at least into account the reference bounding box and / or the distorted bounding box generated(E40) for one said qualified viewpoint to decide whether said qualified viewpoint should be confirmed as a said confirmed viewpoint.

11. The method of claim 10, wherein said qualified viewpoint should be confirmed as a said confirmed viewpoint if:- a size of said reference bounding box (BBREF,j) and a size of said distorted bounding box (BBDIS,j) both exceed a predetermined size ; and if - said reference bounding box and said distorted bounding box both exhibit a percentage of empty area which is lower than a predetermined threshold.

12. The method of any one of claim 1 to 11, wherein a plurality of reference bounding boxes and a plurality of corresponding distorted bounding boxes are generated for at least one confirmed viewpoint and wherein one said 2D metric score (SC2Dj) is obtained (E80) for each pair or bounding boxescomprising one said reference bounding box and its of corresponding distorted bounding box.

13. The method of any one of claims 1 to 12 wherein said step (E80) of obtaining said 2D metric score (SC2Dj) comprises merging the result of said comparison said 2D metric score (SC2Dj) with at least one feature (f) among: - a resolution of one said bounding box (BBREF,j, BBDIS,j< / sub>); -a percentage of empty area in at least one said bounding box (BBREF,j,BBDIS,j< / sub>); - a difference of vertices between the reference representation (OBJREF) and the distorted representation (OBJDIS) of said at least one 3D object; - a type of said reference representation.

14. The method of any one of claim 1 to 9, wherein said reference representation is a representation in the form of point clouds or meshes.

15. A system for determining an objective quality of a reference representation of at least one 3D object, comprising:- a rendering module (RM) configured to project, from at least one qualified viewpoint (VPj), the reference representation (OBJREF) of said at least one 3D object into a 2D reference image (IMREF,j) and a distorted representation (OBJDIS) of said 3D object into a 2D distorted image (IMDIS,j);- at least one bounding box computing module (BBCM) configured to generate, for said at least one qualified viewpoint (VPj), at least one reference bounding box (BBREF,j) in the 2D reference image and at least one distorted bounding box (BBDIS,j) in the 2D distorted image, -said at least one bounding box computing module (BBCM) beingconfigured to refine, for at least one confirmed viewpoint (VPj) among said at least one qualified viewpoint (VPj), at least one said reference bounding box (BBREF,j) generated for said confirmed viewpoint and its corresponding distorted bounding box (BBDIS,j< / sub>); -a cropping module (CM) configured to crop the at least one 2D referenceimage (IMREF,j) and the at least one 2D distorted image (IMDIS,j) obtainedfor said confirmed viewpoint (VPj) respectively into a 2D cropped reference image (CIMREF,j) and into a 2D cropped distorted image (CIMDIS,j), the cropping of an image being based on its at least one bounding box (BBREF,j, BBDIS,j< / sub>); -a first scoring module (2DMSM) configured to compute (E80) at least one2D metric score (SC2Dj) for said confirmed viewpoint (VPj) at least by comparing said cropped images (CIMREF,j, CIMDIS,j); and - a second scoring module configured to compute an overall quality score (OQSj) for said reference representation of the 3D object using the 2D metric scores (SC2Dj) obtained for said at least one confirmed viewpoints(VPj< / sub>).

16. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of a method according to any one of claims 1 to 14.

Citation Information

Patent Citations

  • Projection-based point cloud quality evaluation method and apparatus, device and storage medium

    WO2022183500A1