Three-dimensional point cloud just noticeable distortion threshold prediction method and product

Abstract

The invention provides a three-dimensional point cloud just noticeable distortion threshold prediction method and product. The specific implementation scheme is as follows: extracting three-dimensional texture features based on a reference point cloud and a corresponding distortion point cloud; performing multi-view projection processing on the reference point cloud and the distorted point cloud to obtain a plurality of two-dimensional projection images of different views; extracting two-dimensional texture features based on the two-dimensional projection images of the reference point cloud and the distorted point cloud; splicing the three-dimensional texture features and the two-dimensional texture features to obtain feature vectors, and learning based on a machine learning algorithm to obtain a mapping relationship between the feature vectors and just perceptible distortion thresholds; through the implementation of the scheme, the mapping relation between the feature vector and the just noticeable distortion threshold is established through machine learning after the point cloud three-dimensional feature and the point cloud two-dimensional feature are spliced, a high-precision three-dimensional point cloud just noticeable distortion threshold prediction model is constructed, effective indication can be provided for three-dimensional point cloud coding, and the accuracy of the three-dimensional point cloud coding is improved. And the visual quality of the coded point cloud is ensured.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and product for predicting the just-perceptible distortion threshold of three-dimensional point clouds. Background Technology

[0002] 3D point clouds are a commonly used data format in virtual reality and augmented reality, containing both 3D coordinates and color information, enabling a more accurate and complete representation of the objective world. In practical applications, 3D point clouds typically require encoding, with the core task being to provide visual quality comparable to the original point cloud at a lower bitrate. Visual physiology research indicates that the human eye can only perceive distortion in visual signals when it reaches a certain threshold. Therefore, this just-perceptible distortion threshold plays a crucial role in the encoding of 3D point clouds. However, current research has only addressed just-perceptible distortion in traditional images and videos, and traditional just-perceptible distortion perception models struggle to describe the perceptual mechanism of point clouds. Consequently, current 3D point cloud encoding schemes lack effective indicative information, resulting in poor encoding quality and consequently, poor visual quality of the encoded point clouds, failing to meet application requirements.

[0003] It is important to note that the techniques described in this section are not necessarily those previously conceived or adopted. Unless otherwise specified, no technique described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be recognized in any prior art. Summary of the Invention

[0004] The main objective of this application is to provide a method and product for predicting the just-perceptible distortion threshold of three-dimensional point clouds, which can at least solve one of the above-mentioned problems provided by related technologies.

[0005] The first aspect of this application provides a method for predicting the just-aware distortion threshold of 3D point clouds, including:

[0006] Three-dimensional texture features are extracted based on a reference point cloud and the corresponding distorted point cloud; wherein, the three-dimensional texture features include: kurtosis features, brightness features and color features;

[0007] The reference point cloud and the distorted point cloud are subjected to multi-view projection processing to obtain two-dimensional projection images from multiple different viewpoints.

[0008] Based on the reference point cloud and the two-dimensional projection image of the distorted point cloud, two-dimensional texture features are extracted; wherein, the two-dimensional texture features include: texture complexity features, texture masking features and SIFT features;

[0009] The three-dimensional texture features and the two-dimensional texture features are concatenated to obtain a feature vector, and the mapping relationship between the feature vector and the just-perceptible distortion threshold is learned based on a machine learning algorithm.

[0010] The second aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory, and when the processor executes the computer program, it implements the steps in the perceptible distortion threshold prediction method for three-dimensional point clouds provided in the first aspect of the embodiments of this application.

[0011] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the steps in the perceptible distortion threshold prediction method for three-dimensional point clouds provided in the first aspect of the embodiments of this application.

[0012] As can be seen from the above, the 3D point cloud just-aware distortion threshold prediction method and product provided in this application extracts 3D texture features based on a reference point cloud and the corresponding distorted point cloud; the reference point cloud and the distorted point cloud are subjected to multi-view projection processing to obtain multiple 2D projection images from different viewpoints; 2D texture features are extracted based on the 2D projection images of the reference point cloud and the distorted point cloud; the 3D texture features and the 2D texture features are concatenated to obtain a feature vector, and the mapping relationship between the feature vector and the just-aware distortion threshold is learned based on a machine learning algorithm. Through the implementation of this application, the 3D and 2D features of the point cloud are concatenated, and the mapping relationship between the feature vector and the just-aware distortion threshold is established through machine learning, thus constructing a 3D point cloud just-aware distortion threshold prediction model with high accuracy. This model can provide effective guidance for 3D point cloud encoding and ensure the visual quality of the encoded point cloud.

[0013] It should be understood that the description in this section is not intended to identify key or important features of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0014] The accompanying drawings exemplify embodiments and form part of the specification, working together with the textual description to explain exemplary implementations of the embodiments. The drawings shown are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0015] Figure 1 A schematic diagram illustrating the principle of a 3D point cloud just-perceptible distortion threshold prediction method provided in an embodiment of this application;

[0016] Figure 2A schematic diagram of the basic process of a method for predicting the just-perceptible distortion threshold of a 3D point cloud provided in an embodiment of this application;

[0017] Figure 3 A schematic diagram of the functional modules of a just-perceptible distortion threshold prediction device provided in an embodiment of this application;

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the description of the embodiments of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments of this application, the term "multiple" means two or more, unless otherwise explicitly specified.

[0021] To address the problem that related 3D point cloud encoding processing schemes lack effective indicative information, resulting in poor visual quality of the encoded point cloud and difficulty in meeting application requirements, this application provides an embodiment of a 3D point cloud just-perceptible distortion threshold prediction method, such as... Figure 1 The diagram shown illustrates the principle of the just-aware distortion threshold prediction method for 3D point clouds provided in this embodiment. Specifically, this embodiment extracts 3D texture features and 2D texture features from the 3D space and 2D projection space of the point cloud, respectively. After concatenating the 3D and 2D features, a machine learning algorithm, such as gradient boosting decision tree, is used to learn the nonlinear mapping relationship between the feature vector and the just-aware distortion threshold, thereby constructing a 3D point cloud texture just-aware distortion prediction model with high accuracy.

[0022] like Figure 2 The following is a basic flowchart of the just-aware distortion threshold prediction method for 3D point clouds provided in this embodiment. The just-aware distortion threshold prediction method includes the following steps:

[0023] Step 201: Extract 3D texture features based on the reference point cloud and the corresponding distorted point cloud.

[0024] Specifically, the three-dimensional texture features in this embodiment include: kurtosis features, brightness features, and color features. In this embodiment, a three-dimensional point cloud containing N points is given. Each point is a high-dimensional vector composed of geometric coordinates and attribute information. The i-th point in P is... in Representing geometric coordinates, This indicates color attribute information.

[0025] Originally, point clouds used RGB colors to represent the color attribute of each point. Because the RGB components are highly correlated and contain some redundancy, this invention converts RGB to the YUV color space, where Y represents luminance, and U and V represent the two chromaticity components. The conversion formula is as follows:

[0026]

[0027] It should be noted that the distorted point cloud in this embodiment only exhibits texture distortion; the point geometry information in the reference point cloud and the distorted point cloud is completely identical. In this embodiment, the texture parameters of the reference point cloud can be adjusted to obtain the corresponding distorted point cloud.

[0028] On the one hand, the above-mentioned extraction of 3D texture features based on reference point cloud and corresponding distorted point cloud includes: establishing point-based neighborhood relationships for reference point cloud and corresponding distorted point cloud respectively; obtaining the kurtosis index of each point in reference point cloud and distorted point cloud based on neighborhood relationships; averaging the kurtosis indices of all points in reference point cloud to obtain the overall kurtosis index of reference point cloud, and averaging the kurtosis indices of all points in distorted point cloud to obtain the overall kurtosis index of distorted point cloud; and calculating the difference between the overall kurtosis index of reference point cloud and the overall kurtosis index of distorted point cloud to obtain kurtosis features.

[0029] Specifically, in this embodiment, point p can be found in the reference point cloud and the distorted point cloud using the K-nearest neighbor method. i A set of neighboring points The distance between points is represented by Euclidean distance. In this embodiment, k is selected as 12. Therefore, point neighborhood relationships are established in both the reference point cloud and the distorted point cloud.

[0030] This embodiment extracts the kurtosis index for each neighborhood to measure the degree of deviation between a point and its neighbors. For point p in the point cloud... i The formula for calculating the kurtosis index of its brightness attribute is:

[0031]

[0032]

[0033] in, For point p i The neighborhood of L contains k points. j Let j be the brightness value of the j-th point. For the neighborhood The mean brightness of all points within the range. A larger kurtosis value indicates a greater deviation in the set of points within the neighborhood, while a smaller kurtosis value means a smaller deviation.

[0034] For both the reference and distorted point clouds, the kurtosis index of each point is averaged to obtain the overall kurtosis index of the point clouds. Next, the difference between the kurtosis indices of the reference and distorted point clouds is calculated using the following formula to measure the difference in neighborhood attribute distribution, thus obtaining feature f1.

[0035] On the other hand, the above-mentioned extraction of 3D texture features based on the reference point cloud and the corresponding distorted point cloud also includes: establishing point-based neighborhood relationships for the reference point cloud and the corresponding distorted point cloud respectively; based on the neighborhood relationships, obtaining the neighborhood average brightness difference of each point in the reference point cloud and the distorted point cloud respectively, as a brightness feature f2; wherein, the calculation formula for the neighborhood average brightness difference is expressed as:

[0036]

[0037] Where f2 represents the average brightness difference of the neighborhood, N represents the number of points in the point cloud, k represents the number of points in the neighborhood, and L i,j and These represent the brightness values ​​of the j-th point in the neighborhood of the i-th point in the reference point cloud and the distorted point cloud, respectively.

[0038] It is worth mentioning that the specific implementation method for establishing neighborhood relationships when extracting brightness features is the same as the implementation method when extracting kurtosis features, and will not be repeated in this embodiment.

[0039] Furthermore, the above-mentioned extraction of 3D texture features based on the reference point cloud and the corresponding distorted point cloud also includes: calculating the color information of the reference point cloud and the corresponding distorted point cloud based on a preset color information calculation formula to obtain color feature f3; wherein, the color information calculation formula is expressed as:

[0040]

[0041] rg = RG

[0042] by=0.5(R+G)-B

[0043] Where f3 represents color information, σ rg and μ rg Let σ represent the standard deviation and mean of rg, respectively. by and μ byrepresents the standard deviation and mean of by, respectively, and R, G, and B represent the color components of all points in the point cloud.

[0044] Step 202: Perform multi-view projection processing on the reference point cloud and the distorted point cloud respectively to obtain two-dimensional projection images from multiple different viewpoints.

[0045] Specifically, in this embodiment, based on a preset rotation matrix and different rotation angles, both the reference point cloud and the distorted point cloud are subjected to multi-view projection processing to obtain the corresponding two-dimensional projection image.

[0046] To simulate the visual effect of viewing a 3D point cloud by the human eye, the reference point cloud and the distorted point cloud are projected from multiple perspectives to obtain 2D images with multiple viewpoints. Unlike traditional methods that project the point cloud onto the six faces of a bounding box, this embodiment maps the 3D point cloud to N viewpoints, resulting in N projected images. For the geometric information (x, y, z) of the 3D point cloud P, rotations of rx, ry, and rz are performed along the x, y, and z axes, respectively. The rotation matrices are as follows:

[0047] R v =R z R y R x

[0048]

[0049]

[0050]

[0051] Among them, R v Let R represent the rotation matrix. x R y R z Let rx, ry, and rz represent the rotation matrices around the x-axis, y-axis, and z-axis, respectively, and rx, ry, and rz represent the rotation angles.

[0052] Therefore, by projecting the 3D point cloud P at intervals, a series of projected images I = {I n}, n=1,...,N, where N is the number of viewpoints.

[0053] It is worth mentioning that this embodiment can evaluate the importance index of all viewpoints to the image perception quality, and then perform subsequent two-dimensional texture feature extraction on multiple two-dimensional projection images of multiple viewpoints with importance indexes higher than the preset index threshold.

[0054] The information content contained in the projection images from different perspectives is different, and their contributions to the image perception quality are also different. Too many images will lead to too much subsequent computational effort. Therefore, it is necessary to screen the projected images and select the perspectives that are important for the image perception quality. In this embodiment, M perspectives can be selected according to the importance of each projection image, where M < N. The importance of each perspective image is calculated based on its saliency. In practical applications, an existing saliency calculation method, such as an image saliency calculation method based on histogram contrast (HC), can be used to obtain the saliency value of each image, or other saliency calculation methods can be used, and then M projection images I = {I n}, n = 1,..., M are selected therefrom. In this embodiment, M is selected as 10.

[0055] Step 203: Extract two-dimensional texture features based on the two-dimensional projection images of the reference point cloud and the distorted point cloud.

[0056] The two-dimensional texture features of this embodiment include: texture complexity features, texture masking features, and SIFT features.

[0057] On the one hand, extracting two-dimensional texture features based on the two-dimensional projection images of the reference point cloud and the distorted point cloud includes: calculating a plurality of random graphs for the plurality of two-dimensional projection images of the reference point cloud and the distorted point cloud respectively; obtaining the random histogram of each random graph, and then calculating the histogram difference between the random histograms of the reference point cloud and the distorted point cloud; performing max pooling on the histogram differences of different perspectives to obtain texture complexity features.

[0058] Considering the influence of the randomness of patterns in the image on its just-noticeable distortion threshold, in this embodiment, for a number of projection images obtained by projection previously, the random graph of the image is calculated to measure the texture complexity of the projection image. M random graphs Map ref,i , i = 1,..., M and Map dis,i , i = 1,..., M are calculated for the M projection images of the reference point cloud and the distorted point cloud respectively. The histogram of each random graph is obtained, and then the difference between the random graph histograms of the reference point cloud and the distorted point cloud is calculated. Max pooling is performed on the histogram interpolations of different perspectives to obtain the feature f4.

[0059] On the other hand, the above-mentioned extraction of two-dimensional texture features based on two-dimensional projected images of reference point cloud and distorted point cloud also includes: calculating pixel-level JND (Just Noticeable Distortion) models for multiple two-dimensional projected images of reference point cloud respectively; calculating the pixel value difference between two-dimensional projected images of reference point cloud and distorted point cloud from the same viewpoint; statistically analyzing the proportion of pixels whose pixel value difference exceeds the pixel-level JND threshold of the pixel-level JND model; and performing max pooling on the proportion of pixels exceeding the pixel-level JND threshold in different viewpoints to obtain texture masking features.

[0060] Due to the texture masking effect, the presence of one signal reduces the visibility of another signal; therefore, not all distortions in an image are perceived by the human eye. This embodiment considers brightness adaptation and contrast masking. First, it calculates the pixel-level JND model for each of the M projected images of the reference point cloud. Then, it calculates the pixel value difference between the projected images of the reference and distorted point clouds from the same viewpoint, comparing whether this difference exceeds the value in the pixel-level JND model of the reference point cloud, and statistically analyzes the proportion of pixels exceeding the pixel-level JND threshold. Finally, it performs max pooling on the proportion of pixels exceeding the pixel-level JND threshold from different viewpoints to obtain feature f5.

[0061] On the other hand, the above-mentioned extraction of two-dimensional texture features based on two-dimensional projected images of reference point cloud and distorted point cloud also includes: calculating SIFT feature descriptors for multiple two-dimensional projected images of reference point cloud and distorted point cloud respectively; performing feature matching on two-dimensional projected images of reference point cloud and distorted point cloud from corresponding viewpoints; obtaining the number of matching points in the two-dimensional projected images of each viewpoint; and performing max pooling on the number of matching points from different viewpoints to obtain SIFT features.

[0062] SIFT feature descriptors are calculated for several projected images obtained from the projection. Feature matching is performed on the projected images of the distorted point cloud corresponding to the viewpoint of the reference point cloud. The number of matching points in the projection images of the reference point cloud and the distorted point cloud for each viewpoint is calculated. Max pooling is performed on the number of matching points in different viewpoints to obtain feature f6.

[0063] Step 204: Concatenate the 3D texture features and 2D texture features to obtain a feature vector, and learn the mapping relationship between the feature vector and the just-perceptible distortion threshold based on the machine learning algorithm.

[0064] It is worth mentioning that the just-perceptible distortion prediction model in this embodiment can be viewed as a nonlinear mapping from the characteristics of the human visual system to the visibility threshold. In this embodiment, the modeling of the just-perceptible distortion prediction model can be regarded as a binary classification problem, namely, perceptible distortion and imperceptible distortion. Given an original point cloud P ref A distorted point cloud P distA mapping relationship is established between point cloud pairs and the perceptible distortion threshold, where 0 represents perceptible distortion and 1 represents imperceptible distortion. The visual sensitivity and visibility characteristics of the human visual system are mainly described by various masking effects. Therefore, considering the data characteristics of point clouds, a perceptible distortion perception computational model for texture is established by combining various masking effects and machine learning methods.

[0065] In this embodiment, the aforementioned six-dimensional features are normalized to their maximum and minimum values ​​and then concatenated to obtain a feature vector. Gradient-enhanced decision trees are used for feature selection and classification model building. For a given feature vector of a point cloud... The mapping function ψ(x,θ) is learned through gradient-enhanced decision trees, where θ represents the model parameters. The objective function to be optimized is expressed as:

[0066]

[0067] The loss function on the training set is expressed as:

[0068] l[ψ(x i ,θ),y i ]=y i log[ψ(x i ,θ)]+(1-y i )log[1-ψ(x i ,θ)]

[0069] Where N represents the number of points in the point cloud, Ω(θ) represents the regularization term used to constrain the model complexity, ψ(x,θ) represents the mapping function, and θ is the model parameter;

[0070] The feature vector is represented as The mapping relationship between the feature vector and the just-perceptible distortion threshold is expressed as:

[0071]

[0072] In this embodiment, a gradient-enhanced decision tree is used to learn a non-linear mapping relationship between feature vectors and just-perceptible distortion thresholds. The weights of each feature are learned on the training set, and feature selection and learning are performed based on these weights to learn the non-linear mapping relationship between point cloud texture features and corresponding JNDs.

[0073]

[0074] It should be noted that the mapping relationship between the feature vector and the just-aware distortion threshold obtained in this embodiment can be used to calculate the just-aware distortion threshold of the texture of the 3D point cloud to be processed, and then applied to perception-based 3D point cloud compression to provide visual quality comparable to the original point cloud at a lower bit rate.

[0075] It should be understood that the sequence number of each step in this embodiment does not imply the order in which the steps are executed. The execution order of each step should be determined by its function and internal logic, and should not constitute a unique limitation on the implementation process of this application embodiment.

[0076] Next, one embodiment of this application also provides a device for predicting the just-perceptible distortion threshold of three-dimensional point clouds. Figure 3 This is a functional module diagram of a just-perceptible distortion threshold prediction device according to an embodiment of this application. The just-perceptible distortion threshold prediction device can be used to implement the just-perceptible distortion threshold prediction method for 3D point clouds in the aforementioned embodiments, and mainly includes:

[0077] The first feature extraction module 301 is used to extract three-dimensional texture features based on the reference point cloud and the corresponding distorted point cloud; wherein, the three-dimensional texture features include: kurtosis features, brightness features and color features;

[0078] The point cloud projection module 302 is used to perform multi-view projection processing on the reference point cloud and the distorted point cloud respectively to obtain two-dimensional projection images from multiple different viewpoints.

[0079] The second feature extraction module 303 is used to extract two-dimensional texture features based on the two-dimensional projection image of the reference point cloud and the distorted point cloud; wherein, the two-dimensional texture features include: texture complexity features, texture masking features and SIFT features;

[0080] The model learning module 304 is used to concatenate the three-dimensional texture features and the two-dimensional texture features to obtain a feature vector, and learn the mapping relationship between the feature vector and the just-perceptible distortion threshold based on the machine learning algorithm.

[0081] It should be noted that the just-perceptible distortion threshold prediction methods in the foregoing embodiments can all be implemented based on the just-perceptible distortion threshold prediction device provided in this embodiment. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process of the just-perceptible distortion threshold prediction device described in this embodiment can be implemented by referring to the corresponding working process in the foregoing method embodiments, and will not be repeated here.

[0082] Based on the technical solution of the embodiments of this application described above, three-dimensional texture features are extracted from a reference point cloud and a corresponding distorted point cloud; the reference point cloud and the distorted point cloud are subjected to multi-view projection processing to obtain two-dimensional projection images from multiple different viewpoints; two-dimensional texture features are extracted from the two-dimensional projection images of the reference point cloud and the distorted point cloud; the three-dimensional texture features and the two-dimensional texture features are concatenated to obtain a feature vector, and a mapping relationship between the feature vector and the just-aware distortion threshold is learned based on a machine learning algorithm. Through the implementation of this application's solution, the mapping relationship between the feature vector and the just-aware distortion threshold is established by concatenating the three-dimensional and two-dimensional features of the point cloud and using machine learning, thus constructing a highly accurate three-dimensional point cloud just-aware distortion threshold prediction model. This provides effective guidance for three-dimensional point cloud encoding and ensures the visual quality of the encoded point cloud.

[0083] Figure 4 An electronic device is provided as an embodiment of this application. This electronic device can be used to implement the just-aware distortion threshold prediction method for 3D point clouds in the aforementioned embodiments. It mainly includes: a memory 401 and a processor 402. The memory 401 stores a computer program 403 that can run on the processor 402. The memory 401 and the processor 402 are communicatively connected. When the processor 402 executes the computer program 403, it implements the just-aware distortion threshold prediction method for 3D point clouds in the aforementioned embodiments. The number of processors 402 can be one or more.

[0084] The memory 401 can be a high-speed random access memory (RAM) or a non-volatile memory, such as a disk storage device. The memory 401 is used to store executable program code, and the processor 402 is coupled to the memory 401.

[0085] Furthermore, embodiments of this application also provide a computer-readable storage medium, which may be disposed in the electronic device described in the above embodiments, and the computer-readable storage medium may be as described above. Figure 4 The memory in the illustrated embodiment.

[0086] The computer-readable storage medium stores a computer program that, when executed by a processor, implements the perceptible distortion threshold prediction method for 3D point clouds described in the foregoing embodiments. Furthermore, the computer-readable storage medium can also be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), RAM, magnetic disk, or optical disk.

[0087] It should be understood that the apparatus and methods disclosed in the embodiments provided in this application can also be implemented in any other equivalent manner. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0088] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0089] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0090] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0091] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0092] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0093] The above is a description of the perceptible distortion threshold prediction method and product for 3D point clouds provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for predicting a threshold of just perceptible distortion of a three-dimensional point cloud, characterized in that: include: Extracting three-dimensional texture features based on the reference point cloud and the corresponding distorted point cloud; wherein the three-dimensional texture features include: kurtosis features, brightness features and color features; Perform multi-view projection processing on the reference point cloud and the distorted point cloud respectively to obtain two-dimensional projection images of multiple different view angles; Extracting two-dimensional texture features based on the two-dimensional projection images of the reference point cloud and the distorted point cloud; wherein the two-dimensional texture features include: texture complexity features, texture masking features and SIFT features; The three-dimensional texture feature and the two-dimensional texture feature are concatenated to obtain a feature vector, and a mapping relationship between the feature vector and a just perceptible distortion threshold is obtained based on a machine learning algorithm.

2. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of three-dimensional texture features based on the reference point cloud and the corresponding distorted point cloud includes: A point-based neighborhood relationship is established for the reference point cloud and the corresponding distorted point cloud; Based on the neighborhood relationship, respectively obtaining the kurtosis index of each point in the reference point cloud and the distorted point cloud; Based on the average of the kurtosis indexes of all points in the reference point cloud, an overall kurtosis index of the reference point cloud is obtained; and based on the average of the kurtosis indexes of all points in the distorted point cloud, an overall kurtosis index of the distorted point cloud is obtained; The difference between the overall kurtosis index of the reference point cloud and the overall kurtosis index of the distorted point cloud is calculated to obtain a kurtosis feature.

3. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of three-dimensional texture features based on the reference point cloud and the corresponding distorted point cloud includes: A point-based neighborhood relationship is established for the reference point cloud and the corresponding distorted point cloud; Based on the neighborhood relationship, the neighborhood average brightness difference of each point in the reference point cloud and the distorted point cloud is obtained as a brightness feature; wherein the calculation formula of the neighborhood average brightness difference is expressed as: Where f2 represents the average brightness difference of the neighborhood, N represents the number of points in the point cloud, k represents the number of points in the neighborhood, and L i,j and Respectively represent the brightness value of the j-th point in the neighborhood of the i-th point of the reference point cloud and the distorted point cloud.

4. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of three-dimensional texture features based on the reference point cloud and the corresponding distorted point cloud includes: Based on a preset color information calculation formula, the color information of the reference point cloud and the corresponding distorted point cloud is calculated to obtain the color feature; wherein the color information calculation formula is expressed as: rg=RG by=0.5(R+G)-B Wherein, f3 represents the color information, σ rg and μ rg Respectively represent the standard deviation and mean of rg, σ by and μ by They represent the standard deviation and mean of by respectively, and R, G, and B represent the color components of all points in the point cloud respectively.

5. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of two-dimensional texture features from the two-dimensional projection image based on the reference point cloud and the distorted point cloud includes: Calculating a plurality of random graphs respectively for the plurality of two-dimensional projection images of the reference point cloud and the distorted point cloud; Obtaining a random histogram of each of the random graphs, and then calculating a histogram difference between the random histograms of the reference point cloud and the distorted point cloud; Maximum pooling is performed on the histogram differences of different viewing angles to obtain texture complexity features.

6. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of two-dimensional texture features from the two-dimensional projection image based on the reference point cloud and the distorted point cloud includes: Calculating pixel-level JND models for each of the two-dimensional projection images of the reference point cloud; Calculating a pixel value difference value of the two-dimensional projection image at the same viewing angle between the reference point cloud and the distorted point cloud; Counting the proportion of pixels whose pixel value difference value exceeds the pixel level JND threshold of the pixel level JND model; The pixel ratio exceeding the pixel-level JND threshold in different viewing angles is subjected to maximum pooling to obtain texture masking features.

7. The method for predicting a threshold value of just perceptible distortion according to claim 1, characterized in that: The extracting of two-dimensional texture features from the two-dimensional projection image based on the reference point cloud and the distorted point cloud includes: Calculating SIFT feature descriptors for the reference point cloud and the plurality of two-dimensional projection images of the distorted point cloud respectively, and performing feature matching on the reference point cloud and the two-dimensional projection images of the distorted point cloud at corresponding viewing angles; Obtaining the number of matching points of the two-dimensional projection image at each viewing angle; The number of matching points at different perspectives is pooled to obtain SIFT features.

8. The method for predicting a threshold value of just perceptible distortion according to any one of claims 1 to 7, characterized in that: The machine learning algorithm is a gradient boosted decision tree, and the objective function to be optimized is expressed as: The loss function on the training set is expressed as: l[ψ(x i ,θ),y i ]=y i log[ψ(x i ,θ)]+(1-y i )log[1-ψ(x i ,i)] Where N represents the number of points in the point cloud, Ω(θ) represents the regularization term, ψ(x,θ) represents the mapping function, and θ is the model parameter; The feature vector is expressed as The mapping relationship between the feature vector and the just perceptible distortion threshold is expressed as:

9. An electronic device, characterized in that: The device comprises a memory and a processor, wherein: The processor is used to execute the computer program stored in the memory; When the processor executes the computer program, the steps in the method for predicting a just perceptible distortion threshold of a three-dimensional point cloud as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method for predicting a just perceptible distortion threshold of a three-dimensional point cloud as claimed in any one of claims 1 to 8 are implemented.

Images