Three-dimensional point cloud just noticeable distortion prediction method, device, equipment and medium
By performing voxel downsampling and feature extraction on point cloud datasets, and combining a deep learning model with a multi-head self-attention mechanism, the problem of low prediction accuracy caused by ignoring color information in existing technologies is solved, and more accurate 3D point cloud just-perceptible distortion prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-03
- Publication Date
- 2026-03-31
AI Technical Summary
Existing 3D point cloud just-perceptible distortion prediction methods ignore color information when considering the geometric properties of point clouds, resulting in low prediction accuracy. Furthermore, projection-based methods suffer from severe information loss during the projection process.
By performing voxel downsampling on the point cloud dataset, geometric and color features of the point cloud are extracted. A prediction model is trained using a deep learning model with a multi-head self-attention mechanism, and just-aware distortion prediction is performed by combining point cloud geometric and color information.
It improves the comprehensiveness and accuracy of predicting 3D point cloud distortions that can be detected, and can accurately determine whether the human eye can detect the distortion of the 3D point cloud to be predicted.
Smart Images
Figure CN121767697A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and medium for predicting perceptible distortion in three-dimensional point clouds. Background Technology
[0002] 3D point clouds are a commonly used data format in virtual reality and augmented reality, containing both 3D coordinates and color information, enabling a more accurate and complete representation of the objective world. However, due to limitations in objective conditions, noise or compression can impair the quality of point clouds during acquisition, transmission, storage, and rendering, affecting the user's visual experience. In visual physiology research, the human eye can only perceive point cloud data distortion when the distortion in the visual signal reaches a certain threshold; this is called just-perceptible distortion of point cloud data.
[0003] Currently, the main methods for predicting just-aware distortion in 3D point cloud data include point-based quality assessment methods and projection-based quality assessment methods. Point-based methods first match a reference point cloud with the distorted point cloud, then calculate the geometric distance between the neighborhoods of the matched points, and finally use the local gradient statistics of the point cloud to measure the distortion. However, this method only considers the geometric attributes of the point cloud, while color information plays an important role in human visual perception, leading to lower accuracy in just-aware distortion prediction. Projection-based quality assessment methods first project the point cloud perspective onto the six faces of a bounding box, then apply established image quality assessment metrics to calculate the quality score of the projected image. For example, color and depth features are extracted from the projected texture and depth maps, and these features are then fused to estimate the point cloud quality. However, projection-based quality assessment methods project 3D point clouds onto multiple 2D planes, which leads to information loss during the projection process. Furthermore, they only consider a limited number of point cloud viewpoints, resulting in low accuracy in predicting just-aware distortion from 3D point cloud data.
[0004] Therefore, improving the accuracy of predicting perceptible distortion in 3D point clouds has become an urgent problem to be solved. Summary of the Invention
[0005] This invention provides a method, apparatus, device, and medium for predicting just-perceptible distortion in three-dimensional point clouds, the main purpose of which is to solve the problem of poor accuracy in existing three-dimensional point cloud just-perceptible distortion prediction.
[0006] To achieve the above objectives, the present invention provides a 3D point cloud just-aware distortion prediction method, comprising: acquiring a point cloud dataset; performing voxel downsampling on the point cloud dataset to obtain a target point cloud dataset; calculating the point cloud geometric features of the target point cloud dataset based on the neighborhood points of each target point cloud data in the target point cloud dataset; calculating the point cloud attribute features of the target point cloud dataset based on the color components of each target point cloud data; training a pre-constructed just-aware distortion prediction model based on the point cloud geometric features and the point cloud attribute features to obtain a target point cloud distortion prediction model; and using the target point cloud distortion prediction model to perform just-aware distortion prediction on the 3D point cloud to be predicted to obtain the just-aware distortion prediction result of the 3D point cloud to be predicted.
[0007] This invention also provides a device for predicting just-aware distortion in 3D point clouds. The device includes: a voxel downsampling processing module for acquiring a point cloud dataset and performing voxel downsampling processing on the point cloud dataset to obtain a target point cloud dataset; a point cloud geometric feature calculation module for calculating the point cloud geometric features of the target point cloud dataset based on the neighborhood points of each target point cloud data in the target point cloud dataset; a point cloud attribute feature calculation module for calculating the point cloud attribute features of the target point cloud dataset based on the color components of each target point cloud data; a prediction model training module for training a pre-constructed just-aware distortion prediction model based on the point cloud geometric features and point cloud attribute features to obtain a target point cloud distortion prediction model; and a just-aware distortion prediction module for using the target point cloud distortion prediction model to perform just-aware distortion prediction on the 3D point cloud to be predicted, to obtain the just-aware distortion prediction result of the 3D point cloud to be predicted.
[0008] The present invention also provides an electronic device, comprising: a memory communicatively connected to at least one processor; wherein the processor is configured to execute a computer program stored in the memory; the memory stores a computer program executable by at least one processor, the computer program being executed by at least one processor to enable at least one processor to perform the above-described method for predicting just-perceptible distortion of a three-dimensional point cloud.
[0009] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the three-dimensional point cloud just-perceptible distortion prediction method described above.
[0010] This invention provides a target point cloud dataset by performing voxel downsampling on the point cloud dataset. This reduces the amount of point cloud data while preserving the overall geometric structure, thus improving computational efficiency. The invention calculates the point cloud geometric features and point cloud attribute features of each target point cloud data point in the target point cloud dataset, providing a multi-faceted description of the geometric and color texture features of the point cloud data. A pre-constructed just-aware distortion prediction model is then trained based on these geometric and attribute features to obtain a target point cloud distortion prediction model. This model can predict just-aware distortion of the point cloud data from both geometric and color attribute dimensions, effectively improving the comprehensiveness and accuracy of just-aware distortion prediction. Using the target point cloud distortion prediction model to predict the just-aware distortion of the 3D point cloud to be predicted allows for precise determination of whether the human eye can perceive the just-aware distortion of the 3D point cloud to be predicted, resulting in accurate just-aware distortion prediction results. Attached Figure Description
[0011] Figure 1 This is a flowchart illustrating a 3D point cloud just-perceptible distortion prediction method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the voxel downsampling process for a point cloud dataset provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the process for training a pre-built just-aware distortion prediction model according to an embodiment of the present invention. Figure 4 A functional block diagram of a three-dimensional point cloud perceptible distortion prediction device provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device that implements a method for predicting just-perceptible distortion in three-dimensional point clouds, according to an embodiment of the present invention.
[0012] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0013] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0014] This application provides a method for predicting just-aware distortion in 3D point clouds. The execution entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the 3D point cloud just-aware distortion prediction method can be executed by software or hardware installed on a terminal device or a server device; the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0015] Reference Figure 1 The diagram shown is a flowchart illustrating a 3D point cloud just-aware distortion prediction method according to an embodiment of the present invention. In this embodiment, the 3D point cloud just-aware distortion prediction method includes: S1. Obtain the point cloud dataset, perform voxel downsampling on the point cloud dataset, and obtain the target point cloud dataset.
[0016] In this embodiment of the invention, the point cloud dataset is a collection containing multiple three-dimensional point cloud data. The three-dimensional point cloud data may include original point cloud data and distorted point cloud data. For example, the number of three-dimensional point cloud data may be greater than 10,000. Since the number of point cloud data in the point cloud dataset is large, the amount of computation is very large. Therefore, the point cloud dataset is first downsampled to reduce the amount of computation and improve the computational efficiency while ensuring the geometric features of the point cloud data.
[0017] Each point cloud data is a high-dimensional vector consisting of three-dimensional geometric coordinates and R, G, and B color attribute information. The R, G, and B color attribute information are the color components of the three color channels: red (R), green (G), and blue (B).
[0018] Specifically, see Figure 2 As shown, voxel downsampling is performed on the point cloud dataset to obtain the target point cloud dataset, including: S21. Divide the point cloud dataset into multiple voxel grids using the preset voxel size; S22. Data allocation is performed based on the coordinate distance from each point cloud data point to each voxel grid in the point cloud dataset to obtain grid point cloud data; S23. Select voxel center points from each grid point cloud data and collect the voxel center points as the target point cloud dataset.
[0019] In this embodiment of the invention, the voxel size is the preset side length of each voxel grid when the point cloud dataset is divided into voxel grids (cubes); then, based on the coordinate range of the point cloud data in the point cloud dataset and the voxel size, the number of voxel grids along the X, Y, and Z axes is calculated respectively, and based on the number of voxel grids along the X, Y, and Z axes, the voxel grids are divided along the X, Y, and Z axes with the preset coordinate origin of the point cloud dataset, resulting in multiple voxel grids.
[0020] Furthermore, each point cloud data in the point cloud dataset is traversed, the coordinate distance from each point cloud data to the voxel center point of each voxel grid is calculated, and the point cloud data is assigned to the voxel grid with the closest coordinate distance to obtain the grid point cloud data included in each voxel grid.
[0021] In detail, the geometric center of each voxel grid is taken as the voxel center point, and a preset number of voxel center points are selected from multiple voxel center points as the target point cloud dataset.
[0022] In another optional embodiment of the present invention, the average coordinate of all point cloud coordinates within the voxel grid can be calculated, and the average coordinate is used as the voxel center point, which can preserve the geometric center position of the voxel grid to the greatest extent.
[0023] In this embodiment of the invention, by performing voxel downsampling on the point cloud dataset, the overall geometric structure can be preserved while reducing the amount of point cloud data, thereby improving computational efficiency.
[0024] S2. Calculate the point cloud geometric features of the target point cloud data based on the neighborhood points of each target point cloud data in the target point cloud dataset.
[0025] In this embodiment of the invention, the perceptible or imperceptible distortion caused by the quality loss of point clouds includes geometric distortion and texture distortion caused by color components. Geometric distortion is mainly manifested as displacement or missing unit points in the distorted point cloud. Therefore, the perceptible distortion of a three-dimensional point cloud can be effectively described by the geometric features of the point cloud.
[0026] Among them, the geometric features of the point cloud include the average curvature, Gaussian curvature, coordinate offset, and Laplacian coordinates of the target point cloud data.
[0027] Specifically, the point cloud geometric features of the target point cloud data are calculated based on the neighboring points of each target point cloud data in the target point cloud dataset. This includes: calculating the Euclidean distance between target point cloud data in the target point cloud dataset, and calculating the neighboring points of each target point cloud data based on the Euclidean distance; fitting a quadratic surface of each target point cloud data based on the neighboring points using the least squares method; calculating the normal curvature corresponding to the quadratic surface, and calculating the average curvature and Gaussian curvature of the target point cloud data based on the normal curvature; calculating the Laplacian coordinates based on the coordinate differences between the target point cloud data and its neighboring points; calculating the geometric coordinate center of the target point cloud dataset, and calculating the coordinate offset of the target point cloud data based on the geometric coordinate center; and combining the average curvature, Gaussian curvature, Laplacian coordinates, and coordinate offsets to obtain the point cloud geometric features of the target point cloud data.
[0028] In detail, the neighboring points of each target point cloud data can be calculated based on Euclidean distance using the K-nearest neighbor method. For example, the 12 point cloud data with the closest Euclidean distance can be selected as the neighboring points.
[0029] Furthermore, the least squares method involves substituting the target point cloud data and its corresponding neighborhood points into the general equation of a quadratic surface to construct a quadratic equation matrix. The least squares problem is then solved based on the quadratic equation matrix to obtain the quadratic surface of each target point cloud data.
[0030] In this embodiment of the invention, the normal curvature of each point cloud data on the quadric surface is calculated along the curve direction passing through that point. The maximum and minimum values of the normal curvature on the quadric surface corresponding to each target point cloud data are selected, and the average curvature and Gaussian curvature are calculated based on the maximum and minimum values of the normal curvature.
[0031] Specifically, the average curvature and Gaussian curvature of the target point cloud data can be calculated using preset average curvature formulas and Gaussian curvature formulas, where the average curvature formula and Gaussian curvature formula are expressed as follows:
[0032] in, Indicates the mean curvature. This represents the total number of sampling points in the target point cloud data. Indicates the first The maximum normal curvature of the quadratic surface corresponding to each target point cloud data. Indicates the first The minimum normal curvature of the quadratic surface corresponding to each target point cloud data; The Gaussian curvature formula is expressed as:
[0033] in, This represents Gaussian curvature.
[0034] In detail, each target point cloud data represents a sampling point; therefore, the total number of sampling points is... That is, the total number of point cloud data in the target point cloud data.
[0035] In this embodiment of the invention, the Laplace coordinates represent the first... The weighted average of the differences between the coordinates of a target point cloud data and the coordinates of other target point cloud data in its neighborhood.
[0036] Specifically, the Laplacian coordinates can be calculated using a pre-defined Laplacian coordinate calculation formula, which is expressed as follows:
[0037] in, Indicates the first Laplacian coordinates of the target point cloud data. Indicates the first The three-dimensional coordinates corresponding to each target point cloud data Indicates the first The first target point cloud data Neighboring points, Indicates the first The three-dimensional coordinates of each neighboring point This represents the total number of neighboring points.
[0038] In detail, the mean of the three-dimensional coordinates of the target point cloud data used in the target point cloud dataset is used as the geometric coordinate center of the target point cloud dataset. Then, the Euclidean distance from each target point cloud data to the geometric coordinate center is calculated, and the average of all Euclidean distances is calculated as the coordinate offset.
[0039] Specifically, the coordinate offset can be calculated using a preset coordinate offset calculation formula, where the coordinate offset formula is expressed as:
[0040] in, Indicates the coordinate offset. This represents the total number of sampling points in the target point cloud data. Indicates the first The three-dimensional coordinates of the target point cloud data To the geometric coordinate center Euclidean distance.
[0041] In this embodiment of the invention, the geometric features of the point cloud data can be described from multiple perspectives by using the average curvature, Gaussian curvature, coordinate offset, and Laplacian coordinates of each target point cloud data, thereby improving the accuracy of just-perceptible distortion prediction.
[0042] S3. Calculate the point cloud attribute features of the target point cloud data based on the color components of each target point cloud data.
[0043] In this embodiment of the invention, the color component is the color component of the three color channels (red, green, and blue) included in the R, G, and B color attribute information of each target point cloud data. Each color component is usually represented by an integer from 0 to 255.
[0044] Among them, the point cloud attribute features include the mean and standard deviation of the color components of all target point cloud data in the target point cloud data.
[0045] Specifically, the point cloud attribute features of the target point cloud data are calculated based on the color components of each target point cloud data, including: calculating the mean value of the color components of the target point cloud data in each color channel, and constructing the mean attribute feature based on the mean value of the color components; calculating the standard deviation of the color components in each color channel based on the color components of the target point cloud data, and constructing the standard deviation attribute feature based on the standard deviation of the color components; and combining the mean attribute feature and the standard deviation attribute feature to obtain the point cloud attribute features of the target point cloud data.
[0046] In this embodiment of the invention, the mean values of the color components of the target point cloud data in the three color channels of red (R), green (G), and blue (B) are calculated respectively. The mean values of the color components are then concatenated to obtain the mean attribute feature. The standard deviation of each color channel is then calculated based on the mean values of the color components of the color channels to obtain the color component standard deviation. The standard deviations of the color components are then concatenated to obtain the standard deviation attribute feature.
[0047] In detail, the color and texture features corresponding to point cloud data can be restored through point cloud attribute features, thereby representing the sensitivity of the human visual system to color and more accurately judging whether the human eye can perceive the just-perceptible distortion after point cloud processing.
[0048] S4. Train a pre-constructed just-aware distortion prediction model based on the geometric features and attribute features of the point cloud to obtain the target point cloud distortion prediction model.
[0049] In this embodiment of the invention, the pre-constructed just-aware distortion prediction model is a deep learning model with a multi-head self-attention mechanism, including multiple parallel single-head self-attention modules and a neural network. The neural network includes multiple neurons, and the point cloud geometric features and point cloud attribute features are nonlinearly mapped through a hidden layer composed of multiple neurons, thereby classifying the point cloud geometric features and point cloud attribute features to obtain the just-aware distortion prediction result of the target point cloud data. For example, the just-aware distortion prediction result includes imperceptible distortion that is imperceptible to the human eye, or perceptible distortion that is clearly perceptible to the human eye.
[0050] Specifically, see Figure 3As shown, a pre-constructed just-aware distortion prediction model is trained based on the geometric and attribute features of the point cloud to obtain the target point cloud distortion prediction model, including: S31. Perform feature concatenation on the geometric features and attribute features of the point cloud to obtain the point cloud features of each target point cloud data. S32. Perform multi-head self-attention calculation on the point cloud features to obtain attention feature weights, and perform feature fusion on the point cloud features based on the attention feature weights to obtain multi-head attention features. S33. Use the hidden layer in the just-perceptible distortion prediction model to predict the neurons of multi-head attention features and obtain the prediction distortion results. S34. Calculate the loss value of the just-perceptible distortion prediction model based on the predicted distortion result and the preset real distortion result of the target point cloud data. S35. Optimize the parameters of the just-perceptible distortion prediction model based on the loss value to obtain the target point cloud distortion prediction model.
[0051] In this embodiment of the invention, the point cloud geometric features and point cloud attribute features are first normalized to their maximum and minimum values. Then, the features are concatenated to obtain the point cloud features. Multiple parallel attention mechanisms are used to perform multi-head self-attention calculation to obtain the attention feature weights of each attention mechanism. The point cloud features are weighted according to the attention feature weights to obtain the features of each attention mechanism. The features of each attention mechanism are concatenated to obtain the multi-head attention features.
[0052] Furthermore, the neural network composed of multiple layers of neurons in the hidden layer of the just-aware distortion prediction model transforms low-dimensional multi-head attention features into high-dimensional, abstract features relevant to just-aware distortion prediction through the multiple layers of neurons in the neural network. The activation function is then used to predict the relevant features to obtain the predicted distortion result.
[0053] In detail, the predicted distortion result is the mapping relationship between the perceptible distortion threshold of each target point cloud data and the point cloud features. For example, a mapping relationship of 0 indicates that the target point cloud data has perceptible distortion, and 1 indicates that it has imperceptible distortion.
[0054] The just-perceptible distortion threshold is a quantitative value that measures whether the distortion of the target point cloud data can be perceived by the human eye and what the minimum perceptible degree of distortion is. If the actual degree of distortion of the target point cloud data is less than the just-perceptible distortion threshold, it means that the target point cloud data is imperceptible distortion that cannot be perceived by the human eye. If the actual degree of distortion of the target point cloud data is greater than or equal to the just-perceptible distortion threshold, it means that the target point cloud data is perceptible distortion that can be clearly perceived by the human eye.
[0055] Preferably, the target point cloud data contains both geometric and color attribute features. Therefore, the predicted distortion result includes both geometric and color attribute distortion dimensions. For example, the perceptible geometric distortion threshold for edge points is relatively small, and the perceptible color attribute distortion threshold for high-saturation regions is also relatively small, indicating that even small geometric and color deviations will lead to perceptible distortion.
[0056] Furthermore, the loss value of the just-aware distortion prediction model can be calculated using the cross-entropy loss function. The model parameters in the just-aware distortion prediction model can be optimized based on the loss value until the loss value is less than the preset loss value threshold, thus obtaining the target point cloud distortion prediction model.
[0057] In this embodiment of the invention, point cloud geometric features and point cloud attribute features can be used to predict just-aware distortion from two dimensions: geometric and color attributes, thereby effectively improving the comprehensiveness and accuracy of just-aware distortion prediction.
[0058] S5. Using the target point cloud distortion prediction model, perform just-aware distortion prediction on the 3D point cloud to be predicted, and obtain the just-aware distortion prediction result of the 3D point cloud to be predicted.
[0059] In this embodiment of the invention, the three-dimensional point cloud to be predicted is a point cloud in a point cloud dataset containing both three-dimensional geometric information and color attribute information of a target scene or object after point cloud processing. For example, a point cloud processed by compression, noise interference, registration error, etc. By performing just-perceptible distortion prediction on the three-dimensional point cloud to be predicted, the just-perceptible distortion thresholds of the geometric dimension and color attribute dimension, as well as the quantitative prediction results of whether the visual distortion of the three-dimensional point cloud to be predicted can be perceived by the human eye can be obtained.
[0060] Specifically, the target point cloud distortion prediction model is used to perform just-aware distortion prediction on the 3D point cloud to be predicted, and the just-aware distortion prediction result of the 3D point cloud to be predicted is obtained. This includes: extracting the geometric features and attribute features of the 3D point cloud to be predicted; performing feature stitching on the geometric features and attribute features to obtain the features of the point cloud to be predicted; and using the target point cloud distortion prediction model to perform multi-head attention calculation and neuron prediction on the features of the point cloud to be predicted, and obtaining the just-aware distortion prediction result of the 3D point cloud to be predicted.
[0061] In detail, the geometric features include the mean curvature, Gaussian curvature, Laplacian coordinates, and coordinate offset of the 3D point cloud to be predicted, and the attribute features include the mean attribute features and the standard deviation attribute features. Feature extraction can be performed using the methods described above for calculating the point cloud geometric features and point cloud attribute features of each target point cloud data, which will not be elaborated here.
[0062] Furthermore, the multi-head self-attention module in the target point cloud distortion prediction model is used to perform feature weighting on the features of the point cloud to be predicted, weakening features in the point cloud to be predicted that are irrelevant to just-aware distortion prediction, focusing on key features, and improving the accuracy of subsequent just-aware distortion prediction result calculation.
[0063] In this embodiment of the invention, the just-perceptible distortion prediction result includes the just-perceptible distortion threshold of the three-dimensional point cloud to be predicted, and the quantitative prediction result of whether it can be perceived by the human eye in the geometric dimension and the color attribute dimension (texture). For example, imperceptible distortion in the geometric dimension and the color attribute dimension that the human eye cannot perceive, or perceptible distortion that the human eye can clearly perceive.
[0064] For example, in the geometric dimension, the perceptible distortion threshold for the 3D point cloud to be predicted is 0.8mm. If the movement of the coordinate points between the 3D point cloud to be predicted and the preset original 3D point cloud exceeds 0.8mm, it indicates that the 3D point cloud to be predicted has perceptible distortion in the geometric dimension, otherwise it is imperceptible distortion. In the color attribute dimension, the perceptible distortion threshold is 15. If the pixel grayscale difference between the color component of the 3D point cloud to be predicted and the preset original 3D point cloud exceeds 15, it indicates that the 3D point cloud to be predicted has perceptible distortion in the color attribute dimension, otherwise it is imperceptible distortion.
[0065] In this embodiment of the invention, the target point cloud distortion prediction model is used to perform just-aware distortion prediction on the three-dimensional point cloud to be predicted. Just-aware distortion prediction can be performed from two dimensions: geometric and color attributes, which can effectively improve the accuracy of just-aware distortion prediction.
[0066] like Figure 4 The diagram shown is a functional block diagram of a three-dimensional point cloud perceptible distortion prediction device provided in an embodiment of the present invention.
[0067] The 3D point cloud just-aware distortion prediction device 400 of the present invention can be installed in an electronic device. Depending on the functions implemented, the 3D point cloud just-aware distortion prediction device 400 may include a voxel downsampling processing module 401, a point cloud geometric feature calculation module 402, a point cloud attribute feature calculation module 403, a prediction model training module 404, and a just-aware distortion prediction module 405. A module, also referred to as a unit, is a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and is stored in the memory of the electronic device.
[0068] In this embodiment, the functions of each module / unit are as follows: The voxel downsampling processing module 401 is used to acquire a point cloud dataset, perform voxel downsampling processing on the point cloud dataset, and obtain a target point cloud dataset. The point cloud geometric feature calculation module 402 is used to calculate the point cloud geometric features of the target point cloud data based on the neighborhood points of each target point cloud data in the target point cloud dataset. The point cloud attribute feature calculation module 403 is used to calculate the point cloud attribute features of the target point cloud data based on the color components of each target point cloud data. The prediction model training module 404 is used to train a pre-constructed just-aware distortion prediction model based on the geometric features and attribute features of the point cloud to obtain the target point cloud distortion prediction model. The just-aware distortion prediction module 405 is used to perform just-aware distortion prediction on the three-dimensional point cloud to be predicted using the target point cloud distortion prediction model, and obtain the just-aware distortion prediction result of the three-dimensional point cloud to be predicted.
[0069] In detail, each module in the three-dimensional point cloud just-perceptible distortion prediction device 400 of this embodiment of the invention adopts the same characteristics as described above during use. Figures 1 to 3 The three-dimensional point cloud in the model can be predicted using the same technical means as the distortion detection method, and can produce the same technical effect, which will not be elaborated here.
[0070] like Figure 5 The diagram shown is a structural schematic of an electronic device that implements a method for predicting just-perceptible distortion in three-dimensional point clouds, according to an embodiment of the present invention.
[0071] The electronic device 500 may include a processor 501, a memory 502, a communication bus 503, and a communication interface 504. It may also include a computer program stored in the memory and capable of running on the processor, such as a method program for predicting perceptible distortion of 3D point clouds.
[0072] In some embodiments, the processor 501 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.
[0073] The memory 502 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 502 may be an internal storage unit of an electronic device, such as a portable hard drive of the electronic device.
[0074] The communication bus 503 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 502 and at least one processor 501.
[0075] The communication interface 504 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface.
[0076] Figure 5 The image only shows electronic devices with components; it will be understood by those skilled in the art that... Figure 5 The structure shown does not constitute a limitation on the electronic device 500, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0077] Specifically, the processor's implementation method for the above instructions can be found in the description of the relevant steps in the corresponding embodiments in the accompanying drawings, and will not be repeated here.
[0078] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0079] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0080] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0081] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0082] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0083] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application devices that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0084] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.
[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for predicting just-perceptible distortion in three-dimensional point clouds, characterized in that, The method includes: Obtain a point cloud dataset, and perform voxel downsampling on the point cloud dataset to obtain the target point cloud dataset; Calculate the point cloud geometric features of the target point cloud data based on the neighborhood points of each target point cloud data in the target point cloud dataset; Calculate the point cloud attribute features of the target point cloud data based on the color components of each target point cloud data; A pre-constructed just-aware distortion prediction model is trained based on the geometric features and attribute features of the point cloud to obtain the target point cloud distortion prediction model. The target point cloud distortion prediction model is used to perform just-aware distortion prediction on the three-dimensional point cloud to be predicted, and the just-aware distortion prediction result of the three-dimensional point cloud to be predicted is obtained.
2. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 1, characterized in that, The process of performing voxel downsampling on the point cloud dataset to obtain the target point cloud dataset includes: The point cloud dataset is divided into multiple voxel grids using a preset voxel size; Data is allocated based on the coordinate distance from each point cloud data in the point cloud dataset to each voxel grid to obtain grid point cloud data; Voxel center points are selected from each of the grid point cloud data, and the voxel center points are collected as the target point cloud dataset.
3. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 1, characterized in that, The step of calculating the point cloud geometric features of the target point cloud data based on the neighborhood points of each target point cloud data in the target point cloud dataset includes: Calculate the Euclidean distance between target point cloud data in the target point cloud dataset, and calculate the neighborhood points of each target point cloud data based on the Euclidean distance; A quadratic surface is fitted to each target point cloud data based on the neighborhood points using the least squares method; Calculate the normal curvature corresponding to the quadratic surface, and calculate the average curvature and Gaussian curvature of the target point cloud data based on the normal curvature; Calculate the Laplace coordinates based on the coordinate differences between the target point cloud data and the neighboring points; Calculate the geometric coordinate center of the target point cloud dataset, and calculate the coordinate offset of the target point cloud data based on the geometric coordinate center; The point cloud geometric features of the target point cloud data are obtained by combining the average curvature, the Gaussian curvature, the Laplacian coordinates, and the coordinate offset.
4. The three-dimensional point cloud perceptible distortion prediction method as described in claim 3, characterized in that, The step of calculating the normal curvature corresponding to the quadratic surface, and calculating the average curvature and Gaussian curvature of the target point cloud data based on the normal curvature, includes: The average curvature and Gaussian curvature of the target point cloud data are calculated using a preset average curvature formula and a Gaussian curvature formula, wherein the average curvature formula is expressed as: in, Indicates the mean curvature. This represents the total number of sampling points in the target point cloud data. Indicates the first The maximum normal curvature of the quadratic surface corresponding to each target point cloud data. Indicates the first The minimum normal curvature of the quadratic surface corresponding to each target point cloud data; The Gaussian curvature formula is expressed as follows: in, This represents Gaussian curvature.
5. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 3, characterized in that, The step of calculating the Laplacian coordinates based on the coordinate differences between the target point cloud data and the neighboring points includes: The Laplacian coordinates are calculated using a preset Laplacian coordinate calculation formula, wherein the Laplacian coordinate calculation formula is expressed as follows: in, Indicates the first Laplacian coordinates of the target point cloud data. Indicates the first The three-dimensional coordinates corresponding to each target point cloud data Indicates the first The first target point cloud data Neighboring points, Indicates the first The three-dimensional coordinates of each neighboring point This represents the total number of neighboring points.
6. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 3, characterized in that, The step of calculating the coordinate offset of the target point cloud data based on the geometric coordinate center includes: The coordinate offset is calculated using a preset coordinate offset calculation formula, wherein the coordinate offset formula is expressed as: in, Indicates the coordinate offset. This represents the total number of sampling points in the target point cloud data. Indicates the first The three-dimensional coordinates of the target point cloud data To the geometric coordinate center Euclidean distance.
7. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 1, characterized in that, The step of calculating the point cloud attribute features of the target point cloud data based on the color components of each target point cloud data includes: Calculate the mean value of the color components in each color channel of the target point cloud data, and construct the mean value attribute feature based on the mean value of the color components; Calculate the standard deviation of each color component for each color channel based on the color components of the target point cloud data, and construct a standard deviation attribute feature based on the standard deviation of the color components. By combining the mean attribute features and the standard deviation attribute features, the point cloud attribute features of the target point cloud data are obtained.
8. The three-dimensional point cloud perceptible distortion prediction method as described in claim 1, characterized in that, The step of training a pre-constructed just-aware distortion prediction model based on the point cloud geometric features and the point cloud attribute features to obtain a target point cloud distortion prediction model includes: The point cloud geometric features and the point cloud attribute features are concatenated to obtain the point cloud features of each target point cloud data. Multi-head self-attention calculation is performed on the point cloud features to obtain attention feature weights. Feature fusion is then performed on the point cloud features based on the attention feature weights to obtain multi-head attention features. The hidden layer in the just-perceptible distortion prediction model is used to perform neuron prediction on the multi-head attention features to obtain the prediction distortion result. The loss value of the just-perceptible distortion prediction model is calculated based on the predicted distortion result and the preset real distortion result of the target point cloud data. The parameters of the just-perceptible distortion prediction model are optimized based on the loss value to obtain the target point cloud distortion prediction model.
9. The three-dimensional point cloud just-perceptible distortion prediction method as described in claim 1, characterized in that, The step of using the target point cloud distortion prediction model to perform just-aware distortion prediction on the 3D point cloud to be predicted, and obtaining the just-aware distortion prediction result of the 3D point cloud to be predicted, includes: Extract the geometric and attribute features of the three-dimensional point cloud to be predicted; The geometric features and the attribute features are concatenated to obtain the point cloud features to be predicted. The target point cloud distortion prediction model is used to perform multi-head attention calculation and neuron prediction on the features of the point cloud to be predicted, so as to obtain the just-perceptible distortion prediction result of the three-dimensional point cloud to be predicted.
10. A device for predicting perceptible distortion in three-dimensional point clouds, characterized in that, The device includes: The voxel downsampling processing module is used to acquire a point cloud dataset, perform voxel downsampling processing on the point cloud dataset, and obtain a target point cloud dataset. The point cloud geometric feature calculation module is used to calculate the point cloud geometric features of the target point cloud data based on the neighborhood points of each target point cloud data in the target point cloud dataset. The point cloud attribute feature calculation module is used to calculate the point cloud attribute features of the target point cloud data based on the color components of each target point cloud data. The prediction model training module is used to train a pre-constructed just-aware distortion prediction model based on the geometric features and attribute features of the point cloud to obtain the target point cloud distortion prediction model. The just-aware distortion prediction module is used to perform just-aware distortion prediction on the three-dimensional point cloud to be predicted using the target point cloud distortion prediction model, and obtain the just-aware distortion prediction result of the three-dimensional point cloud to be predicted.
11. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The processor is used to execute computer programs stored in the memory; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the three-dimensional point cloud just-aware distortion prediction method as described in any one of claims 1 to 9.
12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional point cloud just-perceptible distortion prediction method according to any one of claims 1 to 9.