Point cloud spatial position optimization method and system based on detail fine tuning network
By using the local and global optimization modules of the fine-tuning network to perform feature fusion on point clouds, the problem of degraded point cloud data quality was solved, high-precision point cloud localization was achieved, and a theoretical basis for point cloud densification was provided.
Patent Information
- Application Number
- CN202511043323.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-28
- Publication Date
- 2025-11-14
AI Technical Summary
Existing methods generate dense point clouds of poor quality, which cannot meet the requirements of high-precision positioning. Furthermore, point cloud data is easily affected by environmental noise during the acquisition process, leading to a decrease in positioning accuracy.
A method based on fine-tuning networks is adopted to optimize the spatial location of point clouds through local optimization modules and global optimization modules. Global feature maps are extracted using self-attention units, and feature fusion is performed by combining multi-scale channel attention units to optimize the density and quality of point clouds.
Without introducing excessive redundant data, the density and quality of point clouds were improved, the positioning accuracy was enhanced, and the noise optimization and hole problems in the point cloud upsampling process were solved, providing a theoretical basis for subsequent point cloud densification research.
Smart Images
Figure CN120953540A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of point cloud processing and computer vision technology, and in particular to a method and system for optimizing the spatial location of point clouds based on detail fine-tuning networks. Background Technology
[0002] A point cloud is a data set composed of a large number of discrete points in three-dimensional space, each with unique coordinate information. Each point is typically represented by three coordinate values (x, y, z), indicating its position in three-dimensional space. These points can have additional attribute information, such as color (RGB), intensity, and normal direction, to describe additional features of the point cloud. Point cloud processing refers to various operations and analyses performed on point cloud data acquired from three-dimensional sensors (such as LiDAR, depth cameras, or stereo vision) to extract, analyze, optimize, or transform its geometric features. Point cloud processing has wide applications in fields such as autonomous driving, robotics, 3D modeling, and Geographic Information Systems (GIS). However, point cloud data is easily affected by environmental noise during acquisition, leading to a decrease in data quality and indirectly affecting the final positioning accuracy. Existing methods mainly focus on generating dense point clouds through point cloud upsampling, but the generated point clouds are of poor quality and cannot meet the needs of high-precision positioning. Therefore, it is necessary to optimize the location space of dense point clouds to obtain high-quality point cloud data. Thus, an effective point cloud upsampling optimization method is urgently needed to solve the above problems. Summary of the Invention
[0003] To address the aforementioned problems, the present invention aims to provide a point cloud spatial location optimization method and system based on fine-tuning networks, which improves the density and quality of point clouds without introducing excessive redundant data, thereby optimizing positioning accuracy.
[0004] The technical solution provided by this invention is: a point cloud spatial location optimization method based on detail fine-tuning network, comprising the following steps: Fine-tuning the network from a coarse-dense point cloud ensemble and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; A collection of coarse dense point clouds and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
[0005] Preferably, the coarse dense point cloud set is optimized using a local optimization module and a global optimization module. Spatial location optimization of point clouds further includes: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. : Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
[0006] Preferably, the multi-scale channel attention unit uses two attention branches to acquire features at different levels and combines them with global context information to achieve effective selection and fusion of features.
[0007] Preferably, utilizing self-attention units to extract global feature maps to achieve global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features.
[0008] Preferably, the output is the optimized global feature. Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globaljF P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in the intermediate features. Indicates the weighting coefficient. This indicates the characteristics after fusion.
[0009] Based on the same concept, this invention provides a point cloud spatial location optimization system based on detail-fine-tuning networks, comprising: The local optimization module is used to fine-tune the network to achieve a coarse-to-dense point cloud ensemble. and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; The global optimization module is used to optimize coarse-grained dense point cloud datasets. and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
[0010] Preferably, the local optimization module further includes the following for spatial location optimization: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
[0011] Preferably, the global optimization module utilizes self-attention units to extract global feature maps, and the global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features; Output optimized global features Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in the intermediate features. Indicates the weighting coefficient. This indicates the characteristics after fusion.
[0012] Based on the same concept, the present invention provides an electronic device, comprising: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the point cloud spatial location optimization method based on detail-tuning networks as described above.
[0013] Based on the same concept, the present invention also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the point cloud spatial location optimization method based on detail fine-tuning network described above.
[0014] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: This invention proposes a point cloud spatial location optimization method based on fine-tuning networks, which solves the noise optimization and hole problems in the point cloud upsampling process, providing a theoretical basis for subsequent point cloud densification research. This invention first optimizes the coarsely expanded point cloud features based on local spatial geometry. Then, it optimizes the features based on the overall spatial geometry, finally outputting optimized global features, effectively optimizing the spatial location of the point cloud. Attached Figure Description
[0015] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 The network architecture diagram was fine-tuned for the details of this invention; Figure 2 This is a structural diagram of a local optimization module of the present invention; Figure 3 This is a structural diagram of the multi-scale channel attention unit of the present invention; Figure 4 This is a structural diagram of the global optimization module of the present invention; Figure 5 This is a structural diagram of the self-attention unit of the present invention; Figure 6 This is a diagram showing the experimental results of point cloud spatial location optimization according to the present invention. Detailed Implementation
[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.
[0017] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.
[0018] Example like Figure 1 As shown, this embodiment provides a point cloud spatial location optimization method based on detail-tuned networks, including the following steps: Fine-tuning the network from a coarse-dense point cloud ensemble and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; A collection of coarse dense point clouds and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
[0019] This scheme pioneers a hybrid architecture of "local explicit deformation field + global implicit attention," which retains interpretable point-by-point offset control while incorporating the black-box advantages of deep learning, achieving a Pareto optimal solution in terms of both accuracy and efficiency. The point cloud spatial location optimization method based on detailed fine-tuning networks proposed in this invention solves the noise optimization and hole problems in the point cloud upsampling process, providing a theoretical basis for subsequent point cloud densification research. This invention first optimizes the coarsely expanded point cloud features based on local spatial geometry. Then, it optimizes the features based on the overall spatial geometry, finally outputting optimized global features, effectively optimizing the spatial location of the point cloud.
[0020] The overall architecture of the network is as follows Figure 1 As shown.
[0021] exist Figure 1 middle, It is a feature optimized for spatial location, dimension and Consistent, This represents a dense set of points. Due to the diverse spatial distribution of 3D point clouds, compared to directly analyzing features... Regression is performed to obtain the corresponding point cloud coordinates, and the deviation of the point cloud coordinates is obtained through residual learning. Then superimposed on The effect is better on the surface.
[0022] Preferably, the coarse dense point cloud set is optimized using a local optimization module and a global optimization module. Spatial location optimization of point clouds further includes: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. : Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
[0023] The goal of the local optimization module is to optimize features based on local spatial geometry. In the specific implementation process, firstly in and The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. The specific expression is as follows: Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension.
[0024] Subsequently, these feature spaces are input into a depth map convolutional network to extract local feature structures and morphologies at different scales. Next, an MLP is used to concatenate the local features at different scales, and regression is used to obtain the weights corresponding to different scales. Features at different scales based on The features are combined using a weighted summation method. Finally, a multi-scale channel attention unit is used to perform cross-scale fusion of the feature combinations, resulting in spatially optimized local features. The specific structure of the local optimization module is as follows: Figure 2 As shown.
[0025] Preferably, the multi-scale channel attention unit uses two attention branches to acquire features at different levels and combines them with global context information to achieve effective selection and fusion of features.
[0026] exist Figure 2 In this model, the multi-scale channel attention unit is a crucial component of the attention feature fusion network. This unit enhances the model's ability to recognize multi-scale objects by adaptively adjusting the importance of feature channels at different scales. It utilizes two attention branches to acquire features at different levels and combines them with global contextual information to achieve effective feature selection and fusion. This helps the algorithm better identify and distinguish objects of different scales and complexities. (Figure 1) The numbers w1, w2, and w3 represent the feature dimensions extracted by the depth graph convolutional network. This represents the weights corresponding to features at different scales. The specific structure of a multi-scale channel attention unit is as follows: Figure 3 As shown.
[0027] Preferably, utilizing self-attention units to extract global feature maps to achieve global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features.
[0028] Preferably, the output is the optimized global feature. Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in the intermediate features. Indicates the weighting coefficient. This indicates the characteristics after fusion.
[0029] The goal of the global optimization module is to optimize features based on the overall spatial geometry. To achieve better results, the same will be true. and Features after combination As input data, this module utilizes self-attention units to extract global feature maps, achieving high-quality global spatial location optimization, and ultimately outputting optimized global features. The specific structure of the network is as follows: Figure 4 As shown.
[0030] exist Figure 4In this paper, the self-attention unit is a crucial component of the self-attention generative adversarial network (GAN). The self-attention unit enables the network to focus more on important parts of the input data, thereby generating higher-quality features. Through the self-attention mechanism, the network can learn the inherent structure and relationships of the input data, allowing it to better understand and simulate data distribution, thus producing more realistic and diverse samples in the generation task. Furthermore, the discriminator can more accurately enforce complex geometric constraints on the global image structure. This directly benefits the global optimization module in this invention, which needs to consider the overall spatial geometry to optimize features. The specific structure of the self-attention unit is as follows: Figure 5 As shown.
[0031] Figure 5 In this diagram, fself, gself, hself, and vself represent different feature spaces used for the attention mechanism computation. The intermediate features obtained through these feature space transformations are computed as follows: Among them, W f W g and W h The weight matrix is a learnable matrix. , and This represents intermediate features. The final output feature F P3global It can be represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in the intermediate features. Indicates the weighting coefficient. This indicates the characteristics after fusion.
[0032] The point cloud spatial location optimization method based on detail fine-tuning network proposed in this invention is used to improve the quality of acquired point cloud data. The experimental results are as follows: Figure 6 As shown.
[0033] Second Embodiment Based on the same concept, this invention provides a point cloud spatial location optimization system based on detail-fine-tuning networks, comprising: The local optimization module is used to fine-tune the network to achieve a coarse-to-dense point cloud ensemble. and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; The global optimization module is used to optimize coarse-grained dense point cloud datasets. and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
[0034] This scheme pioneers a hybrid architecture of "local explicit deformation field + global implicit attention," which retains interpretable point-by-point offset control while incorporating the black-box advantages of deep learning, achieving a Pareto optimal solution in terms of both accuracy and efficiency. The point cloud spatial location optimization method based on detailed fine-tuning networks proposed in this invention solves the noise optimization and hole problems in the point cloud upsampling process, providing a theoretical basis for subsequent point cloud densification research. This invention first optimizes the coarsely expanded point cloud features based on local spatial geometry. Then, it optimizes the features based on the overall spatial geometry, finally outputting optimized global features, effectively optimizing the spatial location of the point cloud.
[0035] Preferably, the local optimization module further includes the following for spatial location optimization: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
[0036] The multi-scale channel attention unit is a crucial component of attention feature fusion networks. This unit enhances the model's ability to recognize multi-scale targets by adaptively adjusting the importance of feature channels at different scales. It utilizes two attention branches to acquire features at different levels and combines them with global contextual information to achieve effective feature selection and fusion. This helps the algorithm better identify and distinguish objects of different scales and complexities.
[0037] Preferably, the global optimization module utilizes self-attention units to extract global feature maps, and the global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features; Output optimized global features Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in the intermediate features. Indicates the weighting coefficient. This indicates the characteristics after fusion.
[0038] Self-attention units are a crucial component in self-attention generative adversarial networks (GANs). They enable the network to focus more on important parts of the input data, thereby generating higher-quality features. Through the self-attention mechanism, the network learns the inherent structure and relationships of the input data, allowing it to better understand and simulate data distributions, resulting in more realistic and diverse samples in generation tasks. Furthermore, the discriminator can more accurately enforce complex geometric constraints on the global image structure. This directly benefits the global optimization module in this invention, which needs to consider the overall spatial geometry to optimize features.
[0039] Based on the same concept, the present invention provides an electronic device, comprising: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the point cloud spatial location optimization method based on detail-tuning networks as described above.
[0040] Based on the same concept, the present invention also provides a readable storage medium storing a processing program, which, when executed by a processor, implements the point cloud spatial location optimization method based on detail fine-tuning network described above.
[0041] If the point cloud spatial location optimization method based on fine-tuning networks is implemented as program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in software. This computer software is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0042] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.
[0043] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.
Claims
1. A point cloud spatial location optimization method based on detail-fine-tuning networks, characterized in that, Includes the following steps: Fine-tuning the network from a coarse-dense point cloud ensemble and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; A collection of coarse dense point clouds and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
2. The point cloud spatial location optimization method based on detailed fine-tuning networks according to claim 1, characterized in that, The coarse dense point cloud set is optimized using local and global optimization modules. Spatial location optimization of point clouds further includes: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. : Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
3. The point cloud spatial location optimization method based on detail-tuning networks according to claim 2, characterized in that, The multi-scale channel attention unit uses two attention branches to acquire features at different levels and combines them with global context information to achieve effective selection and fusion of features.
4. The point cloud spatial location optimization method based on detail-tuning networks according to claim 1, characterized in that, Utilizing self-attention units to extract global feature maps and achieve global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features.
5. The point cloud spatial location optimization method based on detail-tuning networks according to claim 4, characterized in that, Output optimized global features Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in an intermediate feature. Indicates the weighting coefficient. This indicates the characteristics after fusion.
6. A point cloud spatial location optimization system based on detail-fine-tuning networks, characterized in that, include: The local optimization module is used to fine-tune the network to achieve a coarse-to-dense point cloud ensemble. and its corresponding features As input, the coarse dense point cloud set is optimized using local and global optimization modules. Spatial positioning optimization of point clouds; The global optimization module is used to optimize coarse-grained dense point cloud datasets. and its corresponding features Features after combination As input data, self-attention units are used to extract global feature maps, achieving global spatial location optimization, and outputting optimized global features. .
7. The point cloud spatial location optimization system based on detail-tuning networks according to claim 6, characterized in that, The local optimization module further includes spatial location optimization, which includes: In a coarse dense point cloud collection and its corresponding features The method of K-nearest neighbor grouping is used to search for the K nearest neighbors and their features, and the related neighbor points and features are combined together to form a series of stacked feature spaces. Where r represents the expansion factor, N represents the number of points in the point cloud set, K represents the number of adjacent features, and C1 represents the feature dimension; A series of stacked feature spaces The data is input into a depth map convolutional network to extract local feature structures and morphologies at different scales; We use MLP to concatenate local features at different scales and obtain the weights corresponding to different scales through regression. Features at different scales based on Combined using a weighted summation method; Feature combinations are fused across scales using multi-scale channel attention units to obtain spatially optimized local features. .
8. The point cloud spatial location optimization system based on detailed fine-tuning network according to claim 6, characterized in that, The global optimization module utilizes self-attention units to extract global feature maps, and the global spatial location optimization further includes: Calculating intermediate features via feature space transformation: Among them, W f W g and W h The weight matrix is a learnable matrix. , and Indicates intermediate features; Output optimized global features Further includes: Output feature F P3global Represented as a set (F) P3global1 F P3global2 F P3globalj F P3globalN The calculation method for a single element is as follows: in, , and Represents a single element in an intermediate feature. Indicates the weighting coefficient. This indicates the characteristics after fusion.
9. An electronic device, characterized in that, include: The memory is used to store the processing program; A processor, which, when executing the processing program, implements the point cloud spatial location optimization method based on detail-tuning networks as described in any one of claims 1 to 5.
10. A readable storage medium, characterized in that, The readable storage medium stores a processing program, which, when executed by a processor, implements the point cloud spatial location optimization method based on detail-tuning networks as described in any one of claims 1 to 5.