A three-dimensional point cloud data processing method

Abstract

The present invention proposes a three-dimensional point cloud data processing method, including loading and preprocessing point cloud data, including classifying the three-dimensional point cloud data according to the actual use requirements of the three-dimensional point cloud data, and accessing it using different storage methods and loading strategies according to the classification; extracting features and calculating descriptors of the point cloud data to obtain important feature information of the point cloud data; and registering and reconstructing the point cloud data based on a point cloud registration algorithm. The present invention has significant technical effects such as high speed and efficiency, high precision and accuracy, low resource consumption, and flexible and scalable, and can provide important technical support and guarantee for the development and application of the field of three-dimensional data processing.

Description

Technical Field

[0001] This invention belongs to the field of point cloud data processing technology, and particularly relates to a three-dimensional point cloud data processing method. Background Technology

[0002] With the continuous development and popularization of 3D data acquisition technology, 3D point cloud data, as an important form of 3D data representation, is widely used in fields such as computer vision, robotics, map making, and virtual reality. However, due to the massive and high-dimensional nature of 3D point cloud data, its processing and analysis pose challenges to computing resources and algorithm efficiency.

[0003] Currently, methods for processing 3D point cloud data mainly include data sampling, feature extraction, point cloud registration, and target recognition. However, existing methods often suffer from low efficiency, high computational complexity, and large memory consumption when processing large-scale point cloud data. They also fail to tailor appropriate storage and loading / access methods based on the usage requirements of 3D point cloud data, making it difficult to meet the requirements of real-time performance, efficiency, and practicality.

[0004] Therefore, it is necessary to propose a new and efficient 3D point cloud data processing technology that can effectively address the limitations of existing methods and improve the processing efficiency, accuracy, and practicality of point cloud data. Summary of the Invention

[0005] In response to the problems raised in the background technology, this invention proposes a three-dimensional point cloud data processing method to improve the processing efficiency, accuracy and practicality of point cloud data.

[0006] To achieve this objective, the present invention adopts the following technical solution:

[0007] A method for processing 3D point cloud data, comprising:

[0008] Loading and preprocessing point cloud data includes classifying 3D point cloud data according to the actual usage requirements of the 3D point cloud data, and accessing it using different storage methods and loading strategies according to the classification.

[0009] Feature extraction and descriptor computation are performed on point cloud data to obtain important feature information of the point cloud data;

[0010] Point cloud data is registered and reconstructed based on point cloud registration algorithms.

[0011] Preferably, the 3D point cloud data is classified according to the actual usage requirements of the 3D point cloud data. The classification includes large-scale 3D point cloud data that needs to be used frequently, persistent 3D point cloud data, and 3D point cloud data that is accessed less frequently.

[0012] For large-scale 3D point cloud data that needs to be used frequently, memory-mapped files are used to directly map the point cloud files into the memory address space for storage, and an on-demand loading strategy is used for access.

[0013] For persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, a KD tree data structure is used to store the correspondence between points in the 3D point cloud data, and a block loading strategy is used for access.

[0014] For 3D point cloud data that is accessed less frequently, a memory-mapped file approach is used to directly map the point cloud file into the memory address space for storage, and lazy loading is used for access.

[0015] Preferably, for large-scale 3D point cloud data that is frequently used, memory-mapped files are used to directly map the point cloud files into the memory address space for storage, and an on-demand access strategy is used, including:

[0016] Step A1: Obtain the coordinate information and attribute information contained in each point in the point cloud data, convert the coordinate information and attribute information into binary format, combine the coordinate information and attribute information into a structure, store it in a structured way as a point cloud data file, and determine the number of bytes occupied by the coordinate information and attribute information of each point in the point cloud data file.

[0017] Step A2: For each point cloud data file, the operating system allocates a virtual memory address space to map the point cloud data file from the disk to memory;

[0018] Step A3: Calculate the offset of each point based on the number of bytes occupied by the coordinate and attribute information of each point in the point cloud data file;

[0019] Step A4: Read the point cloud data of the corresponding structure according to the offset. The operating system loads the point cloud data to be read from the disk into memory as needed.

[0020] Preferably, for persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, including:

[0021] Step B1: Construct an octree. Determine the spatial range of the entire point cloud based on the minimum and maximum coordinates of the 3D point cloud data, i.e. the bounding box of the point cloud. Use this as the root node and assign other points to the root node according to their spatial location.

[0022] Step B2: Node segmentation. If the number of points contained in the root node exceeds the threshold, the root node is segmented into eight child nodes according to the spatial range. Each child node corresponds to a cube region. Each cube region contains a part of the space of the root node. The points in the root node are assigned to the corresponding child nodes according to their coordinates.

[0023] Step B3: Recursive segmentation. For each child node, if the number of points in the child node still exceeds the threshold, continue to segment the child node into eight child nodes, and repeat recursively until it cannot be segmented.

[0024] Step B4: When all nodes cannot be divided, these nodes are called leaf nodes and directly store the actual point cloud data.

[0025] Preferably, a KD-tree data structure is used to store the correspondence between points in the 3D point cloud data, including:

[0026] Step B5: For the point cloud data stored in each node of the octree, use the x-axis, y-axis and z-axis of the three-dimensional coordinate dimension of the point cloud data as the cutting axes;

[0027] Step B5: Select one of the x-axis, y-axis, and z-axis in turn as the cutting axis for the current level of the KD tree. On the currently selected cutting axis, select the median as the cutting point of the node at the current level. Assign all point cloud data with coordinates smaller than the cutting point to the left node of the next level, and assign all point cloud data with coordinates larger than the cutting point to the right node of the next level, and so on, until the point cloud data can no longer be segmented.

[0028] Preferably, for persistent 3D point cloud data, a chunked loading strategy is used for access, including:

[0029] The point cloud data of each node in the octree or KD tree is stored independently on the disk as a block. When it is necessary to load and access the point cloud data, the corresponding relationship between the points in the 3D point cloud data stored in the KD tree is used to query and find the point cloud data to be loaded and accessed stored in the octree.

[0030] Preferably, feature extraction and descriptor computation are performed on point cloud data to obtain important feature information of the point cloud data, including:

[0031] Obtain the normal vector of all points in the neighborhood of each point in the point cloud data:

[0032] For each point, select a neighborhood point set, and perform PCA principal component analysis on the points in the neighborhood point set to obtain the covariance matrix of each point. The principal eigenvector of the covariance matrix of each point is the normal vector of each point.

[0033] For all points in the neighborhood, the difference in normal vectors between point pairs is obtained to form a histogram of normal vector differences. This histogram is the PFH feature used to describe the local geometry of the point cloud data.

[0034] Obtain the SIFT descriptor for each normal vector.

[0035] Preferably, point cloud data is registered and reconstructed based on a point cloud registration algorithm, including:

[0036] Find matching point pairs between the source point cloud and the target point cloud;

[0037] Based on the found corresponding point pairs, calculate the rotation matrix and translation vector of the point pairs, and transform the source point cloud to the target position according to the rotation matrix and translation vector so that it overlaps with the target point cloud. Attached Figure Description

[0038] Figure 1 This is a flowchart of the three-dimensional point cloud data processing method of the present invention. Detailed Implementation

[0039] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0040] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or end that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or ends.

[0042] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0043] This application proposes a 3D point cloud data processing method, which can be widely applied in the following fields: Intelligent Manufacturing and Industrial Automation: In the industrial field, this technology can be used for workpiece inspection, quality control, and process optimization. By processing large-scale 3D point cloud data, it enables the detection and analysis of workpiece surface defects and dimensional accuracy issues, improving production efficiency and product quality. Intelligent Transportation and Smart Cities: In intelligent transportation systems and smart city construction, this technology can be used for road monitoring, traffic flow statistics, and road construction planning. By processing collected 3D point cloud data of roads and urban scenes, it enables rapid acquisition and analysis of information such as traffic conditions and road conditions, providing data support for traffic management and urban planning. Virtual Reality and Augmented Reality: In virtual reality and augmented reality applications, this technology can be used for scene reconstruction, environmental simulation, and virtual navigation. By processing 3D point cloud data collected from the real world, it enables high-precision modeling and simulation of scenes, providing users with immersive virtual experiences and augmented reality applications. Mapmaking and Geographic Information Systems (GIS): A key concept in mapmaking and GIS is 3D modeling, one method of which is point cloud data modeling. This technology can be used to represent real-world objects in geographic space, such as terrain modeling, geomorphological analysis, vegetation recording, and urban planning. By processing large-scale point cloud data acquired through LiDAR, photogrammetry, and UAV aerial photography, it enables detailed descriptions of terrain and geomorphology and the generation of 3D maps, providing data support for the construction and application of GIS and offering users a more realistic and intuitive display of geographic information. Medical Imaging and Biological Sciences: In medical image analysis and biological science research, this technology can be used for organ reconstruction, lesion identification, and biological structure analysis. By processing 3D point cloud data from medical images or biological samples, it enables precise reconstruction and quantitative analysis of organ structures and biological tissues, providing data support for medical diagnosis and biological science research.

[0044] Specifically, such as Figure 1 As shown, this method includes:

[0045] Loading and preprocessing point cloud data includes classifying 3D point cloud data according to the actual usage requirements of the 3D point cloud data, and accessing it using different storage methods and loading strategies according to the classification.

[0046] Feature extraction and descriptor computation are performed on point cloud data to obtain important feature information of the point cloud data;

[0047] Point cloud data is registered and reconstructed based on point cloud registration algorithms.

[0048] For large-scale point cloud data, this application achieves rapid loading of point cloud data through effective data structure design and memory management strategies, develops efficient feature extraction and descriptor calculation methods for feature representation and description of point cloud data, and proposes efficient point cloud registration and reconstruction algorithms for accurate alignment and stitching of point cloud data collected from different perspectives or at different times.

[0049] Preferably, the 3D point cloud data is classified according to the actual usage requirements of the 3D point cloud data. The classification includes large-scale 3D point cloud data that needs to be used frequently, persistent 3D point cloud data, and 3D point cloud data that is accessed less frequently.

[0050] For large-scale 3D point cloud data that needs to be used frequently, memory-mapped files are used to directly map the point cloud files into the memory address space for storage, and an on-demand loading strategy is used for access.

[0051] Specifically, it includes:

[0052] Step A1: Obtain the coordinate information and attribute information contained in each point in the point cloud data, convert the coordinate information and attribute information into binary format, combine the coordinate information and attribute information into a structure, store it in a structured way as a point cloud data file, and determine the number of bytes occupied by the coordinate information and attribute information of each point in the point cloud data file.

[0053] Step A2: For each point cloud data file, the operating system allocates a virtual memory address space to map the point cloud data file from the disk to memory;

[0054] Step A3: Calculate the offset of each point based on the number of bytes occupied by the coordinate and attribute information of each point in the point cloud data file;

[0055] Step A4: Read the point cloud data of the corresponding structure according to the offset. The operating system loads the point cloud data to be read from the disk into memory as needed.

[0056] In this embodiment, for frequently used and large-scale point cloud data, a memory-mapped file approach is used to directly map the file into the memory address space. The mapping process is as follows:

[0057] The data storage format needs to be determined. Point cloud data needs to be converted to binary format, with each point containing coordinate information (such as x, y, z) and possible other attributes (such as intensity, color, etc.). For each point, it is necessary to know how many bytes it occupies in the file. For example, if each coordinate is represented using 4 bytes, then a 3D coordinate point will occupy 12 bytes. This information is crucial for subsequent memory mapping.

[0058] When a point cloud data file is mapped to memory, the operating system allocates a virtual memory address space for the file. This process does not require explicitly loading the data into memory. The operating system will automatically load the data from the disk into memory when it needs to access certain data. In this way, the program can directly manipulate the data in the file as if it were accessing memory.

[0059] Calculating the offset: In a memory-mapped file, each data point has a fixed offset (i.e., its position in the file). Assuming each point in the file occupies 12 bytes, the data for the first point starts from the first byte of the file, the data for the second point starts from the 13th byte, and so on. By calculating the offset of each point, we can locate its position in the file.

[0060] Structured access to point cloud data is crucial. Point cloud data typically has multiple attributes (such as x, y, z coordinates, intensity, color, etc.), which can be combined into a structure. The data for each point can be stored in a structured manner, and during access, the structured data is read based on offsets. For example, if a point contains three-dimensional coordinates (x, y, z) and an intensity value, the starting position of each field can be calculated based on the offsets, allowing the attributes to be read sequentially.

[0061] On-demand loading is one advantage of memory-mapped files, as the operating system loads the file's contents only when needed, rather than loading the entire file into memory at once. Therefore, when only certain parts of the file are accessed, the operating system intelligently loads those parts into memory, rather than consuming large amounts of memory. This makes memory-mapped files particularly suitable for processing large datasets.

[0062] In terms of optimization and memory management, when using memory mapping, the operating system automatically manages memory and loads files. If a program only accesses a portion of a file's content, the operating system will only load that portion, avoiding unnecessary memory consumption. Furthermore, the operating system typically performs virtual memory management and supports memory alignment, features that contribute to improved access efficiency.

[0063] For 3D point cloud data that is accessed less frequently, a memory-mapped file approach is used to directly map the point cloud file into the memory address space for storage, and lazy loading is used for access.

[0064] Lazy loading aims to postpone the creation of certain objects or the loading of data until they are actually used. This strategy can optimize system performance, reduce memory consumption, and improve responsiveness. Especially when dealing with large-scale data, lazy loading avoids loading too much irrelevant data at once, reducing unnecessary computational and storage pressure. For applications requiring real-time interaction, lazy loading can improve user experience and reduce loading latency. Examples include Geographic Information Systems (GIS), where detailed data for a region is loaded only when the user approaches it; and game engines and virtual reality, where, in large-scale scenes, relevant object or scene data is loaded only when the user enters a specific area or viewpoint.

[0065] It's important to note that on-demand loading only loads the resources required for the current page or operation, reducing unnecessary resource loading and improving user experience. Users don't need to wait for all resources to load before starting to use the page or application, saving bandwidth and server resources as only essential data is transmitted. Lazy loading, on the other hand, loads only necessary resources or data during the initial load, loading other resources or data only when needed. This is suitable for long pages or pages containing a large number of resources, reducing initial load time and server load. Lazy loading optimizes the user experience because users can continue using the page or application while resources are loading.

[0066] Furthermore, a memory pool can be created to pre-allocate memory space to cope with frequently changing point cloud data.

[0067] For persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, a KD tree data structure is used to store the correspondence between points in the 3D point cloud data, and a block loading strategy is used for access.

[0068] Preferably, for persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, including:

[0069] Step B1: Construct an octree. Determine the spatial range of the entire point cloud based on the minimum and maximum coordinates of the 3D point cloud data, i.e. the bounding box of the point cloud. Use this as the root node and assign other points to the root node according to their spatial location.

[0070] Step B2: Node segmentation. If the number of points contained in the root node exceeds the threshold, the root node is segmented into eight child nodes according to the spatial range. Each child node corresponds to a cube region. Each cube region contains a part of the space of the root node. The points in the root node are assigned to the corresponding child nodes according to their coordinates.

[0071] Step B3: Recursive segmentation. For each child node, if the number of points in the child node still exceeds the threshold, continue to segment the child node into eight child nodes, and repeat recursively until it cannot be segmented.

[0072] Step B4: When all nodes cannot be divided, these nodes are called leaf nodes and directly store the actual point cloud data.

[0073] In this embodiment, each node of the octree represents a cube (or "region") in three-dimensional space. The node recursively divides the space into eight subspaces, each of which is also a cube. If the number of points in a node's subspace exceeds a preset threshold, the node will further split until one of the following conditions is met:

[0074] 1. The number of points within a node does not exceed the set maximum number of points;

[0075] 2. The space reaches the smallest divisible size;

[0076] Each octree node contains the following information:

[0077] 1. Spatial range: The cubic spatial range corresponding to each node defines which points the node can contain.

[0078] 2. Point cloud data: The points stored in this node. If the node is a leaf node, it contains actual points; if it is a non-leaf node, it does not contain points, but only pointers to its child nodes.

[0079] 3. Child node pointers: Point to the 8 child nodes of the current node (if the node is a non-leaf node).

[0080] The steps for allocating point cloud data to octree nodes are:

[0081] A1: Initial Tree Construction: First, the spatial extent of the entire point cloud is determined based on the minimum and maximum coordinates of the point cloud data (i.e., the bounding box of the point cloud). This is the root node of the octree. Then, the other points are sequentially assigned to this root node according to their spatial location.

[0082] A2: Node Segmentation: If the number of points in the root node exceeds a threshold, the root node will be segmented into 8 child nodes based on the spatial extent. Each child node corresponds to a cube region, which includes a portion of the original root node's space. Points from the original root node are assigned to the corresponding child nodes based on their positions. Each point is identified by its coordinates to determine which child node it belongs to.

[0083] A3: Recursive partitioning: For each child node, if the number of points in the child node still exceeds the threshold, continue partitioning it into 8 child nodes. This process is repeated recursively until the number of points in all leaf nodes does not exceed the threshold, or the size of the spatial partition is too small to continue partitioning.

[0084] A4: Leaf nodes store point clouds: Ultimately, when nodes can no longer be segmented, these nodes become leaf nodes and directly store the actual point cloud data. The number of points in a leaf node is usually fixed (less than or equal to a threshold).

[0085] Furthermore, when using octrees to store point cloud data, downsampling and region growing can be used to reduce the amount of data. Retaining only key points can significantly reduce computational overhead. For example:

[0086] a. Downsampling Method: Downsampling using an octree can compress dense point cloud data into fewer points while preserving important geometric features as much as possible. The downsampling process reduces the number of points by dividing the space into larger volume units, thereby improving processing efficiency. For each small cube (i.e., a leaf node of the octree), if the number of points in that region exceeds a preset threshold, the number of points is reduced using a downsampling method. Specific downsampling methods include:

[0087] 1. Voxel Grid: Replace the points in each leaf node with representative points of that region (such as the center point, average point, or other statistics).

[0088] 2. Randomly select a point: Randomly select a point from each leaf node as the representative point.

[0089] 3. Selection of the densest point: Select the point closest to the center point of the leaf node.

[0090] By using the downsampled octree structure, we can ensure that each leaf node retains only one representative point, which can significantly reduce the number of point clouds while still preserving the main features in space.

[0091] b. Region Growing: Region growing is a neighborhood-based segmentation method used to extract connected regions or features from a point cloud by starting with a seed point and progressively growing and merging neighboring points. Region growing combined with octrees is typically used to extract meaningful parts from dense regions while preserving important key points. Starting with a seed point, neighboring points are progressively included in the growing region by checking if they meet certain specific conditions (such as distance, normal vector consistency, curvature similarity, etc.). Octree spatial segmentation processes only adjacent leaf nodes at a time, reducing unnecessary computation and improving the efficiency of region growing.

[0092] Preferably, a KD-tree data structure is used to store the correspondence between points in the 3D point cloud data, including:

[0093] Step B5: For the point cloud data stored in each node of the octree, use the x-axis, y-axis and z-axis of the three-dimensional coordinate dimension of the point cloud data as the cutting axes;

[0094] Step B5: Select one of the x-axis, y-axis, and z-axis in turn as the cutting axis for the current level of the KD tree. On the currently selected cutting axis, select the median as the cutting point of the node at the current level. Assign all point cloud data with coordinates smaller than the cutting point to the left node of the next level, and assign all point cloud data with coordinates larger than the cutting point to the right node of the next level, and so on, until the point cloud data can no longer be segmented.

[0095] In this embodiment, the 3D point cloud data of a node in an octree is used as an example for explanation: {(2,3,5), (5,4,1), (9,6,7), (4,7,8), (8,1,5), (7,2,6)}. First, the x-axis is selected as the cutting axis, and the values ​​on the x-axis are sorted according to their size. The median is 7, so (7,2,6) is taken as the root node. The x-axis values ​​less than 7 are assigned to the left nodes of the second level, i.e., (2,3,5), (4,7,8), (5,4,1), and the x-axis values ​​greater than 7 are assigned to the right nodes of the second level, (8,1,5), (9,6,7). In the second level... In the left and right nodes of the first level; then select the y-axis as the cutting axis, sort the y-axis values ​​in the left node according to size, the median of the left node is 4, so (5,4,1) is the left node of the second level, (2,3,5) is assigned to the left node of the third level of the node where (5,4,1) is located, and (4,7,8) is assigned to the right node of the third level of the node where (5,4,1) is located; in the right node, sort the y-axis values ​​in size, the median of the left node is 6, so (9,6,7) is the right node of the second level, and (8,1,5) is assigned to the left node of the third level of the node where (9,6,7) is located.

[0096] Preferably, for persistent 3D point cloud data, a chunked loading strategy is used for access, including:

[0097] The point cloud data of each node in the octree or KD tree is stored independently on the disk as a block. When it is necessary to load and access the point cloud data, the corresponding relationship between the points in the 3D point cloud data stored in the KD tree is used to query and find the point cloud data to be loaded and accessed stored in the octree.

[0098] In this embodiment, persistent point cloud refers to storing point cloud data on disk or other long-term storage media so that it can be persistently saved, accessed, and processed in subsequent use. Chunked Loading strategy involves dividing point cloud data into smaller, uniform blocks, such as octrees or KD-trees where each node can be considered a block, with each block stored independently on disk. When loading point cloud data, only blocks relevant to the current view or operation are loaded each time; then, the blocks to be loaded are dynamically selected based on different needs (such as view range, processing area, etc.). This approach not only optimizes memory usage and improves loading efficiency but also reduces I / O overhead and supports multi-threaded or parallel loading, further enhancing performance. This strategy is suitable for browsing and real-time processing of large-scale point cloud data, such as 3D city models and LiDAR data visualization.

[0099] Furthermore, we use octrees to store point cloud data and combine this with an algorithm that uses KD trees to store the correspondence between points during queries to create a foundation for efficient point cloud processing. For example, we can use the KD number in an octree node to query the point closest to a given point in the KD number, providing the following query method:

[0100] First, starting from the root node, compare the coordinates of the query point with the current node to determine whether to search the left or right subtree.

[0101] Next, recursively search for the nearest neighbor: recursively enter the left or right subtree until a leaf node is found or the depth of the tree is reached. During the recursion, the nearest neighbor up to the current point is also recorded and updated in subsequent traversals.

[0102] Finally, pruning is performed during backtracking: if the subtree of the current node has already been searched, backtracking checks if a closer point might exist. Specifically, it checks the distance between the current node's splitting plane (cutting axis) and the query point to determine if a closer point could be found in another subtree. If so, the other subtree is recursively searched.

[0103] Preferably, feature extraction and descriptor computation are performed on point cloud data to obtain important feature information of the point cloud data, including:

[0104] Obtain the normal vector of all points in the neighborhood of each point in the point cloud data:

[0105] For each point, select a neighborhood point set, and perform PCA principal component analysis on the points in the neighborhood point set to obtain the covariance matrix of each point. The principal eigenvector of the covariance matrix of each point is the normal vector of each point.

[0106] For all points in the neighborhood, the difference in normal vectors between point pairs is obtained to form a histogram of normal vector differences. This histogram is the PFH feature used to describe the local geometry of the point cloud data.

[0107] Obtain the SIFT descriptor for each normal vector.

[0108] In this embodiment, we develop efficient feature extraction and descriptor calculation methods for feature representation and description of point cloud data. For example, through operations such as local feature point extraction, normal calculation, and curvature estimation, we obtain important feature information of the point cloud data. These features include the spatial location, color information, normal, curvature, density, geometric features, and topological structure of the point cloud. Among them, spatial location and color information are inherent data information of the point cloud. Normal and curvature can be estimated through PCA of neighboring points. Density can be calculated by performing neighborhood search using a KD-tree (such as the KD-tree query method mentioned above) to count the number of points within a certain radius around each point.

[0109] After obtaining the feature data of the point cloud, corresponding descriptors can be calculated for subsequent tasks such as point cloud matching, registration, and target recognition. In this embodiment, the SIFT descriptor is used. SIFT generates key points by constructing a difference-of-Gaussian pyramid, and generates gradient histograms of multiple directional sub-regions within the local region of the key points. These gradient histograms are combined into a feature vector SIFT descriptor.

[0110] In the feature extraction stage:

[0111] We use Feature Extraction Method (PFH): PFH is a feature extraction method for describing the local geometry of point clouds, capable of capturing the relative geometric structure of adjacent points in the point cloud. The extraction steps are as follows:

[0112] a. For each point, calculate the normal vector of all points in its neighborhood;

[0113] b. For points in the neighborhood, calculate the difference in normal vectors between point pairs to form a histogram of normal vector differences;

[0114] c. This histogram is the PFH feature, used to describe local geometry;

[0115] For calculating the normal vector: The normal vector is a key feature describing the local geometry of the point cloud surface, and is usually obtained by fitting a plane to the neighborhood point cloud. This embodiment uses Principal Component Analysis (PCA) to fit the local neighborhood of each point and calculate the normal vector. PCA can determine the normal vector of a local region of the point cloud by finding the directions of maximum and minimum change. The calculation steps are as follows:

[0116] For each point, select a neighborhood set of points (usually the k nearest neighbors within the radius or points within a specified radius), perform PCA on the neighborhood points to obtain its covariance matrix, and the normal vector is the principal eigenvector of the covariance matrix. The direction of the normal vector is usually determined by comparison with the global coordinate system.

[0117] In another embodiment, we provide a method for calculating curvature, which reflects the degree of bending of a local point cloud surface. On a plane, curvature is typically calculated using the second derivative. Curvature of local regions can be calculated through point cloud fitting (e.g., plane fitting, quadratic surface fitting). Common curvature calculation methods include:

[0118] Gaussian Curvature: For planes and curved surfaces, the Gaussian curvature at each point is calculated to describe the degree of local curvature of the surface.

[0119] Principal curvature: The principal curvature is calculated by fitting the local neighborhood of the point cloud using PCA.

[0120] During the descriptor computation phase:

[0121] We employ the SIFT descriptor. The core of SIFT is the construction of the scale space and the search for local extrema within that space. The scale space is constructed by applying multi-scale blurring to the original image.

[0122] Obtain images corresponding to point cloud data at different scales:

[0123] L(x,y,σ)=G(x,y,σ)*I(x,y);

[0124]

[0125] Where L(x, y, σ) represents the image corresponding to the point cloud data at scale σ, G(x, y, σ) represents the Gaussian kernel image, I(x, y) represents the original image corresponding to the point cloud data, and x and y represent the corresponding coordinate values ​​respectively;

[0126] By performing the Difference of Gaussian (DoG) operation on images at different scales, a scale space is constructed: D(x, y, σ) = G(x, y, kσ) * I(x, y) - G(x, y, σ) * I(x, y);

[0127] Where D(x, y, σ) represents the scale space, and G(x, y, kσ) represents the Gaussian kernel image of k neighborhoods at different scales σ;

[0128] Keypoints are identified by detecting local extrema in scale space, finding local extrema that differ from their neighborhood at each scale. SIFT assigns a principal orientation to each keypoint by calculating the gradient directions of its surrounding neighborhood. The orientation of a keypoint can be calculated using the following formula:

[0129]

[0130] in, It is the gradient at point r. R is the magnitude of the gradient, and R is the neighborhood around the keypoint.

[0131] Finally, SIFT descriptors are generated. The descriptor for each keypoint is calculated based on the gradient information of its neighborhood. The descriptor is composed of gradient histograms of the surrounding area, usually a histogram with 16 directions, each histogram having 8 directions: H = [hist(0°), hist(45°), ..., hist(315°)]. The descriptor quantizes and standardizes the directions of these gradients to remove the influence of illumination changes.

[0132] Preferably, point cloud data is registered and reconstructed based on a point cloud registration algorithm, including:

[0133] Find matching point pairs between the source point cloud and the target point cloud;

[0134] Based on the found corresponding point pairs, calculate the rotation matrix and translation vector of the point pairs, and transform the source point cloud to the target position according to the rotation matrix and translation vector so that it overlaps with the target point cloud.

[0135] In this embodiment, we propose an efficient point cloud registration and reconstruction algorithm to accurately align and stitch point cloud data acquired from different viewpoints or at different times. The point cloud registration algorithm first finds the nearest point in the target point cloud for each point. This finding is accomplished using a nearest neighbor search algorithm based on data structures such as KD-trees and octrees. For each point p in the source point cloud... i =(x i y i , z i By using nearest neighbor search, find the point q in the point cloud that is closest to the target. j =(x j y j , z j Record the distance between the pairs of points.

[0136]

[0137] This can be accelerated using a KD-tree, which can efficiently find the nearest point;

[0138] Then, the rigidity changes (translation and rotation) between the two point clouds are calculated. The translation vector t can be obtained from the centroids of the point clouds. Assume the centroids of the source and target point clouds are respectively:

[0139] Source Cloud:

[0140] Target point cloud:

[0141] Then the translation vector t is: t = C q -C p ;

[0142] The rotation matrix R can be calculated using the least squares method. We use the SVD (Singular Value Decomposition) method, in which we first construct a matrix H:

[0143]

[0144] Then, singular value decomposition is performed on matrix H to obtain:

[0145] H=U∑V T ;

[0146] The rotation matrix R is then: VU T ;

[0147] Here, U and V are the singular value decomposition results of H, and R is guaranteed to be an orthogonal matrix. Then, apply the transformation to the source point cloud. Repeat the above steps until convergence or the maximum number of iterations is reached.

[0148] By optimizing the registration strategy and matching algorithm, accurate matching and transformation estimation between point cloud data are achieved, thereby enabling the reconstruction of 3D scenes and model stitching.

[0149] In summary, this application has the following advantages:

[0150] 1. High speed and efficiency: Through parallel computing and optimized algorithms, high-speed processing of large-scale, high-dimensional point cloud data is achieved. Compared with traditional methods, the processing speed is significantly improved, enabling the processing of large-scale point cloud data in a short time and meeting real-time requirements.

[0151] 2. High Precision and Accuracy: Employing optimized feature extraction and registration algorithms, it achieves high-precision processing and reconstruction of point cloud data. Precise feature matching and registration ensure the accuracy and reliability of the processing results, making it suitable for applications with high precision requirements.

[0152] 3. Low resource consumption: An efficient data loading and storage strategy was designed, optimizing algorithm computation overhead and reducing system resource consumption. While ensuring processing efficiency, it reduces memory usage and computing resource requirements, making it suitable for resource-constrained environments.

[0153] 4. Flexible and Scalable: A flexible and scalable system architecture and algorithm framework have been built, supporting the processing of point cloud data of different types and scales. It can adapt to the needs of different application scenarios, supports system expansion and customization, and has strong versatility and applicability.

[0154] 5. Wide Applications: This technology can be widely applied in fields such as computer vision, robotics, map making, and virtual reality. It can provide efficient and accurate technical support for 3D data processing and applications, promoting the development and application of related industries.

[0155] The technical principles of the present invention have been described above with reference to specific embodiments. These descriptions are merely for explaining the principles of the invention and should not be construed as limiting the scope of protection of the invention in any way. Based on this explanation, those skilled in the art can readily conceive of other specific embodiments of the invention without inventive effort, and these embodiments will all fall within the scope of protection of the present invention.

Claims

1. A three-dimensional point cloud data processing method, characterized in that: Load and preprocess point cloud data, including classifying 3D point cloud data according to actual usage requirements, and accessing it using different storage methods and loading strategies based on the classification; Perform feature extraction and descriptor calculation on point cloud data to obtain important feature information of point cloud data; The point cloud data is registered and reconstructed based on the point cloud registration algorithm.

2. A three-dimensional point cloud data processing method according to claim 1, characterized in that: Classify the 3D point cloud data according to the actual usage requirements of the 3D point cloud data, including large-scale 3D point cloud data that needs to be used frequently, persistent 3D point cloud data, and seldom-accessed 3D point cloud data; For large-scale 3D point cloud data that needs to be used frequently, the point cloud file is directly mapped to the memory address space for storage using a memory-mapped file method, and accessed using an on-demand loading strategy; For persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, a KD tree data structure is used to store the correspondence between points in the 3D point cloud data, and a block loading strategy is used for access; For seldom-accessed 3D point cloud data, a memory-mapped file is used to directly map the point cloud file to the memory address space for storage, and lazy loading is used for access.

3. A three-dimensional point cloud data processing method according to claim 2, characterized in that: For large-scale 3D point cloud data that needs to be used frequently, the point cloud file is directly mapped to the memory address space for storage using a memory-mapped file, and accessed using an on-demand add strategy, including: Step A1: obtaining coordinate information and attribute information contained in each point in the point cloud data, converting the coordinate information and attribute information into a binary format, combining the coordinate information and attribute information into a structure, storing them in a structured manner into a point cloud data file, and determining the number of bytes occupied by the coordinate information and attribute information of each point in the point cloud data file; Step A2: For each point cloud data file, the operating system allocates a virtual memory address space to map the point cloud data file from the disk to the memory; Step A3: Calculate the offset of each point according to the number of bytes occupied by the coordinate information and attribute information of each point in the point cloud data file; Step A4: Read the point cloud data of the corresponding structure according to the offset, and the operating system loads the point cloud data to be read from the disk to the memory as needed.

4. A three-dimensional point cloud data processing method according to claim 2, characterized in that: For persistent 3D point cloud data, an octree data structure is used to store the 3D point cloud data, including: Step B1: Construct an octree, determine the spatial range of the entire point cloud according to the minimum and maximum coordinates of the 3D point cloud data, that is, the bounding box of the point cloud, and use this as the root node. Allocate other points to the root node according to their spatial positions. Step B2: Node segmentation. If the number of points contained in the root node exceeds the threshold, the root node is segmented into eight child nodes according to the spatial range. Each child node corresponds to a cubic area. Each cubic area contains a part of the space of the root node. The points in the root node are assigned to the corresponding child nodes according to the coordinates. Step B3: Recursive segmentation: for each child node, if the number of points in the child node still exceeds the threshold, continue to segment the child node into eight child nodes, and recursively perform until it cannot be segmented; Step B4: When all nodes cannot be split, these nodes are leaf nodes and directly store the actual point cloud data.

5. A three-dimensional point cloud data processing method according to claim 4, characterized in that: The KD tree data structure is used to store the correspondence between points in the 3D point cloud data, including: Step B5: For the point cloud data stored in each node in the octree, the x-axis, y-axis and z-axis of the three-dimensional coordinate dimensions of the point cloud data are used as cutting axes; Step B5: Select one of the x-axis, y-axis and z-axis in turn as the cutting axis of the current level of the KD tree. On the currently selected cutting axis, select a median as the cutting point of the node at the current level. Assign all point cloud data smaller than the cutting point coordinates to the left node of the next level, and assign all point cloud data larger than the cutting point coordinates to the right node of the next level, and so on until the point cloud data cannot be split.

6. A three-dimensional point cloud data processing method according to claim 2, characterized in that: For persistent 3D point cloud data, use the block loading strategy to access, including: The point cloud data of each node in the octree or KD tree is stored independently on the disk as a block. When the point cloud data needs to be loaded and accessed, the point cloud data to be loaded and accessed stored in the octree is queried based on the correspondence between the points of the three-dimensional point cloud data stored in the KD tree.

7. A three-dimensional point cloud data processing method according to claim 1, characterized in that: Feature extraction and descriptor calculation are performed on point cloud data to obtain important feature information of point cloud data, including: Get the normal vectors of all points in the neighborhood of each point in the point cloud data: For each point, a neighborhood point set is selected, and the points in the neighborhood point set are subjected to PCA principal component analysis to obtain the covariance matrix of each point. The principal eigenvector of the covariance matrix of each point is the normal vector of each point. For all points in the neighborhood, obtain the normal vector differences between point pairs to form a histogram of normal vector differences. This histogram is the PFH feature used to describe the local geometric shape of the point cloud data. Get the SIFT descriptor for each normal vector.

8. A three-dimensional point cloud data processing method according to claim 1, characterized in that: The point cloud data is registered and reconstructed based on the point cloud registration algorithm, including: Find matching point pairs between the source point cloud and the target point cloud; According to the corresponding point pairs found, the rotation matrix and translation vector of the point pairs are calculated, and the source point cloud is transformed to the target position according to the rotation matrix and translation vector so that it overlaps with the target point cloud.

Images