3D point cloud data enhancement method based on viewpoint shielding simulation

Abstract

The invention discloses a 3D point cloud data enhancement method based on viewpoint occlusion simulation, and the method comprises the steps: selecting a virtual viewpoint direction based on an original point cloud, carrying out the depth projection and sorting of the original point cloud based on the virtual viewpoint direction, simulating the occlusion according to the depth projection and sorting result, and selecting a visible point set; acquiring a supplementary point set based on the visible point set; and combining the visible point set and the supplementary point set to obtain an enhanced point cloud. According to the method provided by the invention, the point cloud with structural and continuous missing is generated by simulating the viewpoint of a single sensor and simulating real physical shielding, which is highly consistent with the shielding condition in the real world and is far superior to the traditional random point discarding method.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud data augmentation technology, and in particular to a 3D point cloud data augmentation method based on viewpoint occlusion simulation. Background Technology

[0002] 3D point cloud classification and analysis, as core technologies in computer vision and autonomous driving, are crucial for machines to understand three-dimensional objects and achieve environmental perception. With the rapid development of deep learning technology, especially deep neural networks (DNNs), models for processing point cloud data are constantly breaking through performance bottlenecks on various benchmark datasets such as ShapeNet and ModelNet40, continuously setting new records for classification accuracy and segmentation precision, demonstrating powerful 3D data parsing capabilities.

[0003] However, the superior performance of these models is highly dependent on clean and complete training datasets. Once faced with complex real-world scenarios, their performance often drops drastically. In practical applications, when 3D sensors (such as LiDAR and depth cameras) collect data, structural and continuous partial occlusion of point clouds is prevalent due to limitations in physical principles and environmental conditions. This is mainly caused by two factors: First, self-occlusion, where the geometry of a 3D object causes partial occlusion. For example, the chassis of a car can obscure the point cloud on the ground below, and a human arm can obscure the point cloud on the side of the torso. Second, sensor field of view limitations; sensors can only capture surface information of objects within their field of view. Areas outside the field of view (e.g., the back of an object, or areas obscured by other objects) cannot be detected, resulting in missing data.

[0004] This kind of non-random, spatially correlated data gap is common in real-world scenarios such as autonomous driving and robot navigation. It directly leads to a significant deviation between the features learned by the model during the training phase and the data distribution in actual applications, seriously affecting the reliability of the model.

[0005] To alleviate this problem, existing research has proposed various point cloud data augmentation techniques, among which Random Point Dropout is one of the most widely used methods. This method increases the diversity of training data by independently and uniformly removing a portion of points from the point cloud, attempting to improve the model's generalization ability. However, fundamentally, this approach has a basic flaw:

[0006] It can only simulate random sparsity and cannot reproduce the structural and continuous occlusion caused by a single viewpoint in real-world scenes. The discarded points are discretely distributed in space and cannot reflect physical realities such as the invisibility of the back of an object or the complete occlusion of local areas. Therefore, models trained on this type of non-physical augmentation data still struggle to adapt to occlusion scenarios in the real world: for example, in autonomous driving, pedestrian point clouds captured by LiDAR may lack leg information due to occlusion, making accurate model identification impossible; in robot grasping tasks, partial occlusion of objects may lead to misclassification of object categories or poses. This disconnect between training and application scenarios severely limits the practical application of deep learning point cloud models in safety-critical fields and becomes a core bottleneck restricting the improvement of their robustness.

[0007] Currently, some related technologies are being researched to enhance point cloud data under specific environments. For example, patent document CN117574799B discloses a method and system for enhancing lidar point cloud data in severe weather scenarios. This application focuses on severe weather scenarios, generating a simulation dataset and optimizing point cloud intensity characteristics by constructing a physical model of rain splash and a lidar model, thus solving the problem of poor realism of point cloud intensity in rainy weather. However, this technology only addresses weather interference and does not address structural data loss caused by viewpoint occlusion. Another example is patent CN117872316A, which discloses a method for enhancing lidar point cloud data in dusty weather using multi-scale physical simulation. This application constructs a three-dimensional distribution model of dust particles based on the Monte Carlo algorithm to simulate the impact of dust on lidar echoes, achieving point cloud enhancement under dusty weather. Its core is to solve the interference of environmental media on sensor signals, which is completely different from the physical mechanisms of object self-occlusion or viewpoint occlusion.

[0008] In summary, existing technologies have not provided an effective solution to the core problem of structural occlusion caused by a single viewpoint. Therefore, there is a need for a point cloud data augmentation method that can simulate the real physical observation process and generate point cloud data with continuous occlusion characteristics, in order to narrow the gap between model training and real-world applications and improve the robustness and reliability of 3D point cloud analysis models in occluded scenarios. Summary of the Invention

[0009] To address the aforementioned problems, the present invention aims to provide a 3D point cloud data enhancement method based on viewpoint occlusion simulation, which solves the structural occlusion problem caused by a single viewpoint and generates point cloud data with continuous occlusion characteristics by simulating the real physical observation process.

[0010] First aspect: A 3D point cloud data augmentation method based on viewpoint occlusion simulation, including the following steps:

[0011] S1. Based on the original point cloud, select the virtual viewpoint direction;

[0012] S2. Based on the virtual viewpoint direction, perform depth projection and sorting of the original point cloud;

[0013] S3. Based on the depth projection and sorting results, simulate occlusion and select the set of visible points;

[0014] S4. Based on the visible point set, obtain the supplementary point set;

[0015] S5. Merge the visible point set and the supplementary point set to obtain the enhanced point cloud.

[0016] In one embodiment of the present invention, step S1 includes the following steps:

[0017] S11, from the three-dimensional standard Gaussian distribution A random direction vector z is sampled from the sample.

[0018] S12. Normalize the direction vector z and calculate the L2 norm of the direction vector z;

[0019] S13. Based on the L2 norm, calculate the unit direction vector v of the direction vector z as the virtual viewpoint direction.

[0020] In one embodiment of the present invention, the unit direction vector v is expressed by the formula:

[0021]

[0022]

[0023] in, It is a direction vector. Let z be the L2 norm of the direction vector z.

[0024] In one embodiment of the present invention, step S2 includes the following steps:

[0025] S21. Calculate the original point cloud The depth set is obtained by taking the dot product with the unit direction vector v. The formula is expressed as:

[0026]

[0027] in, , It is the position direction vector. for The depth value;

[0028] S22, For depth sets Sort the points to obtain the original point cloud P sorted point list.

[0029] In one embodiment of the present invention, step S3 includes the following steps:

[0030] S31. Determine the retention ratio ;

[0031] S32, Based on the retention ratio Calculate the number of points in the set of visible points that need to be retained. The formula is:

[0032]

[0033] in, Original point cloud The number of points, This is a floor function;

[0034] S33. Select the sorting point list Construct a set of visible points from these points. .

[0035] In one embodiment of the present invention, during each enhancement operation, from the value range uniform distribution Random sampling determines the retention ratio .

[0036] In one embodiment of the present invention, S4 includes the step of:

[0037] S41. Calculate the supplementary point set The number of points is expressed by the formula:

[0038]

[0039] in, Original point cloud The number of points, For the set of visible points The number of points, To supplement the point set The number of points;

[0040] S42, Based on Visible Point Set Interpolation method is used to generate These supplementary points constitute the supplementary point set. .

[0041] In one embodiment of the present invention, the supplementary point set is obtained based on interpolation in step S42. The steps include:

[0042] S42a, From the set of visible points A seed point is randomly and uniformly selected from the data.

[0043] S42b. Use the nearest neighbor search algorithm (KD-Tree) to find the seed point in the set of visible points. Find the k nearest neighbors in the array;

[0044] S42c: Calculate a weighted average of the seed point and its k neighbors to generate a new point;

[0045] S42d, repeat S42a~S42c together Next, a supplementary point set is formed. .

[0046] In one embodiment of the present invention, the enhanced point cloud in S5 is expressed by the formula:

[0047]

[0048] in, To enhance point clouds, For the set of visible points, To supplement the point set, The number of points is , The number of points is ,and .

[0049] In one embodiment of the present invention, the enhanced point cloud Perform a random shuffle to disrupt the order of the elements.

[0050] Second aspect: An electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of the method provided in the first aspect.

[0051] Third aspect: A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in the first aspect.

[0052] The beneficial effects of this invention are:

[0053] 1. The method proposed in this invention simulates the viewpoint of a single sensor and simulates real physical occlusion to generate a point cloud with structured and continuous missing data. This is highly consistent with the occlusion situation in the real world and is far superior to the traditional random point discarding method.

[0054] 2. Deep learning models trained on datasets enhanced by the method of this invention can better cope with partially occluded data encountered during testing, significantly improving the model's generalization ability and robustness in real-world application scenarios.

[0055] 3. The method of this invention is a model-agnostic data preprocessing / augmentation strategy that can be seamlessly integrated into the training process of any point cloud-based classification model, and has a wide range of applications.

[0056] 4. The method of this invention ensures that the enhanced point cloud has the same number of points as the original point cloud through a unique point count preservation mechanism, without the need to modify the subsequent network structure or training process, which facilitates batch training in existing deep learning frameworks.

[0057] 5. The method of the present invention adjusts the range of the retention ratio. It allows for flexible control over the severity of occlusion to adapt to the needs of different tasks and datasets. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the principle of the 3D point cloud data enhancement method of the present invention.

[0059] Figure 2 This is a schematic diagram illustrating the occlusion simulation principle of the 3D point cloud data enhancement method of the present invention;

[0060] Figure 3 This is a comparison chart showing the effects of the method of the present invention and the prior art (randomly dropping points);

[0061] Figure 4 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0062] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar symbols denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0063] Existing 3D point cloud data augmentation methods (such as randomly dropping points) can improve the generalization ability of models to some extent, but their strategy is to remove points from the point cloud independently and uniformly. This approach cannot realistically simulate the structural and continuous occlusion caused by a single viewpoint. Therefore, models trained on such non-physically realistic augmented data still perform poorly on viewpoint occlusion problems common in the real world. This significantly limits the reliability and robustness of deep learning models in safety-critical fields such as autonomous driving and robot navigation.

[0064] To address the above problems, this invention provides a 3D point cloud data augmentation method based on viewpoint occlusion simulation. Figure 1 This is a flowchart illustrating the principle of the 3D point cloud data enhancement method of the present invention.

[0065] Example 1:

[0066] This embodiment discloses a 3D point cloud data augmentation method based on viewpoint occlusion simulation, including the following steps:

[0067] S1. Based on the original point cloud, select the virtual viewpoint direction.

[0068] The purpose of this step is to determine a random, unbiased observation direction in three-dimensional space to simulate a scenario where the sensor might observe the target object from any angle. To ensure the diversity and fairness of the augmentation process, i.e., to avoid excessive occlusion simulation in certain specific directions, the viewpoint direction must be uniformly distributed across a unit sphere.

[0069] This embodiment employs a computationally efficient and mathematically proven uniform sampling method to achieve this objective. The specific operation is as follows:

[0070] First, define a three-dimensional standard Gaussian distribution, denoted as . The vector with zero mean and a covariance matrix is ​​a third-order identity matrix. A three-dimensional direction vector is randomly sampled from this distribution. The direction of this direction vector is random in space.

[0071] Then, in order to this direction vector Convert it to a standard, distance-independent direction representation, and normalize it, including calculating the L2 norm (Euclidean norm) of the direction vector z, as follows:

[0072]

[0073] Finally, dividing the direction vector z by its own L2 norm yields the final unit direction vector v, as shown in the formula:

[0074]

[0075] The unit direction vector v has a length of 1 and its direction is uniformly distributed on the unit sphere. The unit direction vector v serves as the reference for all subsequent calculations and represents the line of sight of the virtual sensor, i.e., the virtual viewpoint direction.

[0076] S2. Based on the virtual viewpoint direction, perform depth projection and sorting of the original point cloud.

[0077] After determining the unit direction vector v of the observation, the purpose of this step is to quantize the original point cloud. The visibility or exposure of each point relative to that viewpoint.

[0078] Geometrically, the "protruding" parts of a point cloud facing the viewpoint are easier to see, while the "recessed" or "back" parts are more likely to be obscured.

[0079] This embodiment utilizes vector dot product operations to efficiently implement this quantization process. For each point in the original point cloud P... (It is itself a three-dimensional coordinate vector), calculate its dot product with the viewpoint unit direction vector v, and obtain a scalar value. As a depth value, the formula is expressed as:

[0080]

[0081] Among them, depth value The physical meaning of is point The length of the projection on the observation axis defined by v. The larger the value, the stronger the point. The closer an object's surface is to the virtual viewpoint, the higher its visibility priority.

[0082] Calculate the depth value for each of the N points. Then, all points in the original point cloud P are sorted according to their corresponding depth values. Sort in descending order.

[0083] After sorting, an ordered list of points is obtained, where the point at the beginning of the list has the largest depth value, i.e., the most "visible" point; and the point at the end of the list is the least "visible" point. This step simplifies the complex 3D visibility problem into an efficient 1D sorting problem.

[0084] S3. Based on the depth projection and sorting results, simulate occlusion and select the set of visible points.

[0085] This step is the core of implementing structured occlusion. It simulates different degrees of occlusion based on an adjustable parameter and selects the set of visible points from the sorted list of points.

[0086] First, to enable data augmentation with diverse occlusion intensities, this embodiment introduces a hyperparameter—the retention ratio range. For example, it can be set to [0.5, 0.8].

[0087] This scale range defines the lower and upper limits of the percentage of original points retained in the augmented point cloud. During each augmentation operation, points are uniformly distributed within this range. A specific value is randomly sampled as the retention ratio. .

[0088] Then, based on the sampled proportion and original point cloud Total points Calculate the number of visible points that need to be retained in this operation. The calculation method is to and The product result is rounded down to ensure that an integer is obtained:

[0089]

[0090] Finally, from the list of points sorted in descending order, directly select the first... One point. This Each point, having the highest depth value, constitutes the visible surface of an object as seen from the viewpoint.

[0091] This set of points is defined as the set of visible points. Since these points are adjacent in the original 3D space, the points that are removed (i.e., those at the end of the list) are... These individual points also form a continuous, structured region, thus perfectly simulating the effect of physical occlusion.

[0092] S4. Obtain the supplementary point set based on the visible point set.

[0093] The purpose of this step is to address the training compatibility issues caused by changes in the number of point clouds, and to cleverly utilize this process to further enhance the features of the data.

[0094] In deep learning practice, in order to utilize GPUs for efficient parallel computing, data is usually processed in fixed-size batches, requiring all samples within a batch to have the exact same dimension, i.e. the same number of points.

[0095] The detailed operation procedure for this step is as follows:

[0096] First, calculate the replenishment quantity by determining the number of points removed from the original point cloud, which is the number of points that need to be replenished. The calculation formula is:

[0097]

[0098] Then, it is necessary to generate To achieve this goal, an interpolation-based method can be used to ensure that the newly generated points can smoothly fill the visible surface.

[0099] In this embodiment, these supplementary points are not sampled from the original point cloud P, but from the newly generated set of visible points. Sampling is performed during this process. The reference points for interpolation are derived from the set of visible points. Instead of the original point cloud P, the point cloud density of the visible surface is increased by interpolating between visible points. This can be more precisely interpreted physically as the sensor performing a finer, higher-resolution scan of the visible area.

[0100] Obtaining the supplementary point set based on interpolation method The process includes:

[0101] First, from A "seed point" is randomly and uniformly selected.

[0102] Then, the efficient nearest neighbor search algorithm (KD-Tree) is used to find the seed point in... Find the k nearest neighbors in the array.

[0103] Finally, a completely new point is generated by taking a weighted average (i.e., linear interpolation) of the seed point and its k neighbors.

[0104] Repeat the above process This ultimately forms the supplementary point set. .

[0105] S5. Merge the visible point set and the supplementary point set to obtain the enhanced point cloud.

[0106] First, the set of visible points. With the newly generated supplementary point set The points are then merged to form the final enhanced point cloud. The formula is expressed as

[0107]

[0108] At this point, the total number of points in the augmented point cloud P' is exactly This restored the original number of point clouds, thus solving the compatibility problem of batch training.

[0109] Furthermore, for enhanced point clouds Perform random shuffling, as the merging operation may result in enhanced point clouds. The points in the data have an order (e.g., all of them). The point comes first, all (The points are placed later in the order of the points). To prevent deep learning models from learning useless biases from this artificial order, the point cloud must be augmented. All of them Each point is randomly shuffled to completely disrupt its order, forming the final enhanced point cloud. .

[0110] Example 2:

[0111] This embodiment builds upon the method in Embodiment 1 by extracting an original point cloud sample of the "airplane" category from the ModelNet40 training set. Taking an example, the enhancement process of the method of the present invention is fully and step by step demonstrated.

[0112] Original point cloud It is a NumPy array of shape (1024,3) representing the coordinates of 1024 3D points, and the point cloud is complete and unobstructed.

[0113] First, virtual viewpoint selection is performed. This step is executed dynamically each time an individual sample is augmented to ensure the diversity of augmentations.

[0114] Generate random vectors using the `random.randn` function from the NumPy library, derived from a three-dimensional standard Gaussian distribution. A random direction vector z is sampled. Assume that in this specific execution, the sampled vector is:

[0115]

[0116] After normalization, the L2 norm (i.e., Euclidean length) of vector z is calculated using the following formula:

[0117]

[0118]

[0119] To determine the viewpoint direction, divide each component of the direction vector z by its L2 norm to obtain the unit direction vector v, as shown in the formula:

[0120]

[0121] This unit direction vector v is the virtual viewpoint direction used in this enhancement operation.

[0122] Then, based on the viewpoint unit direction vector v, the original point cloud is generated. Depth projection and sorting. The goal of this step is to... A "visibility score" is calculated for all 1024 points in the data.

[0123] Depth values ​​are calculated using NumPy's matrix multiplication function (@ operator or np.dot) to transform a point cloud matrix of shape (1024, 3). Perform a dot product operation with the viewpoint direction vector v (transposed) of shape (3,1). Calculate the depth value for each point. This yields a depth vector D with shape (1024, 1), expressed by the formula:

[0124]

[0125] Data sorting, from the original point cloud This is associated with the depth value vector D. The values ​​in the depth value vector D are then sorted in descending order. After sorting, a new point cloud set is obtained. The first row corresponds to the point with the maximum depth value, and the last row corresponds to the point with the minimum depth value. Simultaneously, a sorted list of points is also obtained, recording the position of each point in the original point cloud in the new list.

[0126] Then, occlusion simulation and point selection are performed. This step removes some points according to preset rules to form structured occlusion.

[0127] The retention ratio is determined, and in this embodiment, the hyperparameter range of the retention ratio is set as follows: Using Python's `random.uniform` function, a value is randomly sampled within this range. Let's assume the sampled value is... .

[0128] Calculate the retention quantity based on the sampled proportion. Total points Calculate the number of visible points that need to be retained. .

[0129]

[0130] Filter the set of visible points based on the sorted point cloud set. We directly select the first 737 rows. These 737 points constitute the set of visible points. .

[0131] These points correspond in three-dimensional space to the continuous surface of the aircraft model facing the viewpoint v, such as the fuselage and one wing of the aircraft. (Discarded) These points correspond to the back of the aircraft and the other obscured wing, forming a continuous, structured missing area, as shown in the attached diagram. Figure 2 The principle is shown on the right.

[0132] Next, point cloud data preservation is performed. This step is crucial to ensure that the augmented data can be directly fed into the model for training, and it also increases the feature density of the visible area.

[0133] Calculate the replenishment quantity and the number of points that need to be replenished. The formula is:

[0134]

[0135] To perform point cloud supplementation based on neighborhood interpolation, the 287 supplementary points can be generated using a method based on local neighborhood interpolation, rather than simple resampling.

[0136] Local neighborhood interpolation-based methods can generate entirely new points on visible surfaces, thus more realistically simulating high-density scans. Specifically, these methods include:

[0137] Construct a search structure within the set of visible points. Construct a KD-Tree on (containing 737 points) to quickly find the nearest neighbor of any point, repeating the operation less than 287 times: A point is randomly and uniformly selected as the "seed point".

[0138] Use a KD-Tree to find the k nearest neighbors of the seed point (e.g., k=3). Then perform interpolation: generate a set of random weights that sum to 1, and then calculate a weighted average of the seed point and its k neighbors to obtain a new 3D coordinate, which is the supplementary point.

[0139] The newly generated points from these 287 points using interpolation are then combined to form a supplementary point set. In this way, Each point in is based on The points are "created" by local geometric structures, which smoothly fill the spaces between visible points, effectively improving the density and surface detail of the point cloud and avoiding the problems of point overlap and local aggregation caused by simple repeated sampling.

[0140] Finally, the point clouds are merged using NumPy's `concatenate` function, which combines the visible point sets. (Shape is (737,3)) and supplementary point set (The shape is (287,3)) is merged on the first dimension.

[0141]

[0142] at this time, The shape is (1024,3), and the number of point clouds has been restored to the original size.

[0143] Random shuffling, for All 1024 rows are randomly shuffled once, which can be done by generating a random index from 0 to 1023 and then using that index to rearrange the rows. This is achieved through the lines of code. This step is crucial because it eliminates the artificial ordering introduced by the merging operation (i.e., all original visible points first, all supplementary points last), ensuring that the data input to the neural network is unordered, which conforms to the fundamental property of point clouds as unordered sets.

[0144] The above steps together constitute a complete "on-the-fly" data augmentation function. During the PointNet++ model's training loop, this augmentation function is called independently for each point cloud sample in the batch each time a batch of data is retrieved from the DataLoader.

[0145] This means that for the same "aircraft" sample :

[0146] In the first training cycle (epoch 1), it may be obscured by a viewpoint from "directly above".

[0147] In the second training cycle (epoch2), it may be obscured by a viewpoint from a "45-degree angle to the left front".

[0148] Even within the same period, if the dataset is iterated multiple times, the occlusion viewpoints and occlusion ratios it encounters each time are entirely new and random.

[0149] like Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the occlusion simulation principle in this embodiment. Figure 2 (a) in the image is the original point cloud: showing a complete, unobstructed 3D point cloud of the aircraft. Figure 2 (b) Viewpoint and Visible Surface: The red arrows indicate randomly selected viewpoint vectors, which are opposite to the real viewpoint direction. The darker the point cloud color, the greater the projection depth value and the higher the visibility score. Figure 2 (c) Point cloud after occlusion. Blue represents visible points, and gray represents occluded points. Figure 2 (d) Final augmented point cloud: Showing the final result P' after executing S4. It can be observed that... Figure 2 The point cloud density of the visible part in (d) has increased, and the interpolated orange points have been added.

[0150] Figure 2 This embodiment demonstrates how to generate a sample with realistic, continuous occlusion from a complete object, and clearly illustrates the effect of the "point cloud quantity preservation" step.

[0151] Figure 3 This is a comparison chart of the effects of this embodiment and the prior art (randomly dropping points). The results of the method in this embodiment are as follows: Figure 3 As shown in (a), this is the result after applying the method of this embodiment. Structured continuous occlusion samples. The results of the random drop-point method are as follows... Figure 3 As shown in (b), the entire aircraft model becomes uniformly sparse, but the overall outline is still visible.

[0152] Depend on Figure 3 It is evident that the method in this embodiment can simulate real physical occlusion, which is far superior to the traditional random point discarding method.

[0153] This dynamic and diverse augmentation approach exposes the model to occlusion of varying severity from almost every possible angle throughout the training process. This forces the model to avoid relying on any specific local features (as they can be occluded at any time) and instead learn to find more essential and robust global structural information in the remaining, incomplete view to make the classification decision of "airplane".

[0154] The advantages of this embodiment are obvious compared to existing technologies (such as random point discarding). If for P... airplane Using random point discarding (e.g., discarding 25% of the points) results in the entire aircraft model becoming uniformly sparse, but its overall outline remains clearly visible. However, using the method of this invention, the result is that parts of the aircraft (such as the entire right wing and tail) completely disappear, while the point cloud density of the visible parts (left wing and nose) may even increase. The latter is clearly a more difficult and realistic recognition task.

[0155] In summary, this embodiment demonstrates in detail and reproducibly the complete process of the method proposed in this invention. Through this simulation based on physical reality, the method of this invention can generate high-quality, challenging augmentation data, thereby enabling the deep learning model to exhibit robustness and accuracy far exceeding that of existing technologies when facing various unavoidable occlusions in the real world after training. This embodiment uses specific examples to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention; at the same time, for those skilled in the art, based on the ideas of this invention, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0156] This invention, starting from a physical process, innovatively transforms the data augmentation problem into a process simulating physical observation—that is, "photographing" a 3D object from a random viewpoint and retaining only the visible parts. Furthermore, unlike randomly removing individual points, this method employs structured information removal, removing a continuous, structured set of points, forcing the model to learn the ability to infer the overall structure from incomplete local views. Simultaneously, by using a neighborhood-based interpolation method for visible points to maintain a constant total number of points, it cleverly solves the problem of inconsistent sample sizes after data augmentation, allowing it to be directly used in standard training processes.

[0157] In summary, this invention effectively addresses the shortcomings of existing technologies in handling point cloud occlusion by employing a novel, physically based viewpoint occlusion simulation method, providing an efficient and universal solution for training more robust and reliable 3D point cloud classification models.

[0158] The present invention also provides an electronic device, Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can invoke logical instructions from the memory, for example, to execute the following method:

[0159] S1. Based on the original point cloud, select the virtual viewpoint direction;

[0160] S2. Based on the virtual viewpoint direction, perform depth projection and sorting of the original point cloud;

[0161] S3. Based on the depth projection and sorting results, simulate occlusion and select the set of visible points;

[0162] S4. Based on the visible point set, obtain the supplementary point set;

[0163] S5. Merge the visible point set and the supplementary point set to obtain the enhanced point cloud.

[0164] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments, including, for example:

[0166] S1. Based on the original point cloud, select the virtual viewpoint direction;

[0167] S2. Based on the virtual viewpoint direction, perform depth projection and sorting of the original point cloud;

[0168] S3. Based on the depth projection and sorting results, simulate occlusion and select the set of visible points;

[0169] S4. Based on the visible point set, obtain the supplementary point set;

[0170] S5. Merge the visible point set and the supplementary point set to obtain the enhanced point cloud.

[0171] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for augmenting 3D point cloud data based on viewpoint occlusion simulation, characterized in that, include: S1. Based on the original point cloud, select the virtual viewpoint direction; S2. Based on the virtual viewpoint direction, perform depth projection and sorting of the original point cloud; S3. Based on the depth projection and sorting results, simulate occlusion and select the set of visible points; S4. Based on the visible point set, obtain the supplementary point set; S5. Merge the visible point set and the supplementary point set to obtain the enhanced point cloud.

2. The 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 1, characterized in that, S1 includes the following steps: S11, from the three-dimensional standard Gaussian distribution A random direction vector z is sampled from the sample. S12. Normalize the direction vector z and calculate the L2 norm of the direction vector z; S13. Based on the L2 norm, calculate the unit direction vector v of the direction vector z as the virtual viewpoint direction.

3. The 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 2, characterized in that, The unit direction vector v is expressed by the formula: ； ； in, It is a direction vector. Let z be the L2 norm of the direction vector z.

4. The 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 2, characterized in that, S2 includes the following steps: S21. Calculate the original point cloud The depth set is obtained by taking the dot product with the unit direction vector v. The formula is expressed as: ； in, , It is the position direction vector. for The depth value; S22, For depth sets Sort the points to obtain the original point cloud P sorted point list.

5. The 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 4, characterized in that, S3 includes the following steps: S31. Determine the retention ratio ; S32, Based on the retention ratio Calculate the number of points in the set of visible points that need to be retained. The formula is: ； in, Original point cloud The number of points, This is a floor function; S33. Select the sorting point list Construct a set of visible points from these points. .

6. The 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 5, characterized in that, S4 includes the following steps: S41. Calculate the supplementary point set The number of points is expressed by the formula: ； in, Original point cloud The number of points, For the set of visible points The number of points, To supplement the point set The number of points; S42, Based on Visible Point Set Interpolation method is used to generate These supplementary points constitute the supplementary point set. .

7. A 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 6, characterized in that, In step S42, the supplementary point set is obtained based on interpolation. The steps include: S42a, From the set of visible points A seed point is randomly and uniformly selected from the data. S42b: Use the nearest neighbor search algorithm to find the seed point in the set of visible points. Find the k nearest neighbors in the array; S42c: Calculate a weighted average of the seed point and its k neighbors to generate a new point; S42d, repeat S42a~S42c together Next, a supplementary point set is formed. .

8. A 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 6, characterized in that, The enhanced point cloud in S5 is expressed by the following formula: ； in, To enhance point clouds, For the set of visible points, To supplement the point set, The number of points is , The number of points is ,and .

9. A 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 8, characterized in that, During each enhancement operation, from the range of values uniform distribution Random sampling determines the retention ratio .

10. A 3D point cloud data augmentation method based on viewpoint occlusion simulation according to claim 8, characterized in that, Enhanced point cloud Perform a random shuffle to disrupt the order of the elements.

Images