Point cloud data transformation method, point cloud classification method and related devices

Abstract

The invention discloses a point cloud data transformation method, a point cloud classification method and a related device, and the transformation method comprises the steps: carrying out the sampling and feature extraction of an original point cloud sample through an encoder, and obtaining a sampled point cloud sample; extracting skeleton points from the sampled point cloud sample, calculating the nearest distance from each point cloud in the sampled point cloud sample to each skeleton point to obtain a radius value corresponding to each skeleton point, and generating a skeleton ball; calculating a surface point coordinate of each skeleton point on the skeleton ball, and calculating a loss value according to the coordinate of each point cloud in the original point cloud sample, the coordinate of each skeleton point and the surface point coordinate on the skeleton ball; obtaining an optimized skeleton point sample by optimizing the loss value; and carrying out local linear interpolation on the skeleton point sample to obtain a skeleton point sample containing corruption information. According to the method and the device, the technical problem that the recognition accuracy is relatively low when a point cloud classification network recognizes damaged point cloud data due to the fact that a point cloud key structure cannot be captured or corruption information is insufficient in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud data processing, and in particular to a point cloud data transformation method, a point cloud classification method and related devices. Background Art

[0002] At present, the transformation methods used in the preprocessing stage of the point cloud classification network, whether basic data enhancement methods (such as rotation, shearing and noise, etc.) or transformation methods such as Rsmix and PointMixup with better classification effects, have limited improvement on the corruption robustness of the point cloud classification network, making it still impossible for the point cloud classification network to achieve a high recognition accuracy when facing corrupt point cloud data. The corruption robustness of the point cloud classification network is crucial in real-world application scenarios. In a diverse and unpredictable actual environment, a point cloud classification network with good robustness can properly cope with various challenges and ensure the stability and accuracy of its classification performance when facing corrupt point cloud data. This is crucial for the integration and application of point cloud technology in multiple fields such as autonomous driving. It ensures that even when facing complex situations in the real world, the model can maintain the reliability and effectiveness of its classification tasks, thereby enhancing the trust and application value of the entire system. Summary of the invention

[0003] The present application provides a point cloud data transformation method, a point cloud classification method and related devices, which are used to improve the situation that the existing point cloud data enhancement method and point cloud data transformation method cannot capture the key structure of the point cloud or the corruption information is insufficient, so that the point cloud classification network cannot fully learn the point cloud structure and corruption characteristics, resulting in the technical problem of low recognition accuracy of the point cloud classification network when identifying damaged point cloud data.

[0004] In view of this, the first aspect of the present application provides a point cloud data transformation method, comprising:

[0005] The encoder samples and extracts features from the original point cloud samples to obtain sampled point cloud samples;

[0006] Extract a target number of skeleton points from the sampled point cloud samples, obtain the radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball;

[0007] Calculate the surface point coordinates of each skeleton point on the skeleton ball according to the coordinates of each skeleton point and the corresponding radius value, and calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the surface point coordinates of each skeleton point on the skeleton ball;

[0008] Optimizing the position of the skeleton point by using the loss value to obtain an optimized skeleton point sample;

[0009] Perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

[0010] Optionally, extracting a target number of skeleton points from the sampled point cloud samples, obtaining a radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generating a skeleton ball includes:

[0011] Obtain a target number of bone points by multiplying the sampled point cloud samples by a weight matrix;

[0012] Calculate the shortest distance from each point cloud in the sampled point cloud sample to each of the skeleton points to generate a distance vector;

[0013] The radius value corresponding to each skeleton point is obtained by multiplying the weight matrix and the distance vector to generate a skeleton ball.

[0014] Optionally, optimizing the positions of the skeleton points by using the loss values ​​to obtain optimized skeleton point samples includes:

[0015] The weight matrix is ​​updated according to the loss value to minimize the loss value, and the bone point corresponding to the minimum loss value is output.

[0016] Optionally, the calculating of the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the coordinates of the surface point of each skeleton point on the skeleton ball includes:

[0017] Calculate the distance between each point cloud and the surface point of each skeleton point on the skeleton ball according to the coordinates of each point cloud in the original point cloud sample and the coordinates of the surface point of each skeleton point on the skeleton ball, and obtain the sampling loss;

[0018] Calculate the difference between the distance between each point cloud and each skeleton point and the corresponding radius value according to the coordinates of each point cloud in the original point cloud sample and the coordinates of each skeleton point, and obtain the point cloud to skeleton ball loss;

[0019] Sum the radius values ​​of each of the skeleton points to obtain a radius regularization loss;

[0020] The sampling loss, the point cloud to skeleton ball loss and the radius regularization loss are weightedly summed to obtain the final loss value.

[0021] Optionally, performing local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information includes:

[0022] Calculate the distance between each bone point in the bone point sample by using the K nearest neighbor algorithm;

[0023] A skeleton point is randomly selected from the K skeleton points closest to each of the skeleton points to generate a new skeleton point, and a skeleton point sample containing corruption information is obtained.

[0024] The second aspect of the present application provides a point cloud classification method, comprising:

[0025] The acquired original point cloud data is transformed to obtain skeleton point samples containing corruption information; wherein the original point cloud data is transformed using any point cloud data transformation method described in the first aspect to obtain skeleton point samples containing corruption information;

[0026] Inputting the skeleton point samples containing corruption information into a point cloud classification network for training, and obtaining a trained point cloud classification network;

[0027] The collected point cloud data is input into the trained point cloud classification network for classification.

[0028] The third aspect of the present application provides a point cloud data conversion device, comprising:

[0029] A sampling and feature extraction unit, used for sampling and extracting features from the original point cloud samples through an encoder to obtain sampled point cloud samples;

[0030] A skeleton extraction unit is used to extract a target number of skeleton points from the sampled point cloud samples, obtain a radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball;

[0031] A loss calculation unit, used to calculate the surface point coordinates of each skeleton point on the skeleton ball according to the coordinates of each skeleton point and the corresponding radius value, and to calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the surface point coordinates of each skeleton point on the skeleton ball;

[0032] An optimization unit, used for optimizing the position of the skeleton point by using the loss value to obtain a skeleton point sample;

[0033] The interpolation unit is used to perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

[0034] A fourth aspect of the present application provides a point cloud classification device, comprising:

[0035] A transformation unit, used to transform the acquired original point cloud data to obtain skeleton point samples containing corruption information; wherein the original point cloud data adopts any point cloud data transformation method described in the first aspect to obtain skeleton point samples containing corruption information;

[0036] A training unit, used for inputting the skeleton point samples containing corruption information into a point cloud classification network for training, and obtaining a trained point cloud classification network;

[0037] The classification unit is used to input the collected point cloud data into the trained point cloud classification network for classification.

[0038] A fifth aspect of the present application provides an electronic device, the device comprising a processor and a memory;

[0039] The memory is used to store program code and transmit the program code to the processor;

[0040] The processor is used to execute any one of the point cloud data transformation methods described in the first aspect according to the instructions in the program code.

[0041] In a sixth aspect, the present application provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, it implements the point cloud data transformation method described in any one of the first aspects.

[0042] It can be seen from the above technical solutions that this application has the following advantages:

[0043] The point cloud data transformation method provided in the present application, in the process of processing the point cloud data to generate the corresponding skeleton point data, successfully captures the intrinsic structure and topological properties of the object shape by utilizing the geometric transformation and connectivity features learned from the original point cloud data, and creates a new data result that can represent the skeleton model. The use of the skeleton point data as a representation not only simplifies the point cloud sample data, but also enhances the prominent display of key structural features; by linearly interpolating the generated skeleton point cloud samples, the skeleton point cloud samples contain more corruption information, and then the point cloud classification network learns corruption knowledge based on the key structure of the point cloud to deal with corruption phenomena in real scenes. The point cloud data transformation method provided in the present application can more effectively solve the problem that other methods cannot capture the key structure of the point cloud or the corruption information is insufficient, and enables the point cloud classification network to extract the point cloud structure and corruption features at the same time, and can more accurately identify damaged point cloud data, and enhance the corruption robustness of the point cloud classification network. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic diagram of a process of a point cloud data transformation method provided in an embodiment of the present application;

[0046] Figure 2 Another schematic diagram of a flow chart of a point cloud data transformation method provided in an embodiment of the present application;

[0047] Figure 3 The skeleton point data provided in the embodiment of the present application generates an effect diagram;

[0048] Figure 4 A diagram of the local linear interpolation process of the skeleton point samples provided in the embodiment of the present application;

[0049] Figure 5 A schematic diagram of a flow chart of a point cloud classification method provided in an embodiment of the present application;

[0050] Figure 6 The Chinese and English operation table corresponding to the corruption type transformation provided in the embodiment of the present application;

[0051] Figure 7 A comparison chart of experimental results provided in the embodiments of the present application;

[0052] Figure 8 A structural schematic diagram of a point cloud data conversion device provided in an embodiment of the present application;

[0053] Fig. 9 A structural schematic diagram of a point cloud classification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] In order to make the point cloud classification network maintain high stability and more accurately identify point cloud data when faced with corruption phenomena such as noise, missing points or outliers in point cloud data due to improper collection, transmission and processing or other factors such as equipment and environment, the point cloud classification network can deal with some damaged point cloud data well in actual scene applications in fields such as autonomous driving, and improve the corruption robustness of the network. This application proposes a transformation method for application in the preprocessing stage of the point cloud classification network. The method first deeply explores the inherent geometric features of the point cloud data to identify a set of key points. These points constitute the structural framework of the point cloud, which are called point cloud skeleton points. Then, a radius value is calculated for each skeleton point to define the corresponding skeleton ball. By adjusting the radius of these skeleton balls, the position of the skeleton points is further optimized to ensure its accuracy and representativeness.

[0056] For easier understanding, please refer to Figure 1 and Figure 2 , the embodiment of the present application provides a point cloud data transformation method, including:

[0057] Step 110: Sample and extract features of the original point cloud samples through an encoder to obtain sampled point cloud samples.

[0058] The original point cloud sample x is represented as a set , where N represents the total number of point clouds in the original point cloud sample. The PointNet++ network can be used as an encoder to perform point cloud sample sampling and feature extraction to obtain the sampled point cloud sample ,in , Indicates the number of point cloud samples after sampling.

[0059] Step 120: extract a target number of skeleton points from the sampled point cloud samples, obtain the radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball.

[0060] The point cloud data transformation method in this application is to predict the original point cloud samples to obtain N s The target number of bone points is obtained by multiplying the weight matrix with the sampled point cloud sample; the shortest distance from each point cloud to each bone point in the sampled point cloud sample is calculated to generate a distance vector; the radius value corresponding to each bone point is obtained by multiplying the weight matrix with the distance vector to generate a bone ball.

[0061] Specifically, define a weight matrix that satisfies the convex combination , and then use the following formula to perform matrix multiplication to get the skeleton point set , where c j is the coordinate of the jth bone point, that is:

[0062]

[0063] A radius value is calculated for each generated bone point. First, all sampling points are calculated. (i.e., all point clouds in the point cloud sample after sampling) to the nearest distance d of all bone points i , and form a distance vector ; The distance calculation formula is:

[0064]

[0065] Then according to the formula The radius value corresponding to each bone point can be obtained. After calculating the radius value of each bone point, the corresponding bone ball can be defined. By adjusting the radius of these bone balls, the position of the bone point can be further optimized to ensure its accuracy and representativeness.

[0066] Step 130, calculate the surface point coordinates of each bone point on the skeleton ball according to the coordinates of each bone point and the corresponding radius value, and calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each bone point and the surface point coordinates of each bone point on the skeleton ball.

[0067] The positions of the skeleton points are optimized by the loss function, so that the generated skeleton points are more accurate. In the embodiment of the present application, the loss value includes sampling loss, point cloud to skeleton ball loss and radius regularization loss.

[0068] The calculation process of sampling loss is as follows: according to the coordinates of each point cloud in the original point cloud sample and the coordinates of the surface point of each bone point on the bone ball, the distance between each point cloud and the surface point of each bone point on the bone ball is calculated to obtain the sampling loss;

[0069] Specifically, first set a unit vector , a total of 8 values;

[0070] Then use the formula Find the bone point c j The surface point coordinates of the skeleton ball are used to obtain the coordinates of the skeleton surface points, and finally form a skeleton surface point set. ,in Represents the coordinates of the jth bone surface point;

[0071] Finally, the sampling loss L is obtained using the following formula s :

[0072]

[0073] The calculation process of the point cloud to skeleton ball loss is as follows: according to the coordinates of each point cloud in the original point cloud sample and the coordinates of each skeleton point, the difference between the distance between each point cloud and each skeleton point and the corresponding radius value is calculated to obtain the point cloud to skeleton ball loss L p ,Right now:

[0074]

[0075] In the formula, Represents the distance point cloud x i The radius value of the nearest bone point.

[0076] In order to prevent the radius from being too small due to noise factors, the radius regularization loss is also calculated in the embodiment of the present application. The radius regularization loss L is obtained by summing the radius values ​​of each bone point. r ,Right now:

[0077]

[0078] Due to the existence of the negative sign, reducing the loss value can ensure that the radius value is not too small. The previous radius calculation formula can limit the radius value from being too large, thus keeping it within a reasonable range.

[0079] Finally, the sampling loss L s , point cloud to skeleton ball loss L p and radius regularization loss L r Perform weighted summation to get the final loss value L sk ,Right now:

[0080]

[0081] in and Represents the weight coefficient, which is used to measure the importance of different loss functions.

[0082] Step 140: Optimize the positions of the skeleton points by using the loss value to obtain optimized skeleton point samples.

[0083] Update the weight matrix W according to the loss value to minimize the loss value, output the bone point corresponding to the minimum loss value, and obtain the optimized bone point sample. The network minimizes the loss through continuous iteration of back propagation to make the generated bone points more accurate. The effect is shown in the figure below. Figure 3 shown.

[0084] Step 150: Perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

[0085] Local linear interpolation is performed on the skeleton point samples obtained in step 140 so that the skeleton point samples can contain more corruption information, so that the point cloud classification network can enhance the corruption robustness after extracting the skeleton point sample features. Figure 4 , calculate the distance between each bone point in the bone point sample x' by using the K nearest neighbor algorithm; randomly select a bone point from the K (such as K=5) bone points closest to each bone point to generate a new bone point P', and obtain a bone point sample x'' containing corruption information. The generation process of the new bone point P' is:

[0086]

[0087] Where P is a skeleton point in the skeleton point sample; is the interpolation coefficient, and Q is a randomly selected neighboring point.

[0088] The new sample x'' is obtained by the above method, and the generated new sample x'' is imported as input into the point cloud classification network for training, which can effectively improve the corruption robustness of the classification network.

[0089] The point cloud data transformation method provided in the present application, in the process of processing the point cloud data to generate the corresponding skeleton point data, successfully captures the intrinsic structure and topological properties of the object shape by utilizing the geometric transformation and connectivity features learned from the original point cloud data, and creates a new data result that can represent the skeleton model. The use of the skeleton point data as a representation not only simplifies the point cloud sample data, but also enhances the prominent display of key structural features; by linearly interpolating the generated skeleton point cloud samples, the skeleton point cloud samples contain more corruption information, and then the point cloud classification network learns corruption knowledge based on the key structure of the point cloud to deal with corruption phenomena in real scenes. The point cloud data transformation method provided in the present application can more effectively solve the problem that other methods cannot capture the key structure of the point cloud or the corruption information is insufficient, and enables the point cloud classification network to extract the point cloud structure and corruption features at the same time, and can more accurately identify damaged point cloud data, and enhance the corruption robustness of the point cloud classification network.

[0090] The above is an embodiment of a point cloud data transformation method provided by the present application, and the following is an embodiment of a point cloud classification method provided by the present application.

[0091] Please refer to Figure 5 , a point cloud classification method provided in an embodiment of the present application includes:

[0092] Step 210: transform the acquired original point cloud data to obtain skeleton point samples containing corruption information.

[0093] For the acquired original point cloud data, the point cloud data transformation method in the aforementioned method embodiment is used to acquire the skeleton point samples containing the corruption information.

[0094] Step 220: Input the skeleton point samples containing corruption information into the point cloud classification network for training to obtain a trained point cloud classification network.

[0095] The skeleton point samples containing corruption information are input into the point cloud classification network for training to obtain a trained point cloud classification network. The specific training process belongs to the prior art and will not be described in detail here.

[0096] Step 230: input the collected point cloud data into the trained point cloud classification network for classification.

[0097] The collected point cloud data (such as point cloud data collected by autonomous driving vehicles) is classified through the trained point cloud classification network.

[0098] This application transforms the original point cloud data to obtain skeleton point samples containing corruption information, and then uses the skeleton point samples containing corruption information to train the point cloud classification network, so that the point cloud classification network can simultaneously extract point cloud structure and corruption features, and can more accurately identify damaged point cloud data, thereby enhancing the corruption robustness of the point cloud classification network.

[0099] In order to verify the effectiveness of the method proposed in this application, two classification networks, PointNet++ and PointNeXt, were selected for experiments. ModelNet40-C was selected as the corruption experimental dataset, and the number of generated skeleton points N was set. s is 100, the number of skeleton points after linear interpolation is 200, and then repeated sampling is performed to 1000, the minimum batch size is 64, the epoch is 250, and the method of this application (SK) is compared with the standard method (ST) using noise and medial axis transformation.

[0100] The ModelNet40-C dataset is based on the ModelNet40 test set. Each sample is subjected to 15 corruption operations. Each operation contains 5 levels, which is equivalent to 75 corruption transformations for each sample. The total sample size is 75 times the original. The Chinese and English operations corresponding to the corruption type transformation are as follows: Figure 6 The average precision mOA of 15 corruption types in ModelNet40-C is used as the evaluation index. The experimental results are shown in Figure 7 As shown. Figure 7It can be seen that the mOA when using the method of this application with PointNet++ as the benchmark network is improved by 6.88 percentage points, and the mOA when using PointNext as the benchmark network is improved by 9.57 percentage points, which proves that the transformation method proposed in this application can effectively enhance the corruption robustness of the point cloud classification network.

[0101] Please refer to Figure 8 The embodiment of the present application also provides a point cloud data conversion device, including:

[0102] The sampling and feature extraction unit 310 is used to sample and extract features from the original point cloud samples through an encoder to obtain sampled point cloud samples;

[0103] The skeleton extraction unit 320 is used to extract a target number of skeleton points from the sampled point cloud samples, obtain the radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball;

[0104] A loss calculation unit 330 is used to calculate the surface point coordinates of each bone point on the bone ball according to the coordinates of each bone point and the corresponding radius value, and to calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each bone point and the surface point coordinates of each bone point on the bone ball;

[0105] An optimization unit 340 is used to optimize the positions of the skeleton points by using the loss value to obtain optimized skeleton point samples;

[0106] The interpolation unit 350 is used to perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

[0107] As a further improvement, the skeleton extraction unit 320 is specifically used for:

[0108] The target number of bone points is obtained by multiplying the weight matrix with the sampled point cloud samples;

[0109] Calculate the shortest distance from each point cloud to each skeleton point in the sampled point cloud sample and generate a distance vector;

[0110] The radius value corresponding to each bone point is obtained by multiplying the weight matrix and the distance vector to generate a bone sphere.

[0111] As a further improvement, the optimization unit 340 is specifically used to update the weight matrix according to the loss value to minimize the loss value, output the skeleton point corresponding to the minimum loss value, and obtain the optimized skeleton point sample.

[0112] As a further improvement, the loss calculation unit 330 is specifically used to:

[0113] According to the coordinates of each point cloud in the original point cloud sample and the coordinates of the surface point of each bone point on the bone sphere, the distance between each point cloud and the surface point of each bone point on the bone sphere is calculated to obtain the sampling loss;

[0114] According to the coordinates of each point cloud in the original point cloud sample and the coordinates of each skeleton point, the difference between the distance between each point cloud and each skeleton point and the corresponding radius value is calculated to obtain the point cloud to skeleton ball loss;

[0115] Sum the radius values ​​of each bone point to obtain the radius regularization loss;

[0116] The sampling loss, point cloud to skeleton ball loss and radius regularization loss are weighted summed to get the final loss value.

[0117] As a further improvement, the interpolation unit 350 is specifically configured to:

[0118] The distance between each bone point in the bone point sample is calculated by the K nearest neighbor algorithm;

[0119] A new skeleton point is generated by randomly selecting a skeleton point from the K skeleton points closest to each skeleton point, and a skeleton point sample containing corruption information is obtained.

[0120] The point cloud data transformation method provided in the present application, in the process of processing the point cloud data to generate the corresponding skeleton point data, successfully captures the intrinsic structure and topological properties of the object shape by utilizing the geometric transformation and connectivity features learned from the original point cloud data, and creates a new data result that can represent the skeleton model. The use of the skeleton point data as a representation not only simplifies the point cloud sample data and reduces the amount of data, but also enhances the prominent display of key structural features; by performing local linear interpolation on the generated skeleton point cloud samples, the skeleton point cloud samples contain more corruption information, thereby enabling the point cloud classification network to learn corruption knowledge based on the key structure of the point cloud to cope with corruption phenomena in real scenes. The point cloud data transformation method provided in the present application can more effectively solve the problem that other methods cannot capture the key structure of the point cloud or the corruption information is insufficient, and enables the point cloud classification network to extract the point cloud structure and corruption features at the same time, and can more accurately identify damaged point cloud data, thereby enhancing the corruption robustness of the point cloud classification network.

[0121] Please refer to Fig. 9 , the embodiment of the present application also provides a point cloud classification device, including:

[0122] The transformation unit 410 is used to transform the acquired original point cloud data to obtain the skeleton point samples containing the corruption information; wherein the original point cloud data adopts the point cloud data transformation method in the above method embodiment to obtain the skeleton point samples containing the corruption information;

[0123] A training unit 420 is used to input the skeleton point samples containing corruption information into the point cloud classification network for training, and obtain a trained point cloud classification network;

[0124] The classification unit 430 is used to input the collected point cloud data into the trained point cloud classification network for classification.

[0125] This application transforms the original point cloud data to obtain skeleton point samples containing corruption information, and then uses the skeleton point samples containing corruption information to train the point cloud classification network, so that the point cloud classification network can simultaneously extract point cloud structure and corruption features, and can more accurately identify damaged point cloud data, thereby enhancing the corruption robustness of the point cloud classification network.

[0126] The embodiment of the present application also provides an electronic device, the device comprising a processor and a memory;

[0127] The memory is used to store the program code and transmit the program code to the processor;

[0128] The processor is used to execute the point cloud data transformation method in the aforementioned method embodiment according to the instructions in the program code.

[0129] An embodiment of the present application also provides a computer-readable storage medium, which is used to store program code. When the program code is executed by a processor, the point cloud data transformation method in the aforementioned method embodiment is implemented.

[0130] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0131] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0132] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0133] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0134] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for executing all or part of the steps of the method described in each embodiment of the present application through a computer device (which can be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0137] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A point cloud data transformation method, characterized in that: include: The encoder samples and extracts features from the original point cloud samples to obtain sampled point cloud samples; Extract a target number of skeleton points from the sampled point cloud samples, obtain the radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball; Calculate the surface point coordinates of each skeleton point on the skeleton ball according to the coordinates of each skeleton point and the corresponding radius value, and calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the surface point coordinates of each skeleton point on the skeleton ball; Optimizing the position of the skeleton point by using the loss value to obtain an optimized skeleton point sample; Perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

2. The point cloud data transformation method according to claim 1, characterized in that: The step of extracting a target number of skeleton points from the sampled point cloud samples, obtaining a radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generating a skeleton ball includes: Obtain a target number of bone points by multiplying the sampled point cloud samples by a weight matrix; Calculate the shortest distance from each point cloud in the sampled point cloud sample to each of the skeleton points to generate a distance vector; The radius value corresponding to each skeleton point is obtained by multiplying the weight matrix and the distance vector to generate a skeleton ball.

3. The point cloud data transformation method according to claim 2, characterized in that: The step of optimizing the positions of the skeleton points by using the loss values ​​to obtain optimized skeleton point samples includes: The weight matrix is ​​updated according to the loss value to minimize the loss value, and the skeleton point corresponding to the minimum loss value is output to obtain an optimized skeleton point sample.

4. The point cloud data transformation method according to claim 1, characterized in that: The calculating of the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the surface point coordinates of each skeleton point on the skeleton ball comprises: Calculate the distance between each point cloud and the surface point of each skeleton point on the skeleton ball according to the coordinates of each point cloud in the original point cloud sample and the coordinates of the surface point of each skeleton point on the skeleton ball, and obtain the sampling loss; Calculate the difference between the distance between each point cloud and each skeleton point and the corresponding radius value according to the coordinates of each point cloud in the original point cloud sample and the coordinates of each skeleton point, and obtain the point cloud to skeleton ball loss; Sum the radius values ​​of each of the skeleton points to obtain a radius regularization loss; The sampling loss, the point cloud to skeleton ball loss and the radius regularization loss are weightedly summed to obtain the final loss value.

5. The point cloud data transformation method according to claim 1, characterized in that: The step of performing local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information includes: Calculate the distance between each bone point in the bone point sample by using the K nearest neighbor algorithm; A skeleton point is randomly selected from the K skeleton points closest to each of the skeleton points to generate a new skeleton point, and a skeleton point sample containing corruption information is obtained.

6. A point cloud classification method, characterized in that: include: The acquired original point cloud data is transformed to obtain skeleton point samples containing corruption information; wherein the original point cloud data is transformed by using the point cloud data transformation method according to any one of claims 1 to 5 to obtain skeleton point samples containing corruption information; Inputting the skeleton point samples containing corruption information into a point cloud classification network for training, and obtaining a trained point cloud classification network; The collected point cloud data is input into the trained point cloud classification network for classification.

7. A point cloud data conversion device, characterized in that: include: A sampling and feature extraction unit, used for sampling and extracting features from the original point cloud samples through an encoder to obtain sampled point cloud samples; A skeleton extraction unit is used to extract a target number of skeleton points from the sampled point cloud samples, obtain a radius value corresponding to each skeleton point by calculating the shortest distance from each point cloud in the sampled point cloud samples to each skeleton point, and generate a skeleton ball; A loss calculation unit, used to calculate the surface point coordinates of each skeleton point on the skeleton ball according to the coordinates of each skeleton point and the corresponding radius value, and to calculate the loss value according to the coordinates of each point cloud in the original point cloud sample, the coordinates of each skeleton point and the surface point coordinates of each skeleton point on the skeleton ball; An optimization unit, used for optimizing the positions of the skeleton points by using the loss values ​​to obtain final skeleton point samples; The interpolation unit is used to perform local linear interpolation on the skeleton point samples to obtain skeleton point samples containing corruption information.

8. A point cloud classification device, characterized in that: include: A transformation unit, used to transform the acquired original point cloud data to obtain skeleton point samples containing corruption information; wherein the original point cloud data adopts the point cloud data transformation method according to any one of claims 1 to 5 to obtain skeleton point samples containing corruption information; A training unit, used for inputting the skeleton point samples containing corruption information into a point cloud classification network for training, and obtaining a trained point cloud classification network; The classification unit is used to input the collected point cloud data into the trained point cloud classification network for classification.

9. An electronic device, characterized in that: The device comprises a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the point cloud data transformation method described in any one of claims 1 to 5, or execute the point cloud classification method described in claim 6 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and when the program code is executed by a processor, it implements the point cloud data transformation method described in any one of claims 1 to 5, or implements the point cloud classification method described in claim 6.

Images