Point cloud data denoising method, device and medium based on unsupervised network

Abstract

The application discloses a point cloud data denoising method and device based on an unsupervised network, and a medium. The method comprises the following steps: inputting synthetic noise data into a feature coding module constructed by an MAE model and a dynamic Adapter module to obtain feature coding; constructing an initial model by an MLP based on the feature coding and non-local features of the synthetic noise data; iteratively training and verifying the initial model based on a target training set and a target verification set to obtain a supervised point cloud displacement prediction model; supplementarily training the supervised point cloud displacement prediction model by a recurrent unsupervised framework to obtain a target point cloud displacement prediction model; inputting to-be-denoised point cloud into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value; and iteratively denoising the point cloud displacement prediction value by a recurrent denoising module to obtain a target denoising result. The application supplementarily trains the supervised point cloud displacement prediction model to obtain a final unsupervised model used for denoising, and fully considers the noise distribution of the point cloud in the real world in the denoising process, thereby improving the point cloud denoising effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of point cloud processing, and particularly relates to a point cloud data denoising method and device based on an unsupervised network. BACKGROUND

[0002] Point cloud data has important applications in modern computer vision and graphics, and is widely used in fields such as three-dimensional reconstruction, autonomous driving, robot navigation, and cultural heritage protection. However, in actual applications, due to the precision limitations of sensors, environmental interference, and other reasons, the collected point cloud data often contains noise. These noises not only affect the quality of the point cloud data, but also cause error accumulation in subsequent processing. Traditional point cloud denoising methods include filtering-based methods and optimization-based methods. The filtering-based methods mainly use the idea of image, considering noise as high-frequency components, and performing filtering operations based on the positions or normal vectors of points. However, filtering methods may lose some detailed information while denoising, especially when dealing with complex geometric structures, which affects the quality of point cloud data. Optimization-based methods convert the point cloud denoising process into an optimization problem, and complete denoising under a series of constraint conditions defined by prior knowledge of geometry and noise distribution. However, such methods usually involve multiple parameters and require careful experiments and adjustments to achieve good results on complex models, resulting in low efficiency of point cloud denoising. Therefore, a point cloud denoising method based on deep learning is proposed, which optimizes network parameters through a large amount of data training, so that it can correctly encode point cloud features and restore the correct shape. The deep learning network corresponding to the point cloud denoising method based on deep learning is trained based on various manual noise to generate paired data sets, such as Gaussian noise, root mean square noise, and Laplace noise. However, the noise distribution in the real world is not only one kind of noise, and the existing deep learning network cannot accurately express the noise distribution in the real world, resulting in poor denoising effect of point cloud data denoising results. SUMMARY

[0003] The embodiments of the present application provide a point cloud data denoising method, device and medium based on an unsupervised network, which can effectively improve the point cloud denoising effect.

[0004] In a first aspect, the embodiments of the present application provide a point cloud data denoising method based on an unsupervised network, comprising:

[0005] The data obtained by laser radar scanning and the preset synthetic noise data are used to construct a point cloud data set, and the point cloud data set is divided to obtain a target training set and a target validation set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data.

[0006] constructing a feature encoding module based on the MAE model and a dynamic Adapter module, and inputting the synthetic noise data into the feature encoding module to obtain feature encoding;

[0007] performing feature extraction on the synthetic noise data based on a KNN algorithm, and inputting the extracted features into the feature encoding module to obtain non-local features corresponding to the synthetic noise data;

[0008] constructing a displacement prediction module by encoding the feature encoding and the non-local features through a multi-layer perception (MLP), and determining the displacement prediction module as an initial model;

[0009] performing iterative training and verification on the initial model according to the target training set and the target verification set to obtain a trained supervised point cloud displacement prediction model;

[0010] performing supplementary training on the supervised point cloud displacement prediction model through a recurrent unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch, and a third branch, the first branch is used to predict clean point cloud displacement values of point cloud data, the second branch is used to predict related noise displacement values of point cloud data, and the third branch is used to predict irrelevant noise displacement values of point cloud data;

[0011] inputting laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain point cloud displacement prediction values, wherein the point cloud displacement prediction values include first prediction values output by the first branch, second prediction values output by the second branch, and third prediction values output by the third branch;

[0012] performing iterative denoising on the point cloud displacement prediction values based on a recurrent denoising module to obtain a target denoising result.

[0013] In some embodiments, the feature encoding module is constructed based on the MAE model and the dynamic Adapter module, including:

[0014] loading pre-trained parameters of a preset scannet dataset into the MAE model, wherein the scannet dataset is a public dataset used to train network parameters;

[0015] combining the loaded MAE model with the dynamic Adapter module to construct the feature encoding module.

[0016] In some embodiments, the feature extraction is performed on the synthetic noise data based on the KNN algorithm, and the extracted features are input into the feature encoding module to obtain non-local features corresponding to the synthetic noise data, including:

[0017] Each of the reference point cloud data of the synthetic noise data is subjected to feature extraction by a KNN algorithm to obtain a sub-region corresponding to each of the reference point cloud data;

[0018] The point cloud data of each of the sub-regions is input into the feature encoding module to obtain each of the non-local features.

[0019] In some embodiments, the point cloud data of each of the sub-regions is input into the feature encoding module to obtain each of the non-local features, which are obtained according to the following formula:

[0020] s i-1 =s-s i ;

[0021] wherein s is a center feature point of each of the sub-regions, s i is each of the reference point cloud data, and s i-1 is the non-local feature corresponding to each of the reference point cloud data.

[0022] In some embodiments, the supervised point cloud displacement prediction model is supplemented with training by a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, including:

[0023] The supervised point cloud displacement prediction model is expanded in width to construct the first branch, the second branch and the second branch;

[0024] A first loss function and a second loss function corresponding to the cyclic recursive unsupervised framework are determined;

[0025] The first branch is supervised trained by using the first loss function, and the second branch and the third branch are unsupervised trained by using the second loss function to obtain the target point cloud displacement prediction model.

[0026] In some embodiments, the expression of the first loss function is:

[0027]

[0028] wherein L i is the first loss function, P clean is a first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is a first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is a first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q cleanQ is a second clean point cloud displacement value output by the supervised point cloud displacement prediction model for inputting the first clean point cloud displacement value again. n Q is a second relevant noise displacement value output by the supervised point cloud displacement prediction model for inputting the first relevant noise displacement value again. nc Q is a second irrelevant noise displacement value output by the supervised point cloud displacement prediction model for inputting the first irrelevant noise displacement value again, and θ is a network parameter of the supervised point cloud displacement prediction model.

[0029] In some embodiments, an expression of the second loss function is as follows:

[0030]

[0031] wherein L c is the second loss function, and S is noisy point cloud data input into the supervised point cloud displacement prediction model.

[0032] In some embodiments, the point cloud displacement prediction value is iteratively denoised based on a cycle denoising module to obtain a target denoising result, and the target denoising result is obtained according to the following formula:

[0033]

[0034] wherein S i is the laser radar point cloud to be denoised, S i+1 is point cloud output after S i is input into a target point cloud displacement prediction model for the first time, λ is a step length of each time of denoising, is a calculated average offset, o(S) is a predicted displacement value output by the displacement prediction module, and N(S, S i ) is n closest point clouds to S i .

[0035] In a second aspect, the embodiments of the present application provide an electronic device, including at least one control processor and a memory connected in communication with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the point cloud data denoising method based on an unsupervised network as described in the first aspect.

[0036] In a third aspect, the embodiments of the present application further provide a computer readable storage medium storing computer executable instructions, and the computer executable instructions are used to execute the point cloud data denoising method based on an unsupervised network as described in the first aspect.

[0037] This application provides a point cloud data denoising method, device, and medium based on unsupervised networks. The method includes: constructing a point cloud dataset using data obtained from LiDAR scanning and preset synthetic noise data, and dividing the point cloud dataset into a target training set and a target validation set, wherein the synthetic noise data includes multiple noisy reference point cloud data; constructing a feature encoding module based on a MAE model and a dynamic adapter module, and inputting the synthetic noise data into the feature encoding module to obtain feature encoding; extracting features from the synthetic noise data based on the KNN algorithm, and inputting the extracted features into the feature encoding module to obtain non-local features corresponding to the synthetic noise data; constructing a displacement prediction module by encoding the feature encoding and the non-local features using a multilayer perceptron (MLP), and determining the displacement prediction module as the initial model; and performing point cloud data denoising based on the target training set and the target validation set. An initial model is iteratively trained and validated to obtain a trained supervised point cloud displacement prediction model. The supervised point cloud displacement prediction model is then supplemented with training using a recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training. This target point cloud displacement prediction model includes a first branch, a second branch, and a third branch. The first branch predicts noise-free displacement values ​​of the point cloud data, the second branch predicts relevant noise displacement values, and the third branch predicts irrelevant noise displacement values. The LiDAR point cloud to be denoised is input into the target point cloud displacement prediction model to obtain predicted point cloud displacement values. These predicted values ​​include a first predicted value output by the first branch, a second predicted value output by the second branch, and a third predicted value output by the third branch. The predicted point cloud displacement values ​​are iteratively denoised using a recursive denoising module to obtain the target denoising result. According to the scheme provided in this application, supplementing the supervised point cloud displacement prediction model with an unsupervised framework yields a final unsupervised model for denoising. This denoising process fully considers the noise distribution of the point cloud in the real world, thereby improving the point cloud denoising effect. Attached Figure Description

[0038] Figure 1 This is a flowchart of the steps of a point cloud data denoising method based on unsupervised networks provided in one embodiment of this application;

[0039] Figure 2 This is a structural diagram of an electronic device provided in another embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] It is to be understood that, while the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the specification, claims, or above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0042] Point cloud data has important applications in modern computer vision and graphics, and is widely used in fields such as three-dimensional reconstruction, autonomous driving, robot navigation, and cultural heritage protection. However, in practical applications, due to the precision limitations of sensors, environmental interference, and other reasons, the collected point cloud data often contains noise. These noises not only affect the quality of the point cloud data, but also cause error accumulation in the subsequent processing process. Traditional point cloud denoising methods include filtering-based methods and optimization-based methods. The filtering-based methods mainly use the idea of image, considering noise as high-frequency components, and using point positions or point normals for filtering operations. However, the filtering method may lose some detailed information while denoising, especially when dealing with complex geometric structures, affecting the quality of the point cloud data. The optimization-based methods convert the point cloud denoising process into an optimization problem, and complete denoising under a series of constraint conditions defined by prior knowledge of geometry shape and noise distribution. However, this method usually involves multiple parameters and needs careful testing and adjustment to achieve good results on complex models, and the efficiency of point cloud denoising is low. Based on this, existing point cloud denoising methods based on deep learning are proposed, which optimize network parameters through a large amount of data training, so that they can correctly encode point cloud features and restore their correct shape. However, the deep learning method needs a large amount of data for parameter training, although the point cloud denoising network can generate paired data sets through various manual noise, such as Gaussian noise, root mean square noise, and Laplace noise. However, the noise in the real world may not exist in only one noise distribution, and it may be a variety of noise or even a distribution that cannot be expressed by a mathematical model. Therefore, the denoising effect of real-world noise is the key to measuring the model. To solve this problem, existing methods use unsupervised training, but the existing unsupervised training method is based on the principle of image denoising. Images are structured and ordered data, while point clouds are scattered and unordered structures, resulting in poor denoising effect of existing point cloud denoising methods based on unsupervised training.

[0043] To solve the above problems, the embodiment of the present application provides a point cloud data denoising method, device and medium based on an unsupervised network, which comprises the following steps: constructing a point cloud dataset by using data scanned by a laser radar and preset synthetic noise data, and dividing the point cloud dataset to obtain a target training set and a target validation set, wherein the synthetic noise data comprises a plurality of noisy reference point cloud data; constructing a feature encoding module based on an MAE model and a dynamic Adapter module, inputting the synthetic noise data into the feature encoding module to obtain feature encoding; extracting features of the synthetic noise data based on a KNN algorithm, and inputting the extracted features into the feature encoding module to obtain non-local features corresponding to the synthetic noise data; constructing a displacement prediction module by encoding the feature encoding and the non-local features through a multi-layer perception (MLP), and determining the displacement prediction module as an initial model; iteratively training and verifying the initial model according to the target training set and the target validation set to obtain a trained supervised point cloud displacement prediction model; supplementally training the supervised point cloud displacement prediction model through a recurrent unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model comprises a first branch, a second branch and a third branch, the first branch is used for predicting noise-free displacement values of point cloud data, the second branch is used for predicting relevant noise displacement values of point cloud data, and the third branch is used for predicting irrelevant noise displacement values of point cloud data; inputting laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain point cloud displacement prediction values, wherein the point cloud displacement prediction values comprise first prediction values output by the first branch, second prediction values output by the second branch and third prediction values output by the third branch; iteratively denoising the point cloud displacement prediction values based on a recurrent denoising module to obtain a target denoising result. According to the scheme provided by the embodiment of the present application, the supervised point cloud displacement prediction model is supplementally trained by using the unsupervised framework to obtain an unsupervised model finally used for denoising, and the noise distribution of point cloud in the real world can be fully considered in the denoising process, thereby improving the point cloud denoising effect.

[0044] The embodiments of the present application are further described below with reference to the accompanying drawings.

[0045] Reference Figure 1 , Figure 1 is a step flowchart of a point cloud data denoising method based on an unsupervised network provided by an embodiment of the present application. The embodiment of the present application provides a point cloud data denoising method based on an unsupervised network, which comprises but is not limited to the following steps:

[0046] Step S10, constructing a point cloud dataset by the data obtained by the laser radar scanning and the preset synthetic noise data, and dividing the point cloud dataset to obtain a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data.

[0047] Specifically, the data obtained by the laser radar scanning can truly reflect the point cloud noise of the real world, and the synthetic noise data includes noise data simulating various noise types and intensities, thereby supplementing the missing noise conditions in the real data (i.e., the data obtained by the laser radar scanning). It can be understood that the data obtained by the laser radar scanning and the preset synthetic noise data are used to construct an initial model for subsequent training, thereby providing an effective data basis for improving the generalization ability, robustness and training efficiency of the model.

[0048] Step S20, constructing a feature encoding module based on the MAE model and the dynamic Adapter module, and inputting the synthetic noise data into the feature encoding module to obtain feature encoding.

[0049] Specifically, in some embodiments, step S20 includes but is not limited to the following steps:

[0050] Step S21, loading the pre-training parameters of the preset scannet dataset into the MAE model, wherein the scannet dataset is a public dataset for training network parameters;

[0051] Step S22, combining the loaded MAE model with the dynamic Adapter module to construct a feature encoding module.

[0052] Specifically, after migrating the pre-training parameters of the scannet dataset to the MAE model, the upstream MAE model parameters can be frozen, and the dynamic Adapter module is connected to the downstream task. The constructed feature encoding module can reduce the resource consumption of the computer, improve the training speed and enhance the feature extraction capability.

[0053] It can be understood that the feature encoding module of the present embodiment is used for feature extraction of noisy point cloud data. The feature encoding module migrates the pre-training parameters of the scannet dataset to the MAE model, and constructs the feature encoding module by combining the MAE model after migrating the pre-training parameters with the dynamic Adapter module. It not only inherits the advantages of the MAE model in self-supervised learning and efficiency, but also utilizes the characteristics of the Adapter module in interface adaptation and flexibility, thereby significantly enhancing the feature extraction capability and providing effective support for the accuracy of the subsequent target denoising result.

[0054] Step S30, feature extraction is performed on the synthetic noise data based on the KNN algorithm, and the extracted features are input into a feature encoding module to obtain non-local features corresponding to the synthetic noise data;

[0055] Specifically, in some embodiments, step S30 includes but is not limited to the following steps:

[0056] Step S31, feature extraction is performed on each reference point cloud data of the synthetic noise data by the KNN algorithm to obtain a sub-region corresponding to each reference point cloud data.

[0057] Step S32, the point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature.

[0058] Specifically, the sub-region of the present embodiment is a local patch obtained by performing feature extraction on each reference point cloud data by the KNN algorithm.

[0059] It should be noted that in step S32 of the present embodiment, the point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature, which is obtained according to the following formula:

[0060] s i-1 =s-s i ;

[0061] Wherein, s is the center feature point of each sub-region, s i is each reference point cloud data, and s i-1 is the non-local feature corresponding to each reference point cloud data.

[0062] It can be understood that the present embodiment obtains the non-local features of each reference point cloud data in the synthetic noise data, which can provide an effective data basis for constructing an initial model.

[0063] Step S40, a displacement prediction module is constructed by encoding the feature encoding and the non-local features by a multi-layer perception MLP, and the displacement prediction module is determined as the initial model.

[0064] It should be noted that the step of constructing the displacement prediction module according to the present embodiment is obtained according to the following formula:

[0065] o(S)=MLP(T);

[0066] Wherein, o(S) is the displacement prediction module, S is the point cloud data with noise, and T is a feature vector obtained by connecting the feature encoding and the non-local features end to end.

[0067] It should be noted that after the feature encoding and non-local feature corresponding to the synthetic noise data obtained through the above embodiment steps, a displacement prediction module is constructed in combination with a multi-layer perception MLP, that is, an initial model is constructed, and the initial model constructed includes an MAE encoder and an MLP connected in parallel, efficient extraction of input data features is realized, so as to obtain an accurate feature vector, wherein the MLP part is composed of multiple fully connected layers, used for accurate displacement mapping of the feature vector in the latent space of the input point cloud data, so as to accurately predict the displacement of the point cloud.

[0068] Specifically, the feature vector obtained by connecting the feature encoding and non-local feature at the beginning and the end is input to the multi-layer perception MLP for encoding to predict the noise offset of each reference point cloud data and generate a 3-dimensional offset vector (including clean point cloud displacement value, irrelevant noise displacement value and relevant noise displacement value) for adjusting the position of the point cloud data.

[0069] Step S50, iteratively training and verifying the initial model according to the target training set and the target verification set to obtain a trained supervised point cloud displacement prediction model.

[0070] It can be understood that iteratively training and verifying the initial model according to the target training set and the target verification set to obtain a trained supervised point cloud displacement prediction model can provide effective support for capturing detailed feature information in the input laser radar point cloud to be denoised and improving the accuracy of the model.

[0071] Specifically, the expression of the supervised point cloud displacement prediction model of the embodiment is as follows:

[0072] P clean, P nc ,P n =Q(θ,S);

[0073] Wherein, P clean is the first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is the first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is the first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, and Q(θ,S) is the supervised point cloud displacement prediction model, θ is the network parameter of the supervised point cloud displacement prediction model, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

[0074] Step S60, the supervised point cloud displacement prediction model is supplemented and trained through the cyclic recursive unsupervised framework, and a target point cloud displacement prediction model based on unsupervised training is obtained, wherein the target point cloud displacement prediction model comprises a first branch, a second branch and a third branch, the first branch is used for predicting a clean point cloud displacement value of point cloud data, the second branch is used for predicting a relevant noise displacement value of the point cloud data, and the third branch is used for predicting an irrelevant noise displacement value of the point cloud data.

[0075] Specifically, in some embodiments, step S60 includes but is not limited to the following steps:

[0076] Step S61, the supervised point cloud displacement prediction model is expanded in width to construct the first branch, the second branch and the second branch;

[0077] Step S62, a first loss function and a second loss function corresponding to the cyclic recursive unsupervised framework are determined

[0078] Step S63, the first branch is supervised trained by using the first loss function, and the second branch and the third branch are unsupervised trained by using the second loss function, so as to obtain the target point cloud displacement prediction model.

[0079] It should be noted that the expression of the first loss function of the embodiment is:

[0080]

[0081] Wherein, L i is the first loss function, P clean is a first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is a first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is a first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q clean is a second clean point cloud displacement value output by the supervised point cloud displacement prediction model by inputting the first clean point cloud displacement value again, Q n is a second relevant noise displacement value output by the supervised point cloud displacement prediction model by inputting the first relevant noise displacement value again, Q nc is a second irrelevant noise displacement value output by the supervised point cloud displacement prediction model by inputting the first irrelevant noise displacement value again, and θ is a network parameter of the supervised point cloud displacement prediction model.

[0082] It can be understood that, with reference to the description of the above embodiment, the first loss function L iThe first and second terms can ensure that the predicted displacement value output by the supervised point cloud displacement prediction model is the same as the branch corresponding to the predicted displacement value output by the next supervised point cloud displacement prediction model; the third and fourth terms can ensure that the output result of the clean branch (i.e., the first branch) of the supervised point cloud displacement prediction model after the relevant noise and irrelevant noise are input again should be the same as the clean point cloud displacement predicted by the first branch in the previous time. The first loss function and the second loss function are constructed by vertically regarding the process of training the target point cloud displacement prediction model as a cycle, which significantly solves the problem that the model can be trained only from the real noise data without paired (clean data-noise data).

[0083] It should be noted that the expression of the second loss function of the embodiment is:

[0084]

[0085] Wherein, L c is the second loss function, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

[0086] It can be understood that the supervised pre-training of the initial model and the unsupervised training of the supervised point cloud displacement prediction model supplemented by the first loss function and the second loss function can avoid learning an identity mapping directly by unsupervised training, improve the feature capturing ability of the model to real-world noise, and further improve the accuracy of the target denoising result.

[0087] Step S70, inputting the laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch, and a third prediction value output by the third branch.

[0088] Step S80, iteratively denoising the point cloud displacement prediction value based on the cyclic denoising module to obtain a target denoising result.

[0089] It should be noted that the step of the embodiment iteratively denoises the point cloud displacement prediction value based on the cyclic denoising module to obtain a target denoising result, which is obtained according to the following formula:

[0090]

[0091] Wherein, S i is the laser radar point cloud to be denoised, S i+1 is the point cloud output after S i is input into the target point cloud displacement prediction model for the first time, and λ is the step length of each denoising, For the average offset of the calculation, o(S) is the predicted displacement value output by the displacement prediction module, N(S, S i ) is the distance S i The nearest n point clouds.

[0092] It can be understood that the essence of the iterative denoising of the point cloud displacement prediction value of the loop denoising module in the embodiment is to take the previous denoising result as the noise point cloud for the next denoising and iteratively denoise again, which can further improve the accuracy of the denoising result and improve the denoising effect of the target denoising result.

[0093] As Figure 2 shown, Figure 2 is a structural diagram of an electronic device provided by an embodiment of the present application. The present application also provides an electronic device 200, comprising:

[0094] The processor 210 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0095] The memory 220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 220 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 220 and are called and executed by the processor 210 to implement the point cloud data denoising method based on an unsupervised network.

[0096] The input / output interface 230 is used to realize information input and output.

[0097] The communication interface 240 is used to realize the communication interaction between the device and other devices. The communication can be realized by a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0098] The bus 250 transmits information between various components (such as the processor 210, the memory 220, the input / output interface 230, and the communication interface 240) of the device.

[0099] The processor 210, the memory 220, the input / output interface 230, and the communication interface 240 are communicatively connected with each other inside the device through the bus 250.

[0100] In addition, the embodiment of the present application further provides a storage medium, which is a computer readable storage medium, and stores a computer program. The computer program is executed by a processor to implement the point cloud data denoising method based on an unsupervised network.

[0101] The memory is a non-transitory computer readable storage medium, and can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and the remote memory can be connected to the processor through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely illustrative, and units described as separate components can or can not be physically separated, can be located in one place, or can also be distributed to multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment.

[0102] As will be appreciated by one of ordinary skill in the art, all or some of the steps, systems, etc. in the above-disclosed methods can be embodied in software, firmware, hardware, and / or suitable combinations thereof. Some or all of the physical components can be implemented with software executed by a processor, such as a central processing unit, a digital signal processor, or microprocessor, or can be implemented with hardware, or can be implemented with an integrated circuit, such as an application- specific integrated circuit. Such software can be distributed on computer readable media, which can comprise computer storage media (or non-transitory media), and communication media (or transitory media). As is well known to those of ordinary skill in the art, computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.

[0103] The above description is that of the preferred embodiments of the application. Various modifications and changes can be made thereto without departing from the spirit and scope of the application as set forth in the claims.

Claims

1. A method for point cloud data denoising based on an unsupervised network, characterized in that, The method comprises the following steps: Data obtained by laser radar scanning and preset synthetic noise data are used to construct a point cloud data set, and the point cloud data set is divided to obtain a target training set and a target validation set, wherein the synthetic noise data comprises a plurality of noisy reference point cloud data; A feature encoding module is constructed based on an MAE model and a dynamic Adapter module, and the synthetic noise data is input into the feature encoding module to obtain feature encoding; Feature extraction is performed on the synthetic noise data based on a KNN algorithm, and the extracted features are input into the feature encoding module to obtain non-local features corresponding to the synthetic noise data; A displacement prediction module is constructed by encoding the feature encoding and the non-local features through a multi-layer perception (MLP), and the displacement prediction module is determined as an initial model; The initial model is iteratively trained and validated according to the target training set and the target validation set to obtain a trained supervised point cloud displacement prediction model; The supervised point cloud displacement prediction model is supplemented trained through a recurrent unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model comprises a first branch, a second branch and a third branch, the first branch is used to predict a clean point cloud displacement value of point cloud data, the second branch is used to predict a related noise displacement value of point cloud data, and the third branch is used to predict an irrelevant noise displacement value of point cloud data; A laser radar point cloud to be denoised is input into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value comprises a first prediction value output by the first branch, a second prediction value output by the second branch and a third prediction value output by the third branch; The point cloud displacement prediction value is iteratively denoised based on a recurrent denoising module to obtain a target denoising result.

2. The unsupervised network-based point cloud data denoising method according to claim 1, characterized in that, The feature encoding module is constructed based on an MAE model and a dynamic Adapter module, comprising: Pretrained parameters of a preset scannet data set are loaded into the MAE model, wherein the scannet data set is a public data set used to train network parameters; The loaded MAE model is combined with the dynamic Adapter module to construct the feature encoding module.

3. The unsupervised network-based point cloud data denoising method according to claim 1, wherein, The feature extraction is performed on the synthetic noise data based on the KNN algorithm, and the extracted features are input into the feature encoding module to obtain the non-local features corresponding to the synthetic noise data, comprising: Feature extraction is performed on each reference point cloud data of the synthetic noise data through the KNN algorithm to obtain a sub-region corresponding to each reference point cloud data; The point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature.

4. The point cloud data denoising method based on an unsupervised network according to claim 3, characterized in that, The point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature, which is obtained according to the following formula: s i-1 = s - s i ; wherein s is a center feature point of each of the sub-regions, s i is each of the reference point cloud data, s i-1 is the non-local feature corresponding to each of the reference point cloud data.

5. The unsupervised network-based point cloud data denoising method according to claim 1, wherein, The supervised point cloud displacement prediction model is supplemented trained through a recurrent unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, comprising: The supervised point cloud displacement prediction model is expanded in width to construct the first branch, the second branch and the second branch; determining a first loss function and a second loss function corresponding to the cyclic recursive unsupervised framework; supervised training the first branch using the first loss function and unsupervised training the second branch and the third branch using the second loss function to obtain the target point cloud displacement prediction model.

6. The unsupervised network-based point cloud data denoising method according to claim 5, characterized in that, The expression of the first loss function is: wherein, L i is the first loss function, P clean is a first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is a first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is a first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q clean is a second clean point cloud displacement value output by the supervised point cloud displacement prediction model by inputting the first clean point cloud displacement value again, Q n is a second relevant noise displacement value output by the supervised point cloud displacement prediction model by inputting the first relevant noise displacement value again, Q nc is a second irrelevant noise displacement value output by the supervised point cloud displacement prediction model by inputting the first irrelevant noise displacement value again, and θ is a network parameter of the supervised point cloud displacement prediction model.

7. The unsupervised network-based point cloud data denoising method according to claim 6, characterized in that, The expression of the second loss function is: wherein L c is the second loss function, S is the noisy point cloud data input to the supervised point cloud displacement prediction model.

8. The unsupervised network-based point cloud data denoising method according to claim 1, characterized in that, The point cloud displacement prediction value is iteratively denoised based on the cyclic denoising module to obtain a target denoising result, and the following formula is used to obtain: Among them, S i For the lidar point cloud to be denoised, S i+1 To make S i The point cloud output after the first denoising step is performed on the target point cloud displacement prediction model, where λ is the step size for each denoising step. Let O(S) be the average offset calculated, and let O(S) be the predicted displacement value output by the displacement prediction module. Then, N(S,S) = 1 / 2 * ... i () represents the distance S i The most recent n point clouds.

9. An electronic device, comprising: The memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to perform the point cloud data denoising method based on an unsupervised network as claimed in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to execute the point cloud data denoising method based on an unsupervised network as claimed in any one of claims 1 to 8.

Images