Point cloud data denoising method and device based on unsupervised network, and medium

Abstract

The invention discloses a point cloud data denoising method and device based on an unsupervised network, and a medium. The method comprises the following steps: inputting synthetic noise data into a feature coding module constructed by an MAE model and a dynamic Adapter module to obtain a feature code; constructing an initial model for feature codes and non-local features of the synthesized noise data through MLP; iteratively training and verifying the initial model according to the target training set and the target verification set to obtain a supervised point cloud displacement prediction model, and performing supplementary training on the supervised point cloud displacement prediction model through a cyclic recursion unsupervised framework to obtain a target point cloud displacement prediction model; inputting the to-be-denoised point cloud into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value; and iteratively de-noising the point cloud displacement predicted value by using a cyclic de-noising module to obtain a target de-noising result. According to the method, the supervised point cloud displacement prediction model is complementarily trained to obtain a final unsupervised model for denoising, the noise distribution of the point cloud in the real world is fully considered in the denoising process, and the point cloud denoising effect is improved.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of point cloud processing technology, and in particular to a point cloud data denoising method, device, and medium based on an unsupervised network. Background Art

[0002] Point cloud data has important applications in modern computer vision and graphics, and is widely used in 3D reconstruction, autonomous driving, robot navigation, cultural heritage protection and other fields. However, in practical applications, due to the accuracy limitations of sensors, environmental interference and other reasons, the collected point cloud data often contains noise. These noises not only affect the quality of point cloud data, but also lead to error accumulation in subsequent processing. Traditional point cloud denoising methods include filtering-based methods and optimization-based methods. Among them, the filtering-based method mainly uses the idea of ​​image, regards noise as a high-frequency component, and performs filtering operations with the position or normal of the point. However, the filtering method may lose some detail information while denoising, especially when dealing with complex geometric structures, which affects the quality of point cloud data. The optimization-based method is based on the principle of converting the point cloud denoising process into an optimization problem, and completing the denoising under a series of constraints defined by the prior knowledge of full geometric shape and noise distribution. However, this type of method usually involves multiple parameters and requires careful experiments and adjustments to achieve good results on complex models. The efficiency of point cloud denoising is low. Based on this, a point cloud denoising method based on deep learning has been proposed. It optimizes the network parameters through a large amount of data training so that it can correctly encode the point cloud features and restore its correct shape. The deep learning network corresponding to the point cloud denoising method based on deep learning is trained based on various manual noises to generate paired data sets, such as Gaussian noise, root mean square noise and Laplace noise, but the point cloud noise in the real world does not have only one noise distribution. The existing deep learning network cannot accurately express the noise distribution in the real world, resulting in poor denoising effect of the point cloud data denoising results. Summary of the invention

[0003] The embodiments of the present application provide a point cloud data denoising method, device, and medium based on an unsupervised network, which can effectively improve the point cloud denoising effect.

[0004] In a first aspect, an embodiment of the present application provides a point cloud data denoising method based on an unsupervised network, comprising:

[0005] A point cloud data set is constructed by using data obtained by laser radar scanning and preset synthetic noise data, and the point cloud data set is divided into a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data;

[0006] Constructing a feature coding module based on the MAE model and the dynamic Adapter module, and inputting the synthetic noise data into the feature coding module to obtain feature coding;

[0007] Performing feature extraction on the synthetic noise data based on the KNN algorithm, and inputting the extracted features into the feature encoding module to obtain non-local features corresponding to the synthetic noise data;

[0008] Encoding the feature encoding and the non-local feature through a multi-layer perceptron MLP to construct a displacement prediction module, and determining the displacement prediction module as an initial model;

[0009] Iteratively training and validating the initial model according to the target training set and the target validation set to obtain a trained supervised point cloud displacement prediction model;

[0010] The supervised point cloud displacement prediction model is supplemented with training through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch and a third branch, the first branch is used to predict a clean point cloud displacement value of the point cloud data, the second branch is used to predict a related noise displacement value of the point cloud data, and the third branch is used to predict an irrelevant noise displacement value of the point cloud data;

[0011] Inputting the laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch, and a third prediction value output by the third branch;

[0012] The point cloud displacement prediction value is iteratively denoised based on a cyclic denoising module to obtain a target denoising result.

[0013] In some embodiments, a feature encoding module is constructed based on the MAE model and the dynamic Adapter module, including:

[0014] Loading the pre-trained parameters of the preset scannet data set into the MAE model, wherein the scannet data set is a public data set used to train network parameters;

[0015] The loaded MAE model is combined with the dynamic Adapter module to construct the feature encoding module.

[0016] In some embodiments, feature extraction is performed on the synthetic noise data based on the KNN algorithm, and the extracted features are input into the feature encoding module to obtain non-local features corresponding to the synthetic noise data, including:

[0017] Performing feature extraction on each of the reference point cloud data of the synthetic noise data by using a KNN algorithm to obtain a sub-region corresponding to each of the reference point cloud data;

[0018] The point cloud data of each of the sub-areas is input into the feature encoding module to obtain each of the non-local features.

[0019] In some embodiments, the point cloud data of each of the sub-areas is input into the feature encoding module to obtain each of the non-local features, which is obtained according to the following formula:

[0020] s i-1 =ss i ;

[0021] Among them, s is the central feature point of each sub-region, s i is each of the reference point cloud data, s i-1 The non-local features corresponding to each of the reference point cloud data.

[0022] In some embodiments, the supervised point cloud displacement prediction model is supplemented with training through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, including:

[0023] Expanding the supervised point cloud displacement prediction model in width to construct the first branch, the second branch and the second branch;

[0024] Determine a first loss function and a second loss function corresponding to the cyclic recursive unsupervised framework;

[0025] The first branch is supervisedly trained using the first loss function, and the second branch and the third branch are unsupervisedly trained using the second loss function to obtain the target point cloud displacement prediction model.

[0026] In some embodiments, the expression of the first loss function is:

[0027]

[0028] Among them, L i is the first loss function, P clean is the first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is the first correlated noise displacement value output by the supervised point cloud displacement prediction model, P n is the first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q cleanTo input the first clean point cloud displacement value again into the second clean point cloud displacement value output by the supervised point cloud displacement prediction model, Q n To input the first correlated noise displacement value again into the second correlated noise displacement value output by the supervised point cloud displacement prediction model, Q nc In order to input the first irrelevant noise displacement value again into the second irrelevant noise displacement value output by the supervised point cloud displacement prediction model, θ is a network parameter of the supervised point cloud displacement prediction model.

[0029] In some embodiments, the expression of the second loss function is:

[0030]

[0031] Among them, L c is the second loss function, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

[0032] In some embodiments, the point cloud displacement prediction value is iteratively denoised based on a cyclic denoising module to obtain a target denoising result, which is obtained according to the following formula:

[0033]

[0034] Among them, S i is the laser radar point cloud to be denoised, S i+1 To S i The point cloud output after the first denoising is input into the target point cloud displacement prediction model, where λ is the step length of each denoising. is the calculated average offset, o(S) is the predicted displacement value output by the displacement prediction module, N(S,S i ) is the distance S i The nearest n point clouds.

[0035] In a second aspect, an embodiment of the present application provides an electronic device comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the point cloud data denoising method based on an unsupervised network as described in the first aspect.

[0036] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the point cloud data denoising method based on an unsupervised network as described in the first aspect.

[0037] The embodiment of the present application provides a point cloud data denoising method, device, and medium based on an unsupervised network, the method comprising: constructing a point cloud data set through data obtained by laser radar scanning and preset synthetic noise data, and dividing the point cloud data set to obtain a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data; constructing a feature coding module based on a MAE model and a dynamic Adapter module, and inputting the synthetic noise data into the feature coding module to obtain feature coding; performing feature extraction on the synthetic noise data based on a KNN algorithm, and inputting the extracted features into the feature coding module to obtain non-local features corresponding to the synthetic noise data; encoding the feature coding and the non-local features through a multi-layer perceptron MLP to construct a displacement prediction module, and determining the displacement prediction module as an initial model; and The initial model is iteratively trained and verified to obtain a trained supervised point cloud displacement prediction model; the supervised point cloud displacement prediction model is supplemented with training through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch and a third branch, the first branch is used to predict the noise-free displacement value of the point cloud data, the second branch is used to predict the related noise displacement value of the point cloud data, and the third branch is used to predict the irrelevant noise displacement value of the point cloud data; the laser radar point cloud to be denoised is input into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch and a third prediction value output by the third branch; the point cloud displacement prediction value is iteratively denoised based on a cyclic denoising module to obtain a target denoising result. According to the solution provided in the embodiment of the present application, the supervised point cloud displacement prediction model is supplemented with training using an unsupervised framework to obtain an unsupervised model that is ultimately used for denoising, and the noise distribution of the point cloud in the real world can be fully considered during the denoising process, thereby improving the point cloud denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flowchart of the steps of a point cloud data denoising method based on an unsupervised network provided by an embodiment of the present application;

[0039] Figure 2 It is a structural diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] It is understood that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flow chart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0042] Point cloud data has important applications in modern computer vision and graphics, and is widely used in 3D reconstruction, autonomous driving, robot navigation, cultural heritage protection and other fields. However, in practical applications, due to the accuracy limitations of sensors, environmental interference and other reasons, the collected point cloud data often contains noise. These noises not only affect the quality of point cloud data, but also lead to error accumulation in subsequent processing. Traditional point cloud denoising methods include filtering-based methods and optimization-based methods. Among them, the filtering-based method mainly uses the idea of ​​image, regards noise as a high-frequency component, and performs filtering operations with the position or normal of the point. However, the filtering method may lose some detail information while denoising, especially when dealing with complex geometric structures, which affects the quality of point cloud data. The optimization-based method is based on the principle of converting the point cloud denoising process into an optimization problem, and completing the denoising under a series of constraints defined by the prior knowledge of full geometric shape and noise distribution. However, this type of method usually involves multiple parameters and requires careful experiments and adjustments to achieve good results on complex models. The efficiency of point cloud denoising is low. Based on this, a point cloud denoising method based on deep learning has been proposed. It optimizes the network parameters through a large amount of data training so that it can correctly encode the point cloud features and restore its correct shape. However, the deep learning method requires a large amount of data for parameter training. Although the point cloud denoising network can generate paired data sets through various manual noises, such as Gaussian noise, root mean square noise and Laplace noise, the noise in the real world does not have only one noise distribution. It may be a variety of noises or even a distribution that cannot be expressed by a mathematical model. Therefore, the denoising effect of the real world noise is the key to measuring the model. To address this problem, it is currently implemented through unsupervised training. However, the existing unsupervised training method is based on the principle of reference image denoising. The image is an ordered structured data, while the point cloud is characterized by a scattered disordered structure, which leads to the poor denoising effect of the existing point cloud denoising method based on unsupervised training.

[0043] To solve the above-mentioned problems, an embodiment of the present application provides a point cloud data denoising method, device, and medium based on an unsupervised network, the method comprising: constructing a point cloud data set through data obtained by laser radar scanning and preset synthetic noise data, and dividing the point cloud data set to obtain a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data; constructing a feature coding module based on a MAE model and a dynamic Adapter module, and inputting the synthetic noise data into the feature coding module to obtain feature coding; performing feature extraction on the synthetic noise data based on a KNN algorithm, and inputting the extracted features into the feature coding module to obtain non-local features corresponding to the synthetic noise data; encoding the feature coding and the non-local features through a multi-layer perceptron MLP to construct a displacement prediction module, and determining the displacement prediction module as an initial model; and determining the displacement prediction module as an initial model according to the target training set and the target The initial model is iteratively trained and verified by the validation set to obtain a trained supervised point cloud displacement prediction model; the supervised point cloud displacement prediction model is supplementarily trained through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch and a third branch, the first branch is used to predict the noise-free displacement value of the point cloud data, the second branch is used to predict the related noise displacement value of the point cloud data, and the third branch is used to predict the irrelevant noise displacement value of the point cloud data; the laser radar point cloud to be denoised is input into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch and a third prediction value output by the third branch; the point cloud displacement prediction value is iteratively denoised based on a cyclic denoising module to obtain a target denoising result. According to the solution provided in the embodiment of the present application, an unsupervised framework is used to perform supplementary training on the supervised point cloud displacement prediction model to obtain the unsupervised model ultimately used for denoising. During the denoising process, the noise distribution of the point cloud in the real world can be fully considered, thereby improving the point cloud denoising effect.

[0044] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0045] refer to Figure 1 , Figure 1 : is a flowchart of a method for denoising point cloud data based on an unsupervised network provided by an embodiment of the present application. The embodiment of the present application provides a method for denoising point cloud data based on an unsupervised network, and the method includes but is not limited to the following steps:

[0046] Step S10, constructing a point cloud data set through the data obtained by laser radar scanning and preset synthetic noise data, and dividing the point cloud data set to obtain a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data.

[0047] Specifically, the data obtained through lidar scanning in this embodiment can truly reflect the point cloud noise in the real world, and the synthetic noise data includes noise data that simulates various noise types and intensities, thereby supplementing the noise conditions that may be missing in the real data (i.e., the data obtained by lidar scanning). It can be understood that the data obtained by lidar scanning and the preset synthetic noise data are used to construct an initial model for training subsequent ones, providing an effective data basis for improving the generalization ability, robustness and training efficiency of the model.

[0048] Step S20, constructing a feature coding module based on the MAE model and the dynamic Adapter module, and inputting the synthetic noise data into the feature coding module to obtain feature coding.

[0049] Specifically, in some embodiments, step S20 includes but is not limited to the following steps:

[0050] Step S21, loading the pre-trained parameters of the preset scannet data set into the MAE model, wherein the scannet data set is a public data set used to train network parameters;

[0051] Step S22, combining the loaded MAE model with the dynamic Adapter module to construct a feature encoding module.

[0052] Specifically, after migrating the pre-trained parameters of the ScanNet dataset to the MAE model, the upstream MAE model parameters can be frozen. Combined with the dynamic Adapter module to connect downstream tasks, the constructed feature encoding module can reduce computer resource consumption, increase training speed and enhance feature extraction capabilities.

[0053] It can be understood that the feature encoding module of this embodiment is used to extract features from noisy point cloud data. The feature encoding module migrates the pre-trained parameters of the scannet dataset to the MAE model, and combines the MAE model after the pre-trained parameters are migrated with the dynamic Adapter module to construct the feature encoding module. It not only inherits the advantages of the MAE model in self-supervised learning and high efficiency, but also utilizes the characteristics of the Adapter module in interface adaptation and flexibility, thereby significantly enhancing the feature extraction capability and providing effective support for the accuracy of subsequent target denoising results.

[0054] Step S30, extracting features from the synthetic noise data based on the KNN algorithm, and inputting the extracted features into a feature encoding module to obtain non-local features corresponding to the synthetic noise data;

[0055] Specifically, in some embodiments, step S30 includes but is not limited to the following steps:

[0056] Step S31, performing feature extraction on each reference point cloud data of the synthetic noise data by using the KNN algorithm to obtain a sub-region corresponding to each reference point cloud data;

[0057] Step S32, inputting the point cloud data of each sub-region into a feature encoding module to obtain each non-local feature.

[0058] Specifically, the sub-regions in this embodiment are local patches obtained by extracting features from each reference point cloud data through the KNN algorithm.

[0059] It should be noted that in step S32 of this embodiment, the point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature, which is obtained according to the following formula:

[0060] s i-1 =ss i ;

[0061] Among them, s is the central feature point of each sub-region, s i is each reference point cloud data, s i-1 is the non-local feature corresponding to each reference point cloud data.

[0062] It can be understood that, in this embodiment, the non-local features of each reference point cloud data in the synthetic noise data are obtained, which can provide an effective data basis for constructing the initial model.

[0063] Step S40, encoding the feature encoding and the non-local feature through a multi-layer perceptron MLP to construct a displacement prediction module, and determining the displacement prediction module as an initial model.

[0064] It should be noted that the step of constructing the displacement prediction module in the embodiment of the present application is obtained according to the following formula:

[0065] o(S) = MLP(T);

[0066] Among them, o(S) is the displacement prediction module, S is the noisy point cloud data, and T is the feature vector obtained by connecting the feature encoding and non-local features end to end.

[0067] It should be noted that after the feature encoding and non-local features corresponding to the synthetic noise data are obtained through the steps of the above embodiment, a displacement prediction module is constructed in combination with a multi-layer perceptron MLP, that is, an initial model is constructed. The constructed initial model includes a MAE encoder and MLP connected in parallel, so as to achieve efficient extraction of input data features, thereby obtaining an accurate feature vector. Among them, the MLP part is composed of multiple fully connected layers, which is used to accurately perform displacement mapping on the feature vector in the latent space of the input point cloud data, thereby accurately predicting the displacement of the point cloud.

[0068] Specifically, the feature vector obtained by connecting the feature code and the non-local feature end to end is input into the multi-layer perceptron MLP for encoding, the noise offset of each reference point cloud data is predicted, and a three-dimensional offset vector (including clean point cloud displacement value, irrelevant noise displacement value and correlated noise displacement value) is generated for adjusting the position of the point cloud data.

[0069] Step S50, iteratively training and verifying the initial model according to the target training set and the target verification set to obtain a trained supervised point cloud displacement prediction model.

[0070] It can be understood that the initial model is iteratively trained and verified according to the target training set and the target validation set to obtain a trained supervised point cloud displacement prediction model, which can provide effective support for capturing the detailed feature information in the input lidar point cloud to be denoised and improving the accuracy of the model.

[0071] Specifically, the expression of the supervised point cloud displacement prediction model of this embodiment is as follows:

[0072] P clean, P nc ,P n =Q(θ,S);

[0073] Among them, P clean is the first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is the first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is the first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q(θ,S) is the supervised point cloud displacement prediction model, θ is the network parameter of the supervised point cloud displacement prediction model, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

[0074] Step S60, supplementing the training of the supervised point cloud displacement prediction model through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch and a third branch, the first branch is used to predict the clean point cloud displacement value of the point cloud data, the second branch is used to predict the related noise displacement value of the point cloud data, and the third branch is used to predict the irrelevant noise displacement value of the point cloud data.

[0075] Specifically, in some embodiments, step S60 includes but is not limited to the following steps:

[0076] Step S61, expanding the supervised point cloud displacement prediction model in terms of width to construct a first branch, a second branch, and a second branch;

[0077] Step S62, determining the first loss function and the second loss function corresponding to the cyclic recursive unsupervised framework

[0078] Step S63, using the first loss function to perform supervised training on the first branch, and using the second loss function to perform unsupervised training on the second branch and the third branch, to obtain a target point cloud displacement prediction model.

[0079] It should be noted that the expression of the first loss function in this embodiment is:

[0080]

[0081] Among them, L i is the first loss function, P clean is the first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is the first relevant noise displacement value output by the supervised point cloud displacement prediction model, P n is the first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q clean To input the first clean point cloud displacement value into the second clean point cloud displacement value output by the supervised point cloud displacement prediction model, Q n To input the first correlated noise displacement value into the second correlated noise displacement value output by the supervised point cloud displacement prediction model again, Q nc In order to input the first irrelevant noise displacement value again into the second irrelevant noise displacement value output by the supervised point cloud displacement prediction model, θ is the network parameter of the supervised point cloud displacement prediction model.

[0082] It can be understood that, referring to the description of the above embodiment, the first loss function L of this embodiment iThe invention consists of 4 formulas separated by plus signs. The first and second formulas can ensure that the predicted displacement value output by the supervised point cloud displacement prediction model is the same as the branch corresponding to the predicted displacement value output by the supervised point cloud displacement prediction model next time. The third and fourth formulas can ensure that the output result of the clean branch (i.e., the first branch) after the correlated noise and the irrelevant noise are input into the supervised point cloud displacement prediction model again should be equivalent to the clean point cloud displacement predicted by the first branch last time. In this embodiment, the process of training the target point cloud displacement prediction model is vertically regarded as a cycle to construct the first loss function and the second loss function, which significantly solves the problem that the model can be trained only from real noise data without paired (clean data-noise data).

[0083] It should be noted that the expression of the second loss function in this embodiment is:

[0084]

[0085] Among them, L c is the second loss function, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

[0086] It can be understood that supervised pre-training of the initial model and using the first loss function and the second loss function to supplement the unsupervised training of the supervised point cloud displacement prediction model can avoid learning the identity mapping through direct unsupervised training, improve the model's ability to capture the features of real-world noise, and thus improve the accuracy of the target denoising results.

[0087] Step S70, input the laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch, and a third prediction value output by the third branch.

[0088] Step S80, iteratively denoising the point cloud displacement prediction value based on the cyclic denoising module to obtain a target denoising result.

[0089] It should be noted that the steps of this embodiment are based on the cyclic denoising module to iteratively denoise the point cloud displacement prediction value to obtain the target denoising result, which is obtained according to the following formula:

[0090]

[0091] Among them, S i is the laser radar point cloud to be denoised, S i+1 To S i The point cloud output after the first denoising is input into the target point cloud displacement prediction model, where λ is the step length of each denoising. is the calculated average offset, o(S) is the predicted displacement value output by the displacement prediction module, N(S,S i ) is the distance S i The nearest n point clouds.

[0092] It can be understood that the essential idea of ​​the cyclic denoising module of this embodiment to iteratively denoise the point cloud displacement prediction value is to use the previous denoising result as the noise point cloud for the next denoising to iteratively denoise again, which can further improve the accuracy of the denoising result and improve the denoising effect of the target denoising result.

[0093] like Figure 2 As shown, Figure 2 2 is a structural diagram of an electronic device provided by an embodiment of the present application. The present invention also provides an electronic device 200, including:

[0094] The processor 210 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0095] The memory 220 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 220 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 220, and the processor 210 calls and executes the point cloud data denoising method based on an unsupervised network in the embodiment of this application;

[0096] Input / output interface 230, used to implement information input and output;

[0097] Communication interface 240, used to realize communication interaction between the apparatus and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0098] bus 250 , which transmits information between the various components of the device (e.g., processor 210 , memory 220 , input / output interface 230 , and communication interface 240 );

[0099] The processor 210 , the memory 220 , the input / output interface 230 , and the communication interface 240 are connected to each other in communication within the device via the bus 250 .

[0100] In addition, an embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the above-mentioned point cloud data denoising method based on an unsupervised network is implemented.

[0101] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0102] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0103] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A point cloud data denoising method based on an unsupervised network, characterized in that: include: A point cloud data set is constructed by using data obtained by laser radar scanning and preset synthetic noise data, and the point cloud data set is divided into a target training set and a target verification set, wherein the synthetic noise data includes a plurality of noisy reference point cloud data; Constructing a feature coding module based on the MAE model and the dynamic Adapter module, and inputting the synthetic noise data into the feature coding module to obtain feature coding; Performing feature extraction on the synthetic noise data based on the KNN algorithm, and inputting the extracted features into the feature encoding module to obtain non-local features corresponding to the synthetic noise data; Encoding the feature encoding and the non-local feature through a multi-layer perceptron MLP to construct a displacement prediction module, and determining the displacement prediction module as an initial model; Iteratively training and validating the initial model according to the target training set and the target validation set to obtain a trained supervised point cloud displacement prediction model; The supervised point cloud displacement prediction model is supplemented with training through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, wherein the target point cloud displacement prediction model includes a first branch, a second branch and a third branch, the first branch is used to predict a clean point cloud displacement value of the point cloud data, the second branch is used to predict a related noise displacement value of the point cloud data, and the third branch is used to predict an irrelevant noise displacement value of the point cloud data; Inputting the laser radar point cloud to be denoised into the target point cloud displacement prediction model to obtain a point cloud displacement prediction value, wherein the point cloud displacement prediction value includes a first prediction value output by the first branch, a second prediction value output by the second branch, and a third prediction value output by the third branch; The point cloud displacement prediction value is iteratively denoised based on a cyclic denoising module to obtain a target denoising result.

2. The point cloud data denoising method based on an unsupervised network according to claim 1, characterized in that: The feature encoding module is built based on the MAE model and the dynamic Adapter module, including: Loading the pre-trained parameters of the preset scannet data set into the MAE model, wherein the scannet data set is a public data set used to train network parameters; The loaded MAE model is combined with the dynamic Adapter module to construct the feature encoding module.

3. The point cloud data denoising method based on an unsupervised network according to claim 1, characterized in that: Based on the KNN algorithm, feature extraction is performed on the synthetic noise data, and the extracted features are input into the feature encoding module to obtain non-local features corresponding to the synthetic noise data, including: Performing feature extraction on each of the reference point cloud data of the synthetic noise data by using a KNN algorithm to obtain a sub-region corresponding to each of the reference point cloud data; The point cloud data of each of the sub-areas is input into the feature encoding module to obtain each of the non-local features.

4. The point cloud data denoising method based on an unsupervised network according to claim 3, characterized in that: The point cloud data of each sub-region is input into the feature encoding module to obtain each non-local feature, which is obtained according to the following formula: s i-1 =s-s i ; Among them, s is the central feature point of each sub-region, s i is each of the reference point cloud data, s i-1 The non-local features corresponding to each of the reference point cloud data.

5. The point cloud data denoising method based on an unsupervised network according to claim 1, characterized in that: The supervised point cloud displacement prediction model is supplemented with training through a cyclic recursive unsupervised framework to obtain a target point cloud displacement prediction model based on unsupervised training, including: Expanding the supervised point cloud displacement prediction model in width to construct the first branch, the second branch and the second branch; Determine a first loss function and a second loss function corresponding to the cyclic recursive unsupervised framework; The first branch is supervisedly trained using the first loss function, and the second branch and the third branch are unsupervisedly trained using the second loss function to obtain the target point cloud displacement prediction model.

6. The point cloud data denoising method based on an unsupervised network according to claim 5, characterized in that: The expression of the first loss function is: Among them, L i is the first loss function, P clean is the first clean point cloud displacement value output by the supervised point cloud displacement prediction model, P nc is the first correlated noise displacement value output by the supervised point cloud displacement prediction model, P n is the first irrelevant noise displacement value output by the supervised point cloud displacement prediction model, Q clean To input the first clean point cloud displacement value again into the second clean point cloud displacement value output by the supervised point cloud displacement prediction model, Q n To input the first correlated noise displacement value again into the second correlated noise displacement value output by the supervised point cloud displacement prediction model, Q nc In order to input the first irrelevant noise displacement value again into the second irrelevant noise displacement value output by the supervised point cloud displacement prediction model, θ is a network parameter of the supervised point cloud displacement prediction model.

7. The point cloud data denoising method based on an unsupervised network according to claim 6, characterized in that: The expression of the second loss function is: Among them, L c is the second loss function, and S is the noisy point cloud data input into the supervised point cloud displacement prediction model.

8. The point cloud data denoising method based on an unsupervised network according to claim 1, characterized in that: The point cloud displacement prediction value is iteratively denoised based on the cyclic denoising module to obtain the target denoising result, which is obtained according to the following formula: Among them, S i is the laser radar point cloud to be denoised, S i+1 To S i The point cloud output after the first denoising is input into the target point cloud displacement prediction model, where λ is the step length of each denoising. is the calculated average offset, o(S) is the predicted displacement value output by the displacement prediction module, N(S,S i ) is the distance S i The nearest n point clouds.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the point cloud data denoising method based on an unsupervised network as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the point cloud data denoising method based on an unsupervised network as described in any one of claims 1 to 8.

Images