Three-dimensional image point cloud splicing method

Abstract

The invention discloses a three-dimensional image point cloud splicing method. The method comprises the following steps: S1, inputting an actual three-dimensional scene image and a training image to be spliced; s2, modeling a corresponding three-dimensional point cloud scene by using the actual three-dimensional scene image; s3, filtering the point cloud data; s4, complementing the point cloud data according to the to-be-spliced training image by adopting a gaan adversarial generative network, and generating a spliced three-dimensional image according to a complemented point cloud set; the Gan adversarial generative network comprises a generator G and a discriminator D, and the generator G and the discriminator D both adopt a network structure in the form of a convolution unit from convolution to batch standardization to activation of a Relu function. According to the invention, the statistical filtering technology is adopted to filter the point cloud data, so that the point cloud with a large error is removed, and the splicing accuracy is improved; and point cloud data are complemented and spliced by adopting the Gan confrontation generation network, so that the image can be spliced on a three-dimensional structure, and the image processing efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional image point cloud splicing, and in particular to a three-dimensional image point cloud splicing method. Background Art

[0002] A point cloud is a massive set of points that expresses the spatial distribution and surface characteristics of a target in the same spatial reference system. After obtaining the spatial coordinates of each sampling point on the surface of an object, a collection of points is obtained, which is called a "point cloud."

[0003] Point cloud models are often obtained directly from star measurement. Each point corresponds to a measurement point without other processing methods, so it contains the largest amount of information. This information is hidden in the point cloud and needs to be extracted by other extraction methods. The process of extracting information from the point cloud is three-dimensional image processing.

[0004] However, the current 3D image stitching technology is not perfect, the point cloud model cannot be directly used for 3D image stitching, and the existing 3D image stitching efficiency is not high. Summary of the invention

[0005] In order to at least solve or partially solve the above problems, a three-dimensional image point cloud stitching method is provided, which can quickly realize three-dimensional image stitching.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] The present invention provides a three-dimensional image point cloud stitching method, comprising the following steps:

[0008] S1: Input the actual 3D scene image and the training image to be stitched;

[0009] S2: Use the actual 3D scene image to model the corresponding 3D point cloud scene;

[0010] S3: filtering the point cloud data;

[0011] S4: The gan generative adversarial network is used to complete the point cloud data according to the training image to be spliced, and the spliced ​​stereo image is generated according to the completed point cloud set.

[0012] As a preferred technical solution of the present invention, the step of filtering the point cloud is as follows:

[0013] A. For each point, set a search radius to determine its neighbor point set;

[0014] B. Calculate the distance between each point and its neighbor point in the neighbor point set, and calculate the mean and standard deviation of the distance;

[0015] C. Get the score of each point based on the mean and standard deviation; the score indicates the degree of deviation between the distance between a point and its neighboring points and the average distance;

[0016] D. According to the set threshold, points with scores less than the threshold are regarded as outliers and removed from the point cloud.

[0017] As a preferred technical solution of the present invention, the steps of point cloud completion are as follows:

[0018] A: Input the training image to be spliced ​​as the input image into the generator G to obtain the implicit feature point cloud set a;

[0019] B: Add the point cloud set after filtering of the actual 3D scene image to the implicit feature point cloud set a obtained in the generator G to obtain a spliced ​​point cloud set C, and input the spliced ​​point cloud set C into the discriminator D;

[0020] C: Construct a discriminator D with the same convolutional unit form as the generator G to discriminate the spliced ​​point cloud set C and the implicit feature point cloud set a. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into one category, then train the adjustment parameters of the discriminator D again. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into two categories, then train the adjustment parameters of the generator G again and re-execute step A until there is no more room for improvement between the discriminator D and the generator G, and output the point cloud set of the generator G.

[0021] As a preferred technical solution of the present invention, the generator G includes 12 convolution base layers, 12 deconvolution layers and 12 concat layers, the concat layer is used to map the input data to the deconvolution process, the convolution and deconvolution structures are aligned, and the discriminator D adopts a network structure in the form of a convolution unit from convolution to batch normalization to activation of a Relu function.

[0022] As a preferred technical solution of the present invention, the activation function Relu is a linear function. When the input value is a negative value, the output value is 0, and when the input value is a positive value, the output value is the same as the input value.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention uses statistical filtering technology to filter the point cloud data, thereby removing point clouds with large errors and improving the accuracy of splicing;

[0025] The GAN generative adversarial network is used to complete and splice point cloud data, so that images can be spliced ​​on three-dimensional structures, greatly improving the image processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0028] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0029] Furthermore, if a detailed description of known techniques is not necessary for illustrating the characteristics of the present invention, it will be omitted.

[0030] Example 1

[0031] like Figure 1 As shown, the present invention provides a three-dimensional image point cloud stitching method, comprising the following steps:

[0032] S1: Input the actual 3D scene image and the training image to be stitched;

[0033] S2: Use the actual 3D scene image to model the corresponding 3D point cloud scene;

[0034] S3: filtering the point cloud data;

[0035] S4: The gan generative adversarial network is used to complete the point cloud data according to the training image to be spliced, and the spliced ​​stereo image is generated according to the completed point cloud set.

[0036] Specifically, the steps for filtering the point cloud are as follows:

[0037] A. For each point, set a search radius to determine its neighbor point set;

[0038] B. Calculate the distance between each point and its neighbor point in the neighbor point set, and calculate the mean and standard deviation of the distance;

[0039] C. Get the score of each point based on the mean and standard deviation; the score indicates the degree of deviation between the distance between a point and its neighboring points and the average distance;

[0040] D. According to the set threshold, points with scores less than the threshold are regarded as outliers and removed from the point cloud.

[0041] The steps for point cloud completion are as follows:

[0042] A: Input the training image to be spliced ​​as the input image into the generator G to obtain the implicit feature point cloud set a;

[0043] B: Add the point cloud set after filtering of the actual 3D scene image to the implicit feature point cloud set a obtained in the generator G to obtain a spliced ​​point cloud set C, and input the spliced ​​point cloud set C into the discriminator D;

[0044] C: Construct a discriminator D with the same convolutional unit form as the generator G to discriminate the spliced ​​point cloud set C and the implicit feature point cloud set a. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into one category, then train the adjustment parameters of the discriminator D again. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into two categories, then train the adjustment parameters of the generator G again and re-execute step A until there is no more room for improvement between the discriminator D and the generator G, and output the point cloud set of the generator G.

[0045] Furthermore, the generator G includes 12 convolution base layers, 12 deconvolution layers and 12 concat layers, the concat layer is used to map the input data to the deconvolution process, the convolution and deconvolution structures are aligned, and the discriminator D adopts a network structure in the form of a convolution unit from convolution to batch normalization to activation of a Relu function.

[0046] The activation function Relu is a linear function. When the input value is negative, the output value is 0. When the input value is positive, the output value is the same as the input value.

[0047] The present invention uses statistical filtering technology to filter the point cloud data, thereby removing point clouds with large errors and improving the accuracy of splicing;

[0048] The GAN generative adversarial network is used to complete and splice point cloud data, so that images can be spliced ​​on three-dimensional structures, greatly improving the image processing efficiency.

[0049] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional image point cloud stitching method, characterized in that: The following steps are involved: S1: Input the actual 3D scene image and the training image to be stitched; S2: Use the actual 3D scene image to model the corresponding 3D point cloud scene; S3: filtering the point cloud data; S4: The gan generative adversarial network is used to complete the point cloud data according to the training image to be spliced, and the spliced ​​stereo image is generated according to the completed point cloud set.

2. A three-dimensional image point cloud stitching method according to claim 1, characterized in that: The steps of filtering the point cloud are as follows: A. For each point, set a search radius to determine its neighbor point set; B. Calculate the distance between each point and its neighbor point in the neighbor point set, and calculate the mean and standard deviation of the distance; C. Get the score of each point based on the mean and standard deviation; the score indicates the degree of deviation between the distance between a point and its neighboring points and the average distance; D. According to the set threshold, points with scores less than the threshold are regarded as outliers and removed from the point cloud.

3. A three-dimensional image point cloud stitching method according to claim 1, characterized in that: The steps of point cloud completion are as follows: A: Input the training image to be spliced ​​as the input image into the generator G to obtain the implicit feature point cloud set a; B: Add the point cloud set after filtering of the actual 3D scene image to the implicit feature point cloud set a obtained in the generator G to obtain a spliced ​​point cloud set C, and input the spliced ​​point cloud set C into the discriminator D; C: Construct a discriminator D with the same convolutional unit form as the generator G to discriminate the spliced ​​point cloud set C and the implicit feature point cloud set a. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into one category, then train the adjustment parameters of the discriminator D again. If the discriminator D classifies the spliced ​​point cloud set C and the implicit feature point cloud set a into two categories, then train the adjustment parameters of the generator G again and re-execute step A until there is no more room for improvement between the discriminator D and the generator G, and output the point cloud set of the generator G.

4. A three-dimensional image point cloud stitching method according to claim 3, characterized in that: The generator G includes 12 convolution base layers, 12 deconvolution layers and 12 concat layers, the concat layer is used to map the input data to the deconvolution process, the convolution is aligned with the deconvolution structure, and the discriminator D adopts a network structure in the form of a convolution unit from convolution to batch normalization to activation of Relu function.

5. A three-dimensional image point cloud stitching method according to claim 4, characterized in that: The activation function Relu is a linear function. When the input value is negative, the output value is 0. When the input value is positive, the output value is the same as the input value.

Images