Three-dimensional model construction method and device based on point cloud

Abstract

The invention relates to the technical field of image data processing, and discloses a three-dimensional scene construction method and device based on point cloud, electronic equipment, a computer readable storage medium and a computer program product. And the generated three-dimensional model is easy to have a large gap and a poor display effect. The method comprises the following steps: acquiring original three-dimensional point cloud data, and segmenting and projecting the three-dimensional point cloud data to obtain at least one point cloud plane; obtaining color attribute information of each point in the point cloud plane and performing color texture fitting to obtain a color texture image corresponding to each point cloud plane; filling each point cloud plane based on the color texture image; and constructing a three-dimensional model based on the filled point cloud plane. According to the method, the display effect of the three-dimensional model constructed based on the three-dimensional point cloud data is improved, and meanwhile, the data volume needing to be stored is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to a point cloud-based three-dimensional scene construction method, device, electronic device, computer storage medium and computer program product. Background Art

[0002] Point cloud is a data set composed of a large number of discrete points in three-dimensional space. These points usually contain rich information, including location information (such as coordinates), color information (such as RGB values), surface direction information (such as normal vectors), intensity information (such as the signal intensity returned by laser scanners), etc. Point cloud data is widely used in computer vision, robot navigation, 3D reconstruction, reverse engineering, autonomous driving and other fields. For larger scenes, the amount of point cloud data is usually very large, which may contain millions or even billions of points. There are many difficulties in processing such large data sets, such as great challenges in data storage, transmission, processing speed and memory consumption.

[0003] In existing technologies, point cloud data is generally converted into a three-dimensional model to reduce the amount of data, thereby solving the problems faced in the storage and transmission process. However, when point cloud data is compressed and converted into a three-dimensional model, due to the sparsity, noise or insufficient sampling of point cloud data, the converted three-dimensional model may have gaps, resulting in poor results in the generated three-dimensional scene. Therefore, there is an urgent need for a three-dimensional scene construction method based on point cloud data, which can reduce the gaps in the generated three-dimensional model while converting point cloud data into a three-dimensional model to reduce the amount of data, thereby ensuring the generation effect of the three-dimensional model. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem that when point cloud data is converted into a three-dimensional model to reduce the amount of data in the prior art, the generated three-dimensional model is prone to large gaps and poor display effect.

[0005] A first aspect of the present invention provides a method for constructing a three-dimensional model based on a point cloud, comprising: obtaining original three-dimensional point cloud data, segmenting and projecting the three-dimensional point cloud data to obtain point cloud planes, wherein the number of the point cloud planes is at least one; obtaining color attribute information of each point in the point cloud plane and performing color texture fitting to obtain a color texture image corresponding to each point cloud plane; filling each point cloud plane based on the color texture image; and constructing a three-dimensional model based on the filled point cloud planes.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the segmentation, fitting, segmentation and projection of the three-dimensional point cloud data to obtain a point cloud plane includes: fitting and segmenting the three-dimensional point cloud data to obtain a fitting plane, wherein the number of the fitting plane is at least one; generating a reference rectangle corresponding to the fitting plane based on the position information of the point in the three-dimensional point cloud data and the position information of the fitting plane; generating projection planes corresponding to each fitting plane with the center of gravity of the fitting plane as the origin and the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the direction of the spatial coordinate axis; and projecting the points in the three-dimensional point cloud data corresponding to each of the fitting planes onto the projection plane to obtain a point cloud plane.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, the fitting and segmenting of the three-dimensional point cloud data to obtain the fitting plane includes: clustering the three-dimensional point cloud data according to the position information in the three-dimensional point cloud data to obtain a point cloud set, wherein the number of the point cloud sets is at least one; performing iterative fitting on each point cloud set according to the distance information and normal information of the points contained in the point cloud set to obtain the fitting plane corresponding to each point cloud set.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, the color attribute information includes RGB color value information and transparency information; the obtaining of the color attribute information of each point in the point cloud plane and performing color texture fitting to obtain the color texture image corresponding to each point cloud plane includes: obtaining the RGB color value information of each point in the point cloud plane, fitting the RGB color value information, and obtaining the color channel value corresponding to each point cloud plane; obtaining the transparency information of each point in each point cloud plane, fitting the transparency information, and obtaining the transparency channel value corresponding to each point cloud plane; based on the color channel value and the transparency channel value, obtaining the color texture image corresponding to each point cloud plane.

[0009] Optionally, in a fourth implementation manner of the first aspect of the present invention, the acquiring RGB color value information of each point in the point cloud plane, fitting the RGB color value information, and obtaining the color channel value corresponding to each point cloud plane includes: acquiring plane coordinate information of each point in the point cloud plane; generating auxiliary point set data under different color channels according to the plane coordinate information and the RGB color value information of each point;

[0010] The B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surfaces under different color channels, and the fitted RGB color value surfaces are generated based on the fitted color value surfaces under different color channels; the color channel values ​​corresponding to each of the point cloud planes are generated based on the fitted RGB color value surfaces.

[0011] Optionally, in a fifth implementation method of the first aspect of the present invention, filling each of the point cloud planes based on the color texture image includes: filling each of the point cloud planes based on the color texture image to obtain a point cloud plane with color texture; constructing a three-dimensional model based on the filled point cloud plane includes: triangulating the point cloud plane with color texture to obtain a triangular mesh structure; generating a completed three-dimensional model based on the triangular mesh structure and the filled point cloud plane with color texture.

[0012] The second aspect of the present invention provides a point cloud-based three-dimensional model construction device, including: a segmentation module, used to obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data, and obtain point cloud planes, wherein the number of the point cloud planes is at least one; a fitting module, used to obtain color attribute information of each point in the point cloud plane and perform color texture fitting to obtain a color texture image corresponding to each point cloud plane; a filling module, used to fill each point cloud plane based on the color texture image; a construction module, used to construct a three-dimensional model based on the filled point cloud plane.

[0013] The third aspect of the present invention provides a point cloud-based three-dimensional model construction device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the point cloud-based three-dimensional model construction device performs the steps of the above-mentioned point cloud-based three-dimensional model construction method.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned point cloud-based three-dimensional model construction method.

[0015] A fifth aspect of the present invention provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the above-mentioned point cloud-based three-dimensional model construction method are implemented.

[0016] In the technical solution provided by the present invention, the original three-dimensional point cloud data is obtained, the three-dimensional point cloud data is segmented and projected to obtain a point cloud plane, wherein the number of the point cloud plane is at least one; the color attribute information of each point in the point cloud plane is obtained and color texture fitting is performed to obtain a color texture image corresponding to each point cloud plane; each point cloud plane is filled based on the color texture image; and a three-dimensional model is constructed based on the filled point cloud plane. This method improves the display effect of the three-dimensional model constructed based on the three-dimensional point cloud data, while reducing the amount of data that needs to be saved. A device, an electronic device, a computer-readable storage medium, and a computer program product provided by the present invention also solve corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a flowchart of a first embodiment of a method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0019] Figure 2 It is a flowchart of a second embodiment of a method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0020] Figure 3 Schematic diagram of original three-dimensional point cloud data in the method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a fitting plane corresponding to original three-dimensional point cloud data in a three-dimensional model construction method based on point cloud in an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a circumscribed minimum area rectangle constructed based on a point cloud plane in a method for constructing a three-dimensional model based on a point cloud in an embodiment of the present invention;

[0023] Figure 6 An example of an image of a point cloud plane and its color channel image in the point cloud-based three-dimensional model construction method in an embodiment of the present invention;

[0024] Figure 7 This is an example of a color value surface graph of the R channel in the point cloud-based three-dimensional model construction method in an embodiment of the present invention;

[0025] Figure 8 This is an example of a color value surface graph of the G channel in the point cloud-based three-dimensional model construction method in an embodiment of the present invention;

[0026] Fig. 9 This is an example of a color value surface graph of the B channel in the point cloud-based three-dimensional model construction method in an embodiment of the present invention;

[0027] Fig.10 A schematic diagram of triangle splitting of a minimum circumscribed rectangle in a point cloud-based three-dimensional model construction method according to an embodiment of the present invention;

[0028] Fig.11 A schematic diagram of a three-dimensional model constructed in the three-dimensional model construction method based on point cloud in an embodiment of the present invention;

[0029] Fig.12is another schematic diagram of a three-dimensional model constructed in the method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0030] Fig.13 A schematic diagram of an embodiment of a point cloud-based three-dimensional model construction device in an embodiment of the present invention;

[0031] Fig.14 A schematic diagram of another embodiment of a device for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0032] Fig.15 A schematic diagram of an embodiment of a device for constructing a three-dimensional model based on point cloud in an embodiment of the present invention;

[0033] Fig.16 The figure is a schematic diagram of a computer-readable medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in a variety of forms, and should not be construed as limiting the present invention to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and is more convenient for fully conveying the inventive concept to those skilled in the art. The same reference numerals in the figures represent the same or similar elements, components or parts, and thus their repeated description will be omitted.

[0035] Under the premise of being consistent with the technical concept of the present invention, the features, structures, characteristics or other details described in a specific embodiment do not exclude that they can be combined in one or more other embodiments in a suitable manner.

[0036] In the description of specific embodiments, the features, structures, characteristics or other details described in the present invention are intended to enable those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics or other details.

[0037] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0040] See also Figure 1 In the first embodiment of the method for constructing a three-dimensional model based on point cloud in the embodiment of the present invention, the method comprises:

[0041] S101, obtaining original three-dimensional point cloud data, segmenting and projecting the three-dimensional point cloud data, and obtaining a point cloud plane;

[0042] It is understandable that the execution subject of the present invention may be a point cloud-based 3D model construction device, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0043] After receiving a request to build a three-dimensional model based on a point cloud, the server first obtains the original three-dimensional point cloud data, wherein the original three-dimensional point cloud data contains position information and color attribute information of each data point; in a specific embodiment, the position information may include the original space coordinates and the normal direction of the point, and the color attribute information may include RGB color value information and transparency information, wherein RGB represents red (R, Red), green (G, Green), and blue (B, Blue), respectively.

[0044] The three-dimensional point cloud data described in this embodiment can be used in scenes that can be abstracted into a model composed of multiple planes, such as point cloud data of a building. Therefore, after obtaining the original three-dimensional point cloud data, the three-dimensional point cloud data is segmented and projected according to the characteristics of the data to obtain at least one point cloud plane.

[0045] In a specific implementation, an initial clustering operation can be first performed based on the position information of the points and the normal direction of the points, so as to obtain multiple point cloud sets that can be approximated as planes. For example, when the point cloud is point cloud data of a cube, it can be roughly divided into 6 point cloud sets in this step, and each point set is point cloud information of a different surface. After obtaining multiple point cloud sets, for each point cloud set, the points in the set are projected onto the same plane in space to obtain the point cloud plane corresponding to each point cloud set. In this way, at least one point cloud plane can be generated based on the position information of each data point in the acquired original three-dimensional point cloud data.

[0046] In a specific embodiment, the plane fitting segmentation can be performed directly based on the three-dimensional point cloud data through the model fitting segmentation method, and multiple optimal planes can be obtained through multiple iterations, and the points in the three-dimensional point cloud can be projected onto the optimal planes to obtain the point cloud planes; or, clustering can be performed first according to the position information in the three-dimensional point cloud data to obtain a point cloud set, and the plane fitting segmentation can be performed on the points in the point cloud set, and the optimal plane corresponding to each point cloud set can be obtained through multiple iterations, and the points contained in the point cloud set can be projected onto the corresponding optimal planes to obtain the point cloud planes.

[0047] S102, obtaining color attribute information of each point in the point cloud plane and performing color texture fitting to obtain a color texture image corresponding to each point cloud plane;

[0048] The color attribute information of each point in the point cloud plane is obtained. Since there may be some points in the projected point cloud plane that do not contain color attribute information, this will cause gaps in the generated image, resulting in poor generation effects. Therefore, in this embodiment, a fitting smoothing operation is performed based on the obtained color attribute information, and fitting color attribute information is generated based on the color attribute information near the gap, so that the color of the gap part can be filled in later based on the fitting color attribute information.

[0049] In a specific implementation, the color attribute information described in this step includes RGB color value information and transparency information, and these two data will be fitted separately when performing color texture fitting. Specifically, the RGB color value information of each point in the point cloud plane is obtained, and the RGB color value information is fitted to obtain the color channel value corresponding to each point cloud plane; then the transparency information of each point in each point cloud plane is obtained, and the transparency information is fitted to obtain the transparency channel value corresponding to each point cloud plane, and the position of the color value is set to 1, and the position without the color value is set to 0, and the transparency channel value is Gaussian blurred and image closed, and the processed result is used as the transparent channel value information corresponding to each color channel value; based on the color channel value and the transparency channel value, the final color texture image corresponding to each point cloud plane is obtained.

[0050] S103, filling each point cloud plane based on the color texture image;

[0051] S104: construct a three-dimensional model based on the filled point cloud plane.

[0052] Based on the color texture images obtained in the aforementioned steps, each of the point cloud planes is filled to obtain a point cloud plane with color texture; and the point cloud plane with color texture is triangulated to obtain a triangular mesh structure; based on the triangular mesh structure and the filled point cloud plane with color texture, a constructed three-dimensional model is generated; wherein the obtained three-dimensional model can be positioned and recorded by the triangular mesh, and the three-dimensional model contains color texture information after color optimization.

[0053] The method in the embodiment of the present invention can convert the point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, and can reduce the amount of data while ensuring that limited spatial information can be obtained; at the same time, it solves the gap problem of point cloud display when constructing a three-dimensional model, and optimizes the display effect.

[0054] Please see Figure 2-12 , a second embodiment of the method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention includes:

[0055] S201, obtaining original three-dimensional point cloud data, performing fitting and segmentation on the three-dimensional point cloud data, and obtaining a fitting plane;

[0056] In this embodiment, the three-dimensional point cloud data of a building is used as an example for explanation. The specific segmentation method can refer to the specific method in step S101 in the above embodiment, wherein the number of fitting planes obtained in this step is at least one.

[0057] Please continue reading Figure 3 as well as Figure 4 In a specific implementation, the specific segmentation method can also be based on the Random Sample Consensus (RANSAC) algorithm. Based on the idea of ​​the random sample consensus algorithm, the distance information and normal information of the points contained in the three-dimensional point cloud data are obtained, and each point cloud set is iteratively fitted to obtain the fitting plane corresponding to each point cloud set. Figure 4 The colors of the planes in are only used to identify the fitting planes to which different point clouds are iteratively fitted and should not be considered as restrictions on the planes.

[0058] In a specific embodiment, the iterative fitting step includes:

[0059] Step (1): Randomly select a point from the point cloud set as the initial seed point;

[0060] Step (2): randomly selecting two other points from the point cloud set to form a plane model with the initial seed point;

[0061] Step (3): obtaining normal information of other points in the point cloud set except the above-mentioned points, and determining whether the angle between the normal of the point and the normal of the plane model satisfies the normal angle threshold. If so, the current point is a supporting point; if not, the current point is an unsupporting point.

[0062] Step (4): Calculate the distance from other points in the point cloud set except the above-mentioned point to the plane model, and determine whether the distance satisfies the preset distance threshold. If so, the current point is a supporting point; if not, the current point is an unsupporting point.

[0063] Step (5): Count the number of points that are support points in step (3) and step (4); if the number exceeds a preset minimum number of support points, the plane model is considered to be a plane model that meets the conditions;

[0064] Step (6): Repeat steps (2) to (5) until a preset number of iterations is reached or the optimal solution is obtained to obtain a plane model with the maximum number of support points, and use the plane model as the final fitting plane of the current point cloud set.

[0065] S202, generating a reference rectangle corresponding to the fitting plane based on the position information of the midpoint of the three-dimensional point cloud data and the position information of the fitting plane;

[0066] The fitting plane obtained in the above steps is a plane with edges, and its shape may be irregular. The reference rectangle generated according to the position information of the corresponding point in the three-dimensional point cloud data and its own position information may be the circumscribed minimum area rectangle.

[0067] In a specific embodiment, see Figure 5 , Figure 5 The middle gray part is a schematic diagram of the fitting plane obtained in the previous step, and the green rectangular box is a schematic diagram of the circumscribed minimum area rectangle of the fitting plane.

[0068] S203, taking the center of gravity of the fitting plane as the origin, and taking the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the spatial coordinate axis direction to generate the projection plane corresponding to each fitting plane;

[0069] S204, projecting the points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection plane to obtain a point cloud plane;

[0070] Please continue reading Figure 5, take the centroid point G of the fitting plane as the origin, the normal direction of the fitting plane as the vertical axis, and the directions of the reference rectangle sides as the horizontal and vertical axes to obtain the spatial coordinate axis directions, i.e. G-xyz, and generate the projection plane corresponding to the fitting plane based on this spatial coordinate axis, and project it onto the projection plane based on the position of the midpoint of the three-dimensional point cloud data corresponding to the fitting plane. When obtaining the centroid point G of the fitting plane, the coordinates of the centroid point G can be obtained by averaging the coordinates of the points corresponding to the fitting plane.

[0071] S205, obtaining RGB color value information of each point in the point cloud plane, fitting the RGB color value information, and obtaining color channel values ​​corresponding to each point cloud plane;

[0072] After the point cloud plane is obtained by projection, the color attribute information of each point in the point cloud plane is obtained and color texture fitting is performed to obtain the color texture image corresponding to each point cloud plane; the color attribute information described in this embodiment includes RGB color value information and transparency information.

[0073] In this step, the RGB color value information of each point in the point cloud plane is obtained, and the RGB color value information is fitted to obtain the color channel value corresponding to each point cloud plane. Specifically, after the projection in the above steps, the points in each point cloud plane are in the same plane. Please continue to refer to Figure 5 In a specific embodiment, it can be regarded as being in the plane of z=0; then the plane coordinate information of each point in the point cloud plane is obtained; auxiliary point set data under different color channels are generated according to the plane coordinate information and the RGB color value information of each point; the B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain color value surfaces under different color channels, and a fitted RGB color value surface is generated based on the fitted color value surfaces under different color channels; the color channel values ​​corresponding to each point cloud plane are generated based on the fitted RGB color value surface.

[0074] See also Figure 6 , the following is a specific example, an image of a point cloud plane can be Figure 6 As shown in the “original image” in the figure, the grayscale images under different channels are obtained according to the RGB color value information of each point in the point cloud plane, as shown in Figure 6As shown in the “R channel grayscale image”, “G channel grayscale image” and “B channel grayscale image”. Then, according to the plane coordinates of each point in the point cloud plane, the x-axis and y-axis numerical coordinates of the color value information of each channel are used, and the grayscale color value of each point in the point cloud plane is used as the z'-axis numerical coordinate of the color value information of each channel, and the auxiliary point set data of the three channels RGB are generated respectively. Then, the B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surfaces under different color channels, and finally the color channel values ​​corresponding to each point cloud plane are obtained. In this way, the possible colors of the gap parts in the point cloud plane can be generated according to the points with color information in the point cloud plane; in a specific implementation manner, the fitted color value surface is specifically as follows Figure 7-9 shown.

[0075] S206, obtaining transparency information of each point in each point cloud plane, fitting the transparency information, and obtaining a transparency channel value corresponding to each point cloud plane;

[0076] Since there are actual gaps and gaps caused by errors in the point cloud, in this embodiment, the transparency information will be fitted to reduce the gaps caused by errors and retain the actual gaps. Specifically, the transparency value of the point with color in the point cloud plane is recorded as 1, and the position without color value is recorded as 0; then a similar method is executed as step S205, and the plane coordinates of each point in the point cloud plane are used as the x-axis and y-axis numerical coordinates according to the color value information of each channel, and the transparency value of each point in the point cloud plane is used as the z" axis numerical coordinate to generate auxiliary point set data of transparency, and the transparency information is fitted by calling the B-spline surface fitting algorithm to obtain the fitted transparency surface, and the transparency value corresponding to each point cloud plane is obtained.

[0077] This step also includes Gaussian blurring and closing the transparency value information corresponding to each point cloud plane, and using the result as the transparent channel value of the previous texture image; in this way, a small number of transparent pixels around solid pixels (i.e., opaque pixels) can be made solid, thereby repairing the pixels in the image that were originally transparent and look like holes after rendering, thereby filling the holes and gaps in the image.

[0078] S207, obtaining a color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value;

[0079] S208, filling each point cloud plane based on each color texture image to obtain a point cloud plane with color texture;

[0080] Based on the superposition of the color channel values ​​and the transparent channel values ​​in step S205 and step S206, the color texture images after the supplementary holes and the smoothed colors corresponding to each point cloud plane are obtained, and each point cloud plane is filled based on each color texture image to obtain a point cloud plane with color texture;

[0081] S209, triangulating the point cloud plane with color texture to obtain a triangular mesh structure, and generating a constructed three-dimensional model based on the triangular mesh structure and the filled point cloud plane with color texture.

[0082] After obtaining multiple point cloud planes with color textures, a completed three-dimensional model can be generated based on the filled point cloud planes with color textures. In this embodiment, when generating a three-dimensional model, the minimum circumscribed rectangle obtained in the above steps is divided into triangles to construct a triangular network. Specifically, the vertices of each rectangle are numbered. A specific example is given below for explanation. Fig.10 In a specific implementation, the vertices of a minimum enclosing rectangle are labeled with numbers 0, 1, 2, and 3. In a segmentation method, the minimum enclosing rectangle can be split into two triangles with vertices numbered (0, 1, 2) and vertices numbered (1, 3, 2). After the minimum enclosing rectangles corresponding to all point cloud planes are used to construct a triangular network in this way, a completed three-dimensional model with color texture is obtained.

[0083] See also Fig.11 as well as Fig.12 , which are effect diagrams of two specific examples of the generated three-dimensional model with color texture. After the solution in this embodiment, the three-dimensional model can basically achieve the display effect of filling the gaps and holes.

[0084] The method in the embodiment of the present invention can convert the point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, and can reduce the amount of data while ensuring that limited spatial information can be obtained; at the same time, it solves the gap problem of point cloud display when constructing a three-dimensional model, and optimizes the display effect.

[0085] The above describes the method for constructing a three-dimensional model based on point cloud in an embodiment of the present invention. The following describes the device for constructing a three-dimensional model based on point cloud in an embodiment of the present invention. Fig.13 , an embodiment of a three-dimensional model construction device based on point cloud in an embodiment of the present invention includes:

[0086] The segmentation module 1301 is used to obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data to obtain a point cloud plane, wherein the number of the point cloud plane is at least one;

[0087] A fitting module 1302 is used to obtain color attribute information of each point in the point cloud plane and perform color texture fitting to obtain a color texture image corresponding to each point cloud plane;

[0088] A filling module 1303, used to fill each of the point cloud planes based on the color texture image;

[0089] The construction module 1304 is used to construct a three-dimensional model based on the filled point cloud plane.

[0090] The method in the embodiment of the present invention can convert the point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, and can reduce the amount of data while ensuring that limited spatial information can be obtained; at the same time, it solves the gap problem of point cloud display when constructing a three-dimensional model, and optimizes the display effect.

[0091] See also Fig.14 In another embodiment of the present application, the segmentation module 1301 includes:

[0092] The segmentation execution unit 13011 is used to perform fitting segmentation on the three-dimensional point cloud data to obtain a fitting plane, wherein the number of the fitting plane is at least one;

[0093] A rectangle fitting unit 13012, configured to generate a reference rectangle corresponding to a fitting plane based on the position information of the point in the three-dimensional point cloud data and the position information of the fitting plane;

[0094] The projection plane generating unit 13013 is used to generate the projection plane corresponding to each fitting plane by taking the center of gravity of the fitting plane as the origin and the normal direction of the fitting plane and the direction of the reference rectangle side as the spatial coordinate axis direction;

[0095] The projection execution unit 13014 is used to project the points in the three-dimensional point cloud data corresponding to each of the fitting planes onto the projection plane to obtain a point cloud plane.

[0096] In another embodiment of the present application, the segmentation execution unit 13011 is specifically used for:

[0097] Clustering the position information in the three-dimensional point cloud data to obtain a point cloud set, wherein the number of the point cloud set is at least one;

[0098] According to the distance information and normal information of the points contained in the point cloud set, iterative fitting is performed on each point cloud set to obtain the fitting plane corresponding to each point cloud set.

[0099] In another embodiment of the present application, the color attribute information includes RGB color value information and transparency information;

[0100] The fitting module 1302 includes:

[0101] A color fitting unit 13021 is used to obtain RGB color value information of each point in the point cloud plane, and fit the RGB color value information to obtain a color channel value corresponding to each point cloud plane;

[0102] A transparency fitting unit 13022 is used to obtain transparency information of each point in each point cloud plane, and fit the transparency information to obtain a transparency channel value corresponding to each point cloud plane;

[0103] The fitting image generation unit 13023 is used to obtain the color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value.

[0104] In another embodiment of the present application, the color fitting unit 13021 is specifically used for:

[0105] Obtaining plane coordinate information of each point in the point cloud plane;

[0106] Generate auxiliary point set data under different color channels respectively according to the plane coordinate information and the RGB color value information of each point;

[0107] The B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surfaces under different color channels, and the fitted RGB color value surfaces are generated based on the fitted color value surfaces under different color channels;

[0108] The color channel values ​​corresponding to each of the point cloud planes are generated based on the fitted RGB color value surface.

[0109] In another embodiment of the present application, the filling module 1303 is further used to:

[0110] Filling each of the point cloud planes based on each of the color texture images to obtain a point cloud plane with color texture;

[0111] The construction module 1304 is also specifically used for:

[0112] Perform triangulation on the point cloud plane with color texture to obtain a triangular mesh structure;

[0113] A constructed three-dimensional model is generated based on the triangular mesh structure and the filled point cloud plane with color texture.

[0114] The method in the embodiment of the present invention can convert the point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, and can reduce the amount of data while ensuring that limited spatial information can be obtained; at the same time, it solves the gap problem of point cloud display when constructing a three-dimensional model, and optimizes the display effect.

[0115] Based on the same inventive concept, an embodiment of this specification also provides an electronic device constructed based on a three-dimensional model of a point cloud. The electronic device constructed based on a three-dimensional model of a point cloud in an embodiment of the present invention is described in detail below from the perspective of hardware processing.

[0116] Fig.15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Fig.15 15 to 16. The electronic device 1500 according to this embodiment of the present invention is described. Fig.15 The electronic device 1500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0117] like Fig.15 As shown, the electronic device 1500 is in the form of a general computing device. The components of the electronic device 1500 may include, but are not limited to: at least one processing unit 1510, at least one storage unit 1520, a bus 1530 connecting different system components (including the storage unit 1520 and the processing unit 1510), a display unit 1540, etc.

[0118] The storage unit stores program codes, which can be executed by the processing unit 1510, so that the processing unit 1510 performs the steps according to various exemplary embodiments of the present invention described in the above processing method section of this specification. For example, the processing unit 1510 can perform the following steps: Figure 1 or Figure 2 The steps of the method are shown.

[0119] The storage unit 1520 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 15201 and / or a cache memory unit 15202 , and may further include a read-only memory unit (ROM) 15203 .

[0120] The storage unit 1520 may also include a program / utility 15204 having a set (at least one) of program modules 15205, such program modules 15205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0121] Bus 1530 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0122] The electronic device 1500 may also communicate with one or more external devices 100 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 1500, and / or any device that enables the electronic device 1500 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 1550. Furthermore, the electronic device 1500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1560. The network adapter 1560 may communicate with other modules of the electronic device 1500 through the bus 1530. It should be understood that although Fig.15 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the exemplary embodiments described in the present invention can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation method of the present invention can be embodied in the form of a software product, which can be stored in a computer-readable storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including a number of instructions to enable a computing device (which can be a personal computer, server, or network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: Figure 1 or Figure 2 The method shown.

[0124] Fig.16 A schematic diagram of a computer-readable medium provided for an embodiment of this specification.

[0125] accomplish Figure 1 or Figure 2The computer program of the method shown can be stored on one or more computer readable media. The computer readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0126] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0127] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0128] In summary, the present invention can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that general data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0129] In addition, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the point cloud-based three-dimensional model construction method described in any of the above embodiments.

[0130] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the present invention is not inherently related to any specific computer, virtual device or electronic device, and various general devices can also implement the present invention. The above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0131] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0132] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for constructing a three-dimensional model based on point cloud, characterized in that: include: Acquire original three-dimensional point cloud data, segment and project the three-dimensional point cloud data to obtain point cloud planes, wherein the number of the point cloud planes is at least one; Acquire color attribute information of each point in the point cloud plane and perform color texture fitting to obtain a color texture image corresponding to each point cloud plane; Filling each of the point cloud planes based on the color texture image; Construct a 3D model based on the filled point cloud plane.

2. The method for constructing a three-dimensional model based on point cloud according to claim 1, characterized in that: The three-dimensional point cloud data is segmented, fitted, segmented and projected to obtain a point cloud plane, including: Perform fitting segmentation on the three-dimensional point cloud data to obtain a fitting plane, wherein the number of the fitting plane is at least one; Generate a reference rectangle corresponding to the fitting plane based on the position information of the point in the three-dimensional point cloud data and the position information of the fitting plane; The projection planes corresponding to the fitting planes are generated by taking the center of gravity of the fitting plane as the origin and the normal direction of the fitting plane and the direction of the reference rectangle side as the spatial coordinate axis directions; The points in the three-dimensional point cloud data corresponding to each of the fitting planes are projected onto the projection plane to obtain a point cloud plane.

3. The method for constructing a three-dimensional model based on point cloud according to claim 2, characterized in that: The fitting and segmenting of the three-dimensional point cloud data to obtain the fitting plane includes: Clustering the position information in the three-dimensional point cloud data to obtain a point cloud set, wherein the number of the point cloud set is at least one; According to the distance information and normal information of the points contained in the point cloud set, iterative fitting is performed on each point cloud set to obtain the fitting plane corresponding to each point cloud set.

4. The method for constructing a three-dimensional model based on point cloud according to claim 1, characterized in that: The color attribute information includes RGB color value information and transparency information; The acquiring color attribute information of each point in the point cloud plane and performing color texture fitting to obtain the color texture image corresponding to each point cloud plane comprises: Acquire RGB color value information of each point in the point cloud plane, and fit the RGB color value information to obtain color channel values ​​corresponding to each point cloud plane; Obtaining transparency information of each point in each of the point cloud planes, fitting the transparency information, and obtaining a transparency channel value corresponding to each of the point cloud planes; Based on the color channel value and the transparency channel value, a color texture image corresponding to each point cloud plane is obtained.

5. The method for constructing a three-dimensional model based on point cloud according to claim 3, characterized in that: The acquiring RGB color value information of each point in the point cloud plane, and fitting the RGB color value information to obtain the color channel value corresponding to each point cloud plane comprises: Obtaining plane coordinate information of each point in the point cloud plane; Generate auxiliary point set data under different color channels respectively according to the plane coordinate information and the RGB color value information of each point; The B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surfaces under different color channels, and the fitted RGB color value surfaces are generated based on the fitted color value surfaces under different color channels; The color channel values ​​corresponding to each of the point cloud planes are generated based on the fitted RGB color value surface.

6. The method for constructing a three-dimensional model based on point cloud according to claim 1, characterized in that: The filling of each of the point cloud planes based on the color texture image comprises: Filling each of the point cloud planes based on each of the color texture images to obtain a point cloud plane with color texture; The constructing of a three-dimensional model based on the filled point cloud plane comprises: Perform triangulation on the point cloud plane with color texture to obtain a triangular mesh structure; A constructed three-dimensional model is generated based on the triangular mesh structure and the filled point cloud plane with color texture.

7. A three-dimensional model construction device based on point cloud, characterized in that: The point cloud-based three-dimensional model building device comprises: A segmentation module, used to obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data to obtain a point cloud plane, wherein the number of the point cloud plane is at least one; A fitting module, used to obtain the color attribute information of each point in the point cloud plane and perform color texture fitting to obtain a color texture image corresponding to each point cloud plane; A filling module, used for filling each of the point cloud planes based on the color texture image; The construction module is used to construct a 3D model based on the filled point cloud plane.

8. A three-dimensional model building device based on point cloud, characterized in that: The point cloud-based three-dimensional model construction device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the point cloud-based three-dimensional model construction device to perform the steps of the point cloud-based three-dimensional model construction method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the program / instructions are executed by a processor, the steps of the point cloud-based three-dimensional model construction method as described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the point cloud-based three-dimensional model construction method as described in any one of claims 1 to 6 are implemented.

Images