Method and apparatus for constructing a three-dimensional model based on a point cloud

Abstract

The application relates to the technical field of image data processing, and discloses a three-dimensional scene construction method and device based on point clouds, electronic equipment, a computer readable storage medium and a computer program product, and is used for solving the technical problem that a three-dimensional model generated when point cloud data is converted into a three-dimensional model to reduce the data volume is prone to having a large gap and a poor display effect. The method comprises the following steps: obtaining original three-dimensional point cloud data, segmenting and projecting the three-dimensional point cloud data to obtain point cloud planes, wherein the number of the point cloud planes is at least one; obtaining color attribute information of each point in the point cloud planes and performing color texture fitting to obtain a color texture image corresponding to each point cloud plane; filling each point cloud plane based on the color texture image; and constructing a three-dimensional model based on the filled point cloud planes. The method improves the display effect of a three-dimensional model constructed based on three-dimensional point cloud data, and simultaneously reduces the data volume that needs to be saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to a point cloud-based three-dimensional scene construction method and device, electronic equipment, computer storage medium and computer program product. BACKGROUND

[0002] Point cloud is a data set composed of a large number of discrete points in three-dimensional space, which usually contains rich information, including position information (such as coordinates), color information (such as RGB values), surface direction information (such as normal vectors), intensity information (such as signal intensity returned by a laser scanner), etc. Point cloud data is widely used in computer vision, robot navigation, three-dimensional reconstruction, reverse engineering, autonomous driving and other fields. For larger scenes, the data volume of point cloud data is usually very large, and may contain data of millions or even billions of points. Processing such large data sets has many difficulties, such as great challenges in data storage, transmission, processing speed and memory consumption.

[0003] In existing technologies, point cloud data is generally converted into a three-dimensional model to reduce data volume, thereby solving problems encountered in storage, transmission and other processes. However, when point cloud data is compressed and converted into a three-dimensional model, due to the sparsity, noise or insufficient sampling of point cloud data, the three-dimensional model obtained by conversion may have gaps, resulting in poor effect of the generated three-dimensional scene. Therefore, there is an urgent need for a point cloud-based three-dimensional scene construction method that can reduce the gaps in the generated three-dimensional model while converting point cloud data into a three-dimensional model to reduce data volume, and ensure the generation effect of the three-dimensional model. SUMMARY

[0004] The main purpose of the present application is to solve the technical problems of large gaps and poor display effect in the generated three-dimensional model when converting point cloud data into a three-dimensional model to reduce data volume in the prior art.

[0005] The first aspect of the present application provides a point cloud-based three-dimensional model construction method, comprising: obtaining original three-dimensional point cloud data, segmenting and projecting the three-dimensional point cloud data to obtain a point cloud plane, wherein the number of point cloud planes is at least one; obtaining color attribute information of each point in the point cloud plane and performing color texture fitting to obtain a color texture image corresponding to each point cloud plane; filling each point cloud plane based on the color texture image; and constructing a three-dimensional model based on the filled point cloud plane.

[0006] Optionally, in the first implementation manner of the first aspect of the present application, the segmenting and fitting segmentation and projecting the three-dimensional point cloud data to obtain the point cloud planes comprises: performing fitting segmentation on the three-dimensional point cloud data to obtain fitting planes, wherein the number of the fitting planes is at least one; generating a reference rectangle corresponding to a fitting plane based on position information of points in the three-dimensional point cloud data and position information of the fitting plane; generating a projection plane corresponding to the fitting plane with the center of gravity of the fitting plane as the origin and with the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the spatial coordinate axis directions; projecting points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection plane to obtain a point cloud plane.

[0007] Optionally, in the second implementation manner of the first aspect of the present application, the fitting segmentation on the three-dimensional point cloud data to obtain the fitting planes comprises: clustering according to position information in the three-dimensional point cloud data to obtain point cloud sets, wherein the number of the point cloud sets is at least one; performing iterative fitting on each point cloud set according to distance information and normal information of points contained in the point cloud set to obtain a fitting plane corresponding to each point cloud set.

[0008] Optionally, in the third implementation manner of the first aspect of the present application, the color attribute information comprises RGB color value information and transparency information; and the obtaining of the color attribute information of each point in the point cloud plane and the fitting of the color texture to obtain a color texture image corresponding to each point cloud plane comprises: obtaining RGB color value information of each point in the point cloud plane, fitting the RGB color value information to obtain a color channel value corresponding to each point cloud plane; obtaining transparency information of each point in each point cloud plane, fitting the transparency information to obtain a transparency channel value corresponding to each point cloud plane; and obtaining a color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value.

[0009] Optionally, in the fourth implementation manner of the first aspect of the present application, the obtaining of the RGB color value information of each point in the point cloud plane and the fitting of the RGB color value information to obtain a color channel value corresponding to each point cloud plane comprises: obtaining plane coordinate information of each point in the point cloud plane; generating auxiliary point set data under different color channels based on the plane coordinate information and the RGB color value information of each point;

[0010] fitting the auxiliary point set data under different color channels to obtain color value surfaces under different color channels by calling a B-spline surface fitting algorithm, and generating a fitted RGB color value surface based on the fitted color value surfaces under different color channels; and generating a color channel value corresponding to each point cloud plane based on the fitted RGB color value surface.

[0011] Optionally, in a fifth implementation form of the first aspect of the present application, the filling each of the point cloud planes based on the color texture images comprises: filling each of the point cloud planes based on the color texture images to obtain point cloud planes with color texture; and the constructing the three-dimensional model based on the filled point cloud planes comprises: performing triangle segmentation on the point cloud planes with color texture to obtain a triangular mesh structure; and generating the constructed three-dimensional model based on the triangular mesh structure and the filled point cloud planes with color texture.

[0012] The second aspect of the present application provides a three-dimensional model construction device based on point cloud, comprising: a segmentation module configured to obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data to obtain point cloud planes, wherein the number of the point cloud planes is at least one; a fitting module configured to obtain color attribute information of each point in the point cloud planes and perform color texture fitting to obtain color texture images corresponding to each of the point cloud planes; a filling module configured to fill each of the point cloud planes based on the color texture images; and a construction module configured to construct a three-dimensional model based on the filled point cloud planes.

[0013] The third aspect of the present application provides a three-dimensional model construction device based on point cloud, comprising: a memory and at least one processor, wherein the memory stores instructions; and the at least one processor invokes the instructions in the memory to enable the three-dimensional model construction device based on point cloud to perform the steps of the three-dimensional model construction method based on point cloud.

[0014] The fourth aspect of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions are executed on a computer, the computer is enabled to perform the steps of the three-dimensional model construction method based on point cloud.

[0015] The fifth aspect of the present application provides a computer program product, comprising computer programs / instructions, and the computer programs / instructions are configured to enable a processor to perform the steps of the three-dimensional model construction method based on point cloud when the computer programs / instructions are executed by the processor.

[0016] In the technical solution provided by the present application, original three-dimensional point cloud data is obtained, the three-dimensional point cloud data is segmented and projected to obtain point cloud planes, wherein the number of the point cloud planes is at least one; color attribute information of each point in the point cloud planes is obtained and color texture fitting is performed to obtain color texture images corresponding to each of the point cloud planes; each of the point cloud planes is filled based on the color texture images; and a three-dimensional model is constructed based on the filled point cloud planes. This method improves the display effect of the three-dimensional model constructed based on three-dimensional point cloud data, and reduces the amount of data to be saved. The device, electronic equipment, computer readable storage medium and computer program product provided by the present application also solve the corresponding technical problems. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the point cloud-based 3D model construction method of the present invention.

[0019] Figure 2 This is a flowchart illustrating the second embodiment of the point cloud-based 3D model construction method in this invention.

[0020] Figure 3 This is a schematic diagram of the original 3D point cloud data in the point cloud-based 3D model construction method in this embodiment of the invention;

[0021] Figure 4 This is a schematic diagram of the fitting plane corresponding to the original three-dimensional point cloud data in the point cloud-based three-dimensional model construction method in this embodiment of the invention;

[0022] Figure 5 This is a schematic diagram of the circumscribed minimum area rectangle constructed based on the point cloud plane in the point cloud-based 3D model construction method of this invention.

[0023] Figure 6 This is an example of an image of a point cloud plane and its color channel image in a point cloud-based 3D model construction method according to an embodiment of the present invention;

[0024] Figure 7 This is an example of the color value surface plot of the R channel in the point cloud-based 3D model construction method of this invention;

[0025] Figure 8 This is an example of the color value surface plot of the G channel in the point cloud-based 3D model construction method of this invention;

[0026] Figure 9 This is an example of the color value surface plot of the B channel in the point cloud-based 3D model construction method of this invention;

[0027] Figure 10 This is a schematic diagram illustrating the triangular decomposition of the minimum bounding rectangle in the point cloud-based 3D model construction method of this invention.

[0028] Figure 11 This is a schematic diagram of a three-dimensional model constructed in the point cloud-based three-dimensional model construction method in an embodiment of the present invention;

[0029] Figure 12Another schematic view of a three-dimensional model constructed by the method for constructing a three-dimensional model based on a point cloud in an embodiment of the present application;

[0030] Figure 13 A schematic view of one embodiment of the device for constructing a three-dimensional model based on a point cloud in an embodiment of the present application;

[0031] Figure 14 A schematic view of another embodiment of the device for constructing a three-dimensional model based on a point cloud in an embodiment of the present application;

[0032] Figure 15 A schematic view of one embodiment of the device for constructing a three-dimensional model based on a point cloud in an embodiment of the present application;

[0033] Figure 16 A schematic view of one embodiment of the device for constructing a three-dimensional model based on a point cloud in an embodiment of the present application; DETAILED DESCRIPTION

[0034] Exemplary embodiments of the present application will now be described more fully hereinafter with reference to the accompanying drawings. The exemplary embodiments of the present application may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. Like reference numerals refer to like elements throughout the specification.

[0035] In the case of a certain specific embodiment, the features, structures, characteristics or other details described do not exclude the possibility of being combined in one or more other embodiments in a suitable manner, in accordance with the technical idea of the present application.

[0036] In the description of the specific embodiments, the features, structures, characteristics or other details described are intended to enable a full understanding of the embodiments by those skilled in the art. However, one or more of the features, structures, characteristics or other details can not be practiced by those skilled in the art without the specific feature, structure, characteristic or other detail.

[0037] The flowcharts shown in the drawings are only exemplary illustrations, and do not necessarily include all the contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further divided, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0038] The block diagrams shown in the drawings are functional entities, and do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0039] The term "and / or" or "and / or" includes all combinations of one or more of the associated listed items.

[0040] Please refer to Figure 1 The first embodiment of the point cloud-based three-dimensional model construction method in the embodiment of the application includes:

[0041] S101, obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data, and obtain a point cloud plane;

[0042] It can be understood that the execution subject of the application can be a point cloud-based three-dimensional model construction device, and can also be a terminal or a server, and the specific implementation is not limited herein. The embodiment of the application takes the server as an execution subject for example.

[0043] After receiving the request for constructing a three-dimensional model based on point cloud, the server first obtains original three-dimensional point cloud data, wherein the original three-dimensional point cloud data contains position information and color attribute information of each data point; in one specific embodiment, the position information can include original spatial coordinates and normal direction of the point, and the color attribute information can include RGB color value information and transparency information, wherein RGB respectively represents red (R, Red), green (G, Green) and blue (B, Blue).

[0044] In the embodiment, the three-dimensional point cloud data can be used in a scene that can be abstracted as a model composed of multiple planes, for example, point cloud data of a building. Therefore, after obtaining the original three-dimensional point cloud data, the three-dimensional point cloud data is segmented and projected according to the characteristics of the data, and at least one point cloud plane is obtained.

[0045] In one specific embodiment, an initial clustering operation can be first performed based on the position information of the point and the normal direction of the point, so as to obtain multiple point cloud sets that can be approximately planar, for example, when the point cloud is a point cloud data of a cube, the point cloud set can be roughly segmented into 6 point cloud sets in this step, and each point set is point cloud information of a different surface. After obtaining the multiple point cloud sets, the points in each point cloud set are projected onto the same plane in space to obtain a point cloud plane corresponding to each point cloud set. In this way, at least one point cloud plane can be generated based on the position information of each data point in the original three-dimensional point cloud data.

[0046] In one specific embodiment, the plane fitting segmentation can be directly based on the three-dimensional point cloud data through a model fitting segmentation method, a plurality of optimal planes are obtained through multiple iterations, and the points in the three-dimensional point cloud are projected into the optimal planes to obtain point cloud planes; or, the point cloud sets are obtained according to the position information in the three-dimensional point cloud data, the points in the point cloud sets are respectively subjected to plane fitting segmentation, the optimal planes corresponding to each point cloud set are obtained through multiple iterations, and the points contained in the point cloud sets are projected into the corresponding optimal planes to obtain point cloud planes.

[0047] S102, color attribute information of each point in the point cloud plane is obtained, and color texture fitting is performed to obtain a color texture image corresponding to each point cloud plane;

[0048] The color attribute information of each point in the point cloud plane is obtained. Since there are some points in the projected point cloud plane that do not contain color attribute information, this will cause gaps in the generated image, resulting in poor generation effect. Therefore, in this embodiment, a fitting and smoothing operation is performed based on the obtained color attribute information, and fitting color attribute information is generated according to the color attribute information near the gap, so as to fill the color of the gap part according to the fitting color attribute information subsequently.

[0049] In one specific embodiment, the color attribute information in this step includes RGB color value information and transparency information, and the two data are fitted respectively when performing color texture fitting. Specifically, the RGB color value information of each point in the point cloud plane is obtained, the RGB color value information is fitted to obtain a color channel value corresponding to each point cloud plane; then, the transparency information of each point in each point cloud plane is obtained, the transparency information is fitted to obtain a transparency channel value corresponding to each point cloud plane, the position of the color value is set to 1, the position without color value is set to 0, and the transparency channel value is subjected to Gaussian blur and image closing processing, and the result obtained is used as the transparency channel value information corresponding to each color channel value; based on the color channel value and the transparency channel value, a final color texture image corresponding to each point cloud plane is obtained.

[0050] S103, filling each point cloud plane based on the color texture image;

[0051] S104, constructing a three-dimensional model based on the filled point cloud plane.

[0052] Based on the color texture images obtained in the preceding steps, each point cloud plane is filled to obtain a point cloud plane with color texture; and the point cloud plane with color texture is triangularly segmented to obtain a triangular mesh structure; a completed three-dimensional model is generated based on the triangular mesh structure and the filled point cloud plane with color texture; wherein, the obtained three-dimensional model can be located and recorded through the triangular mesh, and the three-dimensional model contains color texture information after color optimization.

[0053] The method in this embodiment of the invention can convert point clouds into a three-dimensional model composed of multiple planes to solve the problem of point cloud data compression. It can reduce the amount of data while ensuring that limited spatial information can be obtained. At the same time, it solves the problem of gaps in point cloud display when constructing the three-dimensional model and optimizes the display effect.

[0054] Please refer to Figures 2-12 The second embodiment of the point cloud-based 3D model construction method in this invention includes:

[0055] S201. Obtain the original 3D point cloud data, perform fitting and segmentation on the 3D point cloud data, and obtain the fitting plane;

[0056] This embodiment uses three-dimensional point cloud data of a building as an example for illustration. The specific segmentation method can be found in the specific method in step S101 of the aforementioned embodiment. In this step, the number of fitting planes obtained is at least one.

[0057] Please continue reading. Figure 3 as well as Figure 4 In one specific implementation, the segmentation method can also be based on the Random Sample Consensus (RANSAC) algorithm. Based on the idea of ​​RANSAC, the distance and normal information of points contained in the 3D point cloud data are obtained, and then iteratively fitted to each point cloud set to obtain the fitting plane corresponding to each point cloud set. Wherein, Figure 4 The color of the plane in the diagram is only used to identify the fitted plane to which different point clouds are iterated and should not be regarded as a restriction on the plane.

[0058] In one specific implementation, the iterative fitting steps include:

[0059] Step (1): Randomly select a point from the point cloud set as the initial seed point;

[0060] Step (2): Randomly select two more points from the point cloud set to form a planar model with the initial seed point;

[0061] Step (3): Obtain normal information of other points contained in the point cloud set except the above points, judge whether the normal angle of the points and the normal angle of the plane model satisfy the normal angle threshold, if yes, the current point is a support point, if not, the current point is a non-support point;

[0062] Step (4): Calculate the distance from other points contained in the point cloud set except the above points to the plane model, judge whether the distance satisfies the preset distance threshold, if yes, the current point is a support point, if not, the current point is a non-support point;

[0063] Step (5): Count the number of points that are support points in steps (3) and (4), if the number exceeds the preset minimum support point number, the plane model is regarded as a plane model meeting the conditions;

[0064] Step (6): Repeat steps (2)-(5) until a preset iteration number is reached or an optimal solution is obtained to obtain a plane model with the maximum number of support points, and the plane model is taken as the final fitting plane of the current point cloud set.

[0065] S202, generating a reference rectangle corresponding to the fitting plane based on the position information of the points in the three-dimensional point cloud data and the position information of the fitting plane;

[0066] The fitting plane obtained in the foregoing steps is a plane with edges, and its shape can be irregular. The reference rectangle generated based on the position information of the points in the corresponding three-dimensional point cloud data and the position information of the fitting plane can be a circumscribed minimum area rectangle.

[0067] In one specific embodiment, please refer to Figure 5 , Figure 5 The gray part in the figure is a schematic diagram of the fitting plane obtained in the foregoing steps, and the green rectangular frame is a schematic diagram of the circumscribed minimum area rectangle of the fitting plane.

[0068] S203, generating a projection plane corresponding to each fitting plane with the center of gravity of the fitting plane as the origin and the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the direction of the spatial coordinate axis;

[0069] S204, projecting the points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection plane respectively to obtain a point cloud plane;

[0070] Please continue to refer to Figure 5The space coordinate axis direction G-xyz is obtained by taking the gravity point G of the fitting plane as the origin, taking the normal direction of the fitting plane as the vertical axis, and taking the directions of the reference rectangular edges as the horizontal axis and the vertical axis, a projection plane corresponding to the fitting plane is generated based on the space coordinate axis, and the points in the three-dimensional point cloud data corresponding to the fitting plane are projected onto the projection plane based on the positions of the points.

[0071] S205, RGB color value information of each point in the point cloud plane is obtained, and the RGB color value information is fitted to obtain color channel values corresponding to each point cloud plane.

[0072] After the point cloud plane is projected, the color attribute information of each point in the point cloud plane is obtained and color texture fitting is performed to obtain a color texture image corresponding to each point cloud plane. In this embodiment, the color attribute information includes RGB color value information and transparency information.

[0073] In this step, the RGB color value information of each point in the point cloud plane is obtained, and the RGB color value information is fitted to obtain color channel values corresponding to each point cloud plane. Specifically, after the foregoing step, the points in each point cloud plane are in the same plane, please continue to refer to Figure 5 In a specific embodiment, it can be regarded as being in the plane z=0; then, the plane coordinate information of each point in the point cloud plane is obtained; the auxiliary point set data under different color channels is generated based on the plane coordinate information and the RGB color value information of each point; the B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surface under different color channels, and the fitted RGB color value surface is generated based on the color value surface under different color channels; and the color channel values corresponding to each point cloud plane are generated based on the fitted RGB color value surface.

[0074] Please refer to Figure 6 The image of a point cloud plane can be shown in the "original image" of Figure 6 , and then the gray scale images under different channels are obtained based on the RGB color value information of each point in the point cloud plane, such as Figure 6The R channel gray image, the G channel gray image, and the B channel gray image are shown. Then, according to the plane coordinates of each point in the point cloud plane as the x-axis and y-axis numerical coordinates of the color value information of each channel, and the gray color value of each point in the point cloud plane as the z'-axis numerical coordinate of the color value information of each channel, the auxiliary point set data of the RGB three channels are generated respectively. Then, the B-spline surface fitting algorithm is called to fit the auxiliary point set data under different color channels to obtain the color value surface under different color channels, and finally the color channel value corresponding to each point cloud plane is obtained. In this way, the gap part in the point cloud plane can be generated with the possible color according to the point with color information in the point cloud plane; in a specific embodiment, the fitted color value surface is specifically as shown in Figures 7-9

[0075] S206, obtaining the transparency information of each point in each point cloud plane, fitting the transparency information to obtain the transparency channel value corresponding to each point cloud plane;

[0076] Since there are actual gaps and gaps caused by errors in the point cloud, in this embodiment, the transparency information is fitted to reduce the gaps caused by errors and retain the actual existing gaps. Specifically, the transparency value of the point with color in the point cloud plane is recorded as 1, and the position without color value is recorded as 0; then, the similar method as in step S205 is performed, the plane coordinates of each point in the point cloud plane are taken as the x-axis and y-axis numerical coordinates of the color value information of each channel, and the transparency value of each point in the point cloud plane is taken as the z"-axis numerical coordinate to generate the auxiliary point set data of the transparency, and the B-spline surface fitting algorithm is called to fit the transparency information to obtain the fitted transparency surface, and the transparency value corresponding to each point cloud plane is obtained.

[0077] In this step, it also includes Gaussian blur and closing processing on the transparency value information corresponding to each point cloud plane, and the result is taken as the transparency channel value of the previous texture image; in this way, a small amount of transparent pixel points around the solid pixel (i.e. non-transparent pixel) can be changed into solid, so that the fitted pixel with hole after rendering in the original image with transparency is repaired, thereby completing the holes and gaps in the image.

[0078] S207, obtaining the color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value;

[0079] S208, filling each point cloud plane based on each color texture image to obtain the point cloud plane with color texture;

[0080] ​Based on the color channel value and the transparency channel value in step S205 and step S206, a color texture image corresponding to each point cloud plane is obtained after the supplementary hole and the smooth color are filled, and each point cloud plane is filled based on each color texture image to obtain a point cloud plane with color texture;

[0081] S209, the point cloud plane with color texture is triangularly divided to obtain a triangular mesh structure, and a three-dimensional model is generated based on the triangular mesh structure and the filled point cloud plane with color texture.

[0082] After obtaining a plurality of point cloud planes with color texture, a three-dimensional model is generated based on the filled point cloud plane with color texture. In this embodiment, when generating a three-dimensional model, the minimum circumscribed rectangle obtained in the foregoing steps is divided into triangles to construct a triangular network. Specifically, the vertices of each rectangle are numbered, and a specific example is described below. Please refer to Figure 10 In a specific embodiment, the vertices of a minimum circumscribed rectangle are numbered as 0, 1, 2 and 3, and in a segmentation manner, the minimum circumscribed rectangle can be divided into two triangles with vertex numbers (0, 1, 2) and (1, 3, 2). After constructing a triangular network for all the minimum circumscribed rectangles corresponding to the point cloud planes, a three-dimensional model with color texture is obtained.

[0083] Please refer to Figure 11 and Figure 12 which are two specific examples of the generated three-dimensional model with color texture. After the scheme in this embodiment, the three-dimensional model basically achieves the display effect of filling the gap hole.

[0084] The method in this embodiment can convert point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, reduce the data amount while ensuring that limited spatial information can be obtained, and solve the gap problem of point cloud display when constructing a three-dimensional model, thereby optimizing the display effect.

[0085] The three-dimensional model construction method based on point cloud in this embodiment is described above, and the three-dimensional model construction device based on point cloud in this embodiment is described below. Please refer to Figure 13 One embodiment of the three-dimensional model construction device based on point cloud in this embodiment includes:

[0086] The segmentation module 1301 is configured to obtain original three-dimensional point cloud data, segment and project the three-dimensional point cloud data, and obtain a point cloud plane, wherein the number of the point cloud planes is at least one.

[0087] The fitting module 1302 is configured to acquire color attribute information of each point in the point cloud plane and perform color texture fitting to obtain a color texture image corresponding to each point cloud plane.

[0088] The filling module 1303 is configured to fill each point cloud plane based on the color texture image.

[0089] The construction module 1304 is configured to construct a three-dimensional model based on the filled point cloud plane.

[0090] The method in the embodiment of the application can convert point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of point cloud data, reduce the data volume while ensuring that limited spatial information can be acquired, and solve the gap problem of point cloud display when constructing a three-dimensional model, and optimize the display effect.

[0091] Please refer to Figure 14 In another embodiment of the application, the segmentation module 1301 comprises:

[0092] The segmentation execution unit 13011 is configured to perform fitting segmentation on the three-dimensional point cloud data to obtain fitting planes, wherein the number of the fitting planes is at least one.

[0093] The rectangular fitting unit 13012 is configured to generate a reference rectangle corresponding to a fitting plane based on position information of points in the three-dimensional point cloud data and position information of the fitting plane.

[0094] The projection plane generation unit 13013 is configured to generate a projection plane corresponding to each fitting plane with the center of gravity of the fitting plane as the origin and with the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the direction of the spatial coordinate axis.

[0095] The projection execution unit 13014 is configured to project points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection plane to obtain a point cloud plane.

[0096] In another embodiment of the application, the segmentation execution unit 13011 is specifically configured to:

[0097] According to the position information in the three-dimensional point cloud data, a point cloud set is obtained by clustering, wherein the number of the point cloud sets is at least one.

[0098] According to distance information and normal information of points contained in the point cloud set, iterative fitting is performed on each point cloud set to obtain a fitting plane corresponding to each point cloud set.

[0099] In another embodiment of the application, the color attribute information comprises RGB color value information and transparency information.

[0100] The fitting module 1302 comprises:

[0101] A color fitting unit 13021 configured to acquire RGB color value information of each point in the point cloud plane, and fit the RGB color value information to obtain a color channel value corresponding to each point cloud plane;

[0102] A transparency fitting unit 13022 configured to acquire transparency information of each point in each point cloud plane, and fit the transparency information to obtain a transparency channel value corresponding to each point cloud plane;

[0103] A fitted image generation unit 13023 configured to obtain a color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value.

[0104] In another embodiment of the present application, the color fitting unit 13021 is specifically configured to:

[0105] acquire plane coordinate information of each point in the point cloud plane;

[0106] generate auxiliary point set data under different color channels according to the plane coordinate information and the RGB color value information of each point;

[0107] call a B-spline surface fitting algorithm to fit the auxiliary point set data under different color channels to obtain color value surfaces under different color channels, and generate a fitted RGB color value surface based on the fitted color value surfaces under different color channels;

[0108] generate the color channel value corresponding to each point cloud plane based on the fitted RGB color value surface.

[0109] In another embodiment of the present application, the filling module 1303 is specifically configured to:

[0110] fill each point cloud plane based on each color texture image to obtain a point cloud plane with color texture;

[0111] The construction module 1304 is specifically configured to:

[0112] triangulate the point cloud plane with color texture to obtain a triangular mesh structure;

[0113] generate a constructed three-dimensional model based on the triangular mesh structure and the filled point cloud plane with color texture.

[0114] The method in the embodiment of the present application can convert the point cloud into a three-dimensional model composed of multiple planes to solve the compression problem of the point cloud data, reduce the data volume while ensuring that limited spatial information can be obtained, and solve the gap problem of point cloud display when constructing the three-dimensional model, and optimize the display effect.

[0115] Based on the same inventive concept, the embodiment of the present specification also provides an electronic device based on point cloud three-dimensional model construction. The electronic device based on point cloud three-dimensional model construction in the embodiment of the present application is described in detail from the perspective of hardware processing.

[0116] Figure 15 The structure schematic diagram of an electronic device provided by the embodiment of the present specification is shown in FIG. 15. The electronic device 1500 according to the embodiment of the present application is described below with reference to FIG. 15. Figure 15 Figure 15 The electronic device 1500 shown in FIG. 15 is only an example, and should not bring any limitation to the function and use range of the embodiment of the present application.

[0117] As shown in FIG. 15, the electronic device 1500 is in the form of a general computing device. The components of the electronic device 1500 can include but are not limited to at least one processing unit 1510, at least one storage unit 1520, a bus 1530 connecting different system components including the storage unit 1520 and the processing unit 1510, a display unit 1540, etc. Figure 15 The storage unit stores program codes, which can be executed by the processing unit 1510, so that the processing unit 1510 performs the steps according to various exemplary embodiments of the present application described in the processing method part of the present specification. For example, the processing unit 1510 can perform the steps of the method as shown in FIG. 16 or 17.

[0118] Figure 1 Figure 2

[0119] The storage unit 1520 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 15201 and / or a cache memory unit 15202, and can further include a read-only memory (ROM) 15203.

[0120] The storage unit 1520 can also include program / utilities 15204 with a set of (at least one) program modules 15205, such as an operating system, one or more application programs, other program modules, and program data, each of which or some combination of which can include the implementation of a network environment.

[0121] ​​​​Bus 1530 can be one or more of several types of bus structure including a memory bus or memory controller, a peripheral bus, a graphics bus, a processor or local bus using any of a variety of bus architectures.

[0122] Electronic device 1500 can also communicate with one or more external devices 100 such as a keyboard or a pointing device, for example, by I / O interface 1550. Additionally, electronic device 1500 can communicate with one or more devices that enable user interaction with electronic device 1500, and / or one or more devices that enable communication of electronic device 1500 with other computing devices. Such communication can occur via I / O interface 1550. Still yet, electronic device 1500 can communicate with one or more networks (such as one or more local area networks (LANs), wide area networks (WANs), and / or the Internet, for example) via network adapter 1560. Network adapter 1560 can communicate with the other components of electronic device 1500 via bus 1530. It will be appreciated that various components (e.g., those not shown in FIG. 15) of electronic device 1500 can be used in conjunction with the example embodiments. Figure 15 It should be appreciated that FIG. 15 depicts software layers that can be comprised of instructions 1570 that enable the electronic device 1500 to perform the processes described herein. These software layers include at least one application layer 1572, at least one operating system layer 1574, and at least one device driver layer 1576. These software layers, can in some embodiments, be comprised of platform software 1578 that interfaces with hardware of electronic device 1500 to enable performance of the processes described herein. It should be appreciated that other hardware and / or software modules not shown in FIG. 15 can be used in conjunction with the example embodiments, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0123] From the above description of embodiments of the application, it is manifest that the example embodiments described herein can be implemented in software and / or in a combination of software and hardware, in many possible ways. Furthermore, it will be appreciated that the example embodiments described herein are provided by way of example only. Numerous different embodiments can be constructed while staying within the scope of the present application as defined by the appended claims. Figure 1 or Figure 2 the method shown in FIG. 15.

[0124] Figure 16 A schematic diagram of a computer readable medium embodying an example embodiment of the present application.

[0125] implementation Figure 1 or Figure 2A computer program in the method can be stored on one or more computer readable media. The computer readable media can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] The computer readable storage medium can include a computer-readable medium in the form of a data signal embodied in a carrier wave, wherein the data signal modulates an electromagnetic wave, a magnetic field, or other transport mechanism. The computer readable storage medium can also include any computer-readable medium excluding propagated signals per se. The computer readable storage medium can further be in the form of a non-transitory computer readable medium. The computer readable storage medium can be a computer readable storage medium excluding a transitory, propagating signal per se.

[0127] The program code can be executed by one or more programmable processors, which can be individually, or collectively, programmed to perform the operations described above. The program code can be implemented in any of various ways including one or more computer readable medium(s) that tangibly embody computer readable and / or instructions that, when executed by one or more programmable processor(s), perform the various operations described above. The computer readable medium(s) can be non-transitory computer readable medium(s). The one or more programmable processors can be one or more physical processors or processor cores.

[0128] In summary, the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that some or all of the functions of some or all of the components according to the embodiments of the present application can be implemented in practice using a general-purpose data processing device such as a microprocessor or a digital signal processor (DSP). The present application can also be implemented as a device or apparatus program (for example, a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0129] In addition, the present application also provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the point cloud-based three-dimensional model construction method as described in any of the above embodiments.

[0130] The above-described specific embodiments further illustrate the purposes, technical solutions and beneficial effects of the present application. It should be understood that the present application is not inherently related to any specific computer, virtual device or electronic device, and various general-purpose devices can implement the present application. The above description is merely for specific embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0131] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be mutually referred to. Each embodiment focuses on the differences from other embodiments.

[0132] The above description is merely for specific embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A point cloud based three-dimensional model construction method, characterized by, The method comprises the following steps: obtaining original three-dimensional point cloud data, fitting and segmenting the three-dimensional point cloud data to obtain fitting planes, wherein the number of the fitting planes is at least one; generating the minimum area rectangle circumscribed around the fitting planes based on the position information of the points in the three-dimensional point cloud data and the position information of the fitting planes to obtain corresponding reference rectangles; generating the projection planes corresponding to each fitting plane by taking the center of gravity of the fitting plane as the origin and taking the normal direction of the fitting plane and the direction of the edge of the reference rectangle as the spatial coordinate axis direction; projecting the points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection planes to obtain point cloud planes, wherein the number of the point cloud planes is at least one; obtaining the RGB color value information of each point in the point cloud planes, fitting the RGB color value information to obtain the color channel value corresponding to each point cloud plane; obtaining the transparency information of each point in each point cloud plane, fitting the transparency information to obtain the transparency channel value corresponding to each point cloud plane; obtaining the color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value; filling each point cloud plane based on each color texture image to obtain a point cloud plane with color texture; dividing each minimum area rectangle circumscribed around the fitting planes into triangles to perform triangular segmentation on the point cloud plane with color texture to obtain a triangular mesh structure; generating a completed three-dimensional model based on the triangular mesh structure and the filled point cloud plane with color texture.

2. The point cloud based three-dimensional model building method of claim 1, wherein, The method of fitting and segmenting the three-dimensional point cloud data to obtain the fitting planes comprises the following steps: performing clustering according to the position information in the three-dimensional point cloud data to obtain point cloud sets, wherein the number of the point cloud sets is at least one; performing iterative fitting on each point cloud set according to the distance information and normal information of the points contained in the point cloud set to obtain the fitting plane corresponding to each point cloud set. 3.The point cloud based three-dimensional model construction method of claim 1, wherein, The method of obtaining the RGB color value information of each point in the point cloud plane, fitting the RGB color value information, and obtaining the color channel value corresponding to each point cloud plane comprises the following steps: obtaining the plane coordinate information of each point in the point cloud plane; generating auxiliary point set data under different color channels according to the plane coordinate information and the RGB color value information of each point; calling a B-spline surface fitting algorithm to fit the auxiliary point set data under different color channels to obtain color value surfaces under different color channels, and generating a fitted RGB color value surface based on the fitted color value surfaces under different color channels; generating the color channel value corresponding to each point cloud plane based on the fitted RGB color value surface.

4. A point cloud-based three-dimensional model construction apparatus, characterized by comprising: The three-dimensional model construction device based on point cloud comprises: The segmentation module is configured to obtain original three-dimensional point cloud data, fit and segment the three-dimensional point cloud data, and obtain a fitting plane, wherein the number of the fitting planes is at least one; generate a minimum area rectangle circumscribed around the fitting plane based on position information of points in the three-dimensional point cloud data and position information of the fitting plane, and obtain a corresponding reference rectangle; generate a projection plane corresponding to each fitting plane with a center of gravity of the fitting plane as an origin, with a normal direction of the fitting plane and a direction of an edge of the reference rectangle as spatial coordinate axis directions; and project points in the three-dimensional point cloud data corresponding to each fitting plane onto the projection plane, and obtain a point cloud plane, wherein the number of the point cloud planes is at least one. The fitting module is configured to obtain RGB color value information of points in each point cloud plane, fit the RGB color value information, and obtain a color channel value corresponding to each point cloud plane; obtain transparency information of points in each point cloud plane, fit the transparency information, and obtain a transparency channel value corresponding to each point cloud plane; and obtain a color texture image corresponding to each point cloud plane based on the color channel value and the transparency channel value. The filling module is configured to fill each point cloud plane based on each color texture image, and obtain a point cloud plane with color texture. The construction module is configured to divide each minimum area rectangle circumscribed around the fitting plane into a triangle, perform triangle segmentation on the point cloud plane with color texture, and obtain a triangular mesh structure; and generate a constructed three-dimensional model based on the triangular mesh structure and the filled point cloud plane with color texture.

5. A point cloud based three-dimensional model construction device, characterized by, The point cloud-based three-dimensional model construction device includes a memory and at least one processor, and the memory stores instructions. The at least one processor invokes the instructions in the memory, so that the point cloud-based three-dimensional model construction device performs the steps of the point cloud-based three-dimensional model construction method in any one of claims 1-3.

6. A computer-readable storage medium having stored thereon computer programs / instructions, characterized in that, The program / instructions are executed by the processor to implement the steps of the point cloud-based three-dimensional model construction method in any one of claims 1-3.

7. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the point cloud-based three-dimensional model construction method in any one of claims 1-3.

Images