Point cloud processing method and apparatus, electronic device, and computer-readable storage medium

Abstract

Embodiments of the present application provide a point cloud processing method and device, electronic equipment and computer readable storage medium, relating to the field of image processing. The method extracts a weak texture region in an orthographic image corresponding to a surveying and mapping region, obtains point cloud data at the boundary of the weak texture region according to the boundary line of the weak texture region and original point cloud data corresponding to the surveying and mapping region, and fills the weak texture region with point cloud data according to the point cloud data at the boundary of the weak texture region. In this way, the point cloud data at the boundary of the weak texture region is used to fill the point cloud in the weak texture region, which can fill the holes in the original point cloud data, thereby effectively solving the problem of a large number of noise points and holes in the weak texture region of the point cloud data, and improving the recovery effect of the three-dimensional product.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular, to a point cloud processing method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] Weak texture areas are areas with only a small amount of texture or no texture, such as water surfaces, roads, and partial roofs, which are always a challenge for three-dimensional reconstruction based on aerial photogrammetry. Most existing three-dimensional reconstruction software (such as Pix4DMapper, Context Capture, etc.) does not have special processing for weak texture areas during three-dimensional reconstruction, resulting in three-dimensional products generated in weak texture areas with varying degrees of noise, layering, deformation, and holes.

[0003] Since most three-dimensional reconstruction software such as Pix4DMapper and Context Capture does not have a special mechanism to handle weak texture areas, although the calculation parameters can be adjusted, weak texture areas still cannot achieve a balance between noise and holes. For example, Pix4DMapper can reduce noise in weak texture areas by increasing the minimum number of matching images in the point cloud generation step, but this will result in the formation of holes in this area. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a point cloud processing method, device, electronic equipment and computer readable storage medium to solve the problem of a large number of noise and holes in point cloud data in weak texture areas in the prior art.

[0005] To achieve the above purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0006] In a first aspect, the present application provides a point cloud processing method, which comprises:

[0007] extracting a weak texture area in an orthophoto map corresponding to a survey area;

[0008] obtaining point cloud data at the boundary of the weak texture area according to the boundary line of the weak texture area and the original point cloud data corresponding to the survey area;

[0009] performing point cloud filling on the weak texture area according to the point cloud data at the boundary of the weak texture area.

[0010] In an optional implementation, the performing point cloud filling on the weak texture area according to the point cloud data at the boundary of the weak texture area comprises:

[0011] calculating the vertical direction of the weak texture area according to the point cloud data at the boundary of the weak texture area.

[0012] obtaining a normal direction of the weak-texture region according to the point cloud data at the boundary of the weak-texture region and the normal direction of the weak-texture region;

[0013] performing point cloud interpolation processing on the weak-texture region according to the plane equation of the weak-texture region, so as to complete the point cloud filling of the weak-texture region.

[0014] In an optional implementation, the point cloud interpolation processing on the weak-texture region comprises:

[0015] performing the point cloud interpolation processing on the weak-texture region according to a preset point cloud interpolation density or a resolution of the original point cloud data.

[0016] In an optional implementation, the obtaining of the normal direction of the weak-texture region according to the point cloud data at the boundary of the weak-texture region comprises:

[0017] performing distribution statistics on the point cloud data at the boundary of the weak-texture region, and selecting point cloud data within a preset range according to a statistical result;

[0018] calculating a normal vector of each point cloud in the point cloud data within the preset range;

[0019] calculating an average value of the normal vectors of the point clouds in the point cloud data within the preset range, so as to obtain the normal direction of the weak-texture region.

[0020] In an optional implementation, the method further comprises:

[0021] coloring each point cloud filled in the weak-texture region.

[0022] In an optional implementation, the extracting of the weak-texture region in the orthographic image corresponding to the surveying region comprises:

[0023] extracting the weak-texture region in the orthographic image corresponding to the surveying region according to a pixel value in the orthographic image corresponding to the surveying region or a pre-trained convolutional neural network, or selecting the weak-texture region in the orthographic image corresponding to the surveying region by a user.

[0024] In an optional implementation, before the extracting of the weak-texture region in the orthographic image corresponding to the surveying region, the method further comprises:

[0025] generating the orthographic image corresponding to the surveying region according to original point cloud data or original image data corresponding to the surveying region.

[0026] In a second aspect, the present application provides a point cloud processing device, the device comprising:

[0027] a weak-texture region extraction module configured to extract a weak-texture region in the orthographic image corresponding to the surveying region;

[0028] a point cloud selection module configured to obtain point cloud data at a boundary of the weak-texture region according to a boundary line of the weak-texture region and original point cloud data corresponding to the surveying region;

[0029] a point cloud filling module configured to perform point cloud filling on the weak-texture region according to the point cloud data at the boundary of the weak-texture region.

[0030] In a third aspect, the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the point cloud processing method according to any one of the preceding embodiments.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps of the point cloud processing method according to any one of the preceding embodiments.

[0032] The point cloud processing method, device, electronic device, and computer-readable storage medium provided by the embodiments of the present application extract a weak-texture region in the orthographic image corresponding to the surveying region, obtain point cloud data at a boundary of the weak-texture region according to a boundary line of the weak-texture region and original point cloud data corresponding to the surveying region, and perform point cloud filling on the weak-texture region according to the point cloud data at the boundary of the weak-texture region. In this way, the point cloud filling in the weak-texture region is performed using the point cloud data at the boundary of the weak-texture region, which can fill the holes in the original point cloud data, thereby effectively solving the problem that a large number of noise points and holes appear in the weak-texture region of the point cloud data, and improving the recovery effect of the three-dimensional product.

[0033] In order to make the above objectives, characteristics and advantages of the present application more apparent, clear and easy to understand, the following preferred embodiments are specifically described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be considered as a limitation to the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0035] Figure 1 A schematic diagram of point cloud holes is shown;

[0036] Figure 2 Fig. 1 shows a flow diagram of a point cloud processing method according to an embodiment of the present application;

[0037] Figure 3 Fig. 2 shows another flow diagram of a point cloud processing method according to an embodiment of the present application;

[0038] Figure 4 Fig. 3 shows a diagram of a boundary line of a weak texture region;

[0039] Figure 5 Fig. 4 shows another flow diagram of a point cloud processing method according to an embodiment of the present application;

[0040] Figure 6 Fig. 5 shows a diagram of a weak texture region after point cloud filling according to an embodiment of the present application;

[0041] Figure 7 Fig. 6 shows a functional module diagram of a point cloud processing device according to an embodiment of the present application;

[0042] Figure 8 Fig. 7 shows another functional module diagram of a point cloud processing device according to an embodiment of the present application;

[0043] Figure 9 Fig. 8 shows a block diagram of an electronic device according to an embodiment of the present application.

[0044] Fig. 9 shows a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0046] Therefore, the detailed description of the embodiments of the present application provided in the accompanying drawings below is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0047] It should be noted that the relational terms herein, such as first and second and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0048] A point cloud is a collection of points in a three-dimensional space, each of which is assigned a Cartesian coordinate. Points can also be assigned other attributes, which will typically indicate how they were obtained. For example, the time at which a measuring device collecting the data "saw" the point. It can also include the intensity or position error of the point.

[0049] In recent years, with the great development of three-dimensional image scanning technology, the advantages of point cloud model are more and more obvious. The graphics based on point cloud is more and more concerned, which has very wide application in the fields of medicine, modern satellite remote sensing measurement, multimedia, machine vision, intelligent monitoring, three-dimensional reconstruction, body sense interaction and 3D printing.

[0050] In the three-dimensional reconstruction based on vision, since there are often water surface, highway, part of roof and other weak texture areas in the surveying area, and most of the existing three-dimensional reconstruction software does not specially process the three-dimensional reconstruction of the weak texture area, resulting in that there are point cloud holes (such as Figure 1 indicated) in the point cloud data of the generated surveying area, which greatly affects the recovery effect of the three-dimensional product.

[0051] Based on this, the embodiment of the present application provides a point cloud processing method, device, electronic equipment and computer readable storage medium, which uses the point cloud data at the boundary of the weak texture area to fill the point cloud in the weak texture area, can fill the holes in the original point cloud data, thereby effectively solving the problem that the point cloud data in the weak texture area will appear a large number of noise points and holes, and improving the recovery effect of the three-dimensional product. Next, the point cloud processing method provided by the embodiment of the present application will be described in detail.

[0052] Please refer to Figure 2 , a flowchart of the point cloud processing method provided by the embodiment of the present application. It should be noted that the point cloud processing method of the embodiment of the present application does not necessarily include all the steps shown in the flowchart. Figure 2and the specific sequence below is limited, it should be understood that in other embodiments, the point cloud processing method of the embodiment of the present application can exchange the sequence of part of the steps according to actual needs, or part of the steps can be omitted or deleted. The point cloud processing method can be applied in an electronic device, and the specific process shown below will be described in detail. Figure 2

[0053] Step S201, extracting a weak texture region in the orthographic image corresponding to the surveying area.

[0054] In this embodiment, the surveying area is the area that needs to be surveyed; the DOM (Digital Orthophoto Map, orthographic image) is a digital differential correction and mosaic of aerospace photographs, and is cut into a digital orthographic image set according to a certain sheet range. It is an image that has both map geometric accuracy and image features. DOM has the advantages of high precision, rich information, intuitive and realistic, and fast acquisition.

[0055] After the electronic device obtains the orthographic image corresponding to the surveying area, it can extract the weak texture region in the orthographic image using a deep learning network. The weak texture region can be a region without texture or a region with only a small amount of texture. Deep learning (DL) is a branch of machine learning, which is an algorithm that uses artificial neural networks as the architecture to learn the representation of data.

[0056] Step S202, obtaining point cloud data at the boundary of the weak texture region according to the boundary line of the weak texture region and the original point cloud data corresponding to the surveying area.

[0057] In this embodiment, the surveying area can be photographed by a aerial photography device, and three-dimensional reconstruction can be performed according to the photographed two-dimensional image to obtain the original point cloud data corresponding to the surveying area.

[0058] After the electronic device extracts the weak texture region, it can superimpose the boundary line of the weak texture region and the original point cloud data corresponding to the surveying area to obtain the point cloud data at the boundary of the weak texture region.

[0059] Step S203, filling the point cloud of the weak texture region according to the point cloud data at the boundary of the weak texture region.

[0060] In this embodiment, the electronic device fills the point cloud in the weak texture region using the point cloud data at the boundary of the weak texture region, which can fill the holes in the original point cloud data, thereby effectively solving the problem of a large number of noise points and holes in the point cloud data in the weak texture region, and improving the recovery effect of the three-dimensional product.

[0061] ​It can be seen that the point cloud processing method provided in the embodiment of the present application extracts the weak texture region in the orthographic image corresponding to the surveying and mapping region, obtains the point cloud data at the boundary of the weak texture region according to the boundary line of the weak texture region and the original point cloud data corresponding to the surveying and mapping region, and fills the point cloud in the weak texture region according to the point cloud data at the boundary of the weak texture region. In this way, the point cloud in the weak texture region is filled by using the point cloud data at the boundary of the weak texture region, the holes in the original point cloud data can be filled, and the problem that a large number of noise points and holes may appear in the weak texture region of the point cloud data is effectively solved, thereby improving the recovery effect of the three-dimensional product.

[0062] In actual application, there are many ways to produce the orthographic image, please refer to Figure 3 Before step S201, the point cloud processing method provided in the embodiment of the present application can further include:

[0063] In step S301, the orthographic image corresponding to the surveying and mapping region is generated according to the original point cloud data or the original image data corresponding to the surveying and mapping region.

[0064] In the embodiment, the original image data corresponding to the surveying and mapping region can include a plurality of two-dimensional images photographed by the aerial photography device for the surveying and mapping region, and the original point cloud data can be obtained by three-dimensional reconstruction according to the original image data corresponding to the surveying and mapping region.

[0065] The orthographic image of the surveying and mapping region can be generated based on the original point cloud data or the original image data photographed. For example, when the orthographic image is generated based on the original point cloud data, the original point cloud data corresponding to the surveying and mapping region can be projected to the orthographic plane, the image coordinates of each projection point are calculated, and finally the orthographic image corresponding to the surveying and mapping region is generated. When the orthographic image is generated based on the original image data, the orthographic image can be generated by using the stereo imaging of aerial images or the single sheet differential correction method, or the orthographic image can be generated by using the stereo pair of satellite images.

[0066] In the embodiment, the weak texture region can be extracted based on different methods. That is, step S201 includes: extracting the weak texture region in the orthographic image according to the pixel value in the orthographic image corresponding to the surveying and mapping region or the pre-trained convolutional neural network, or selecting the weak texture region in the orthographic image corresponding to the surveying and mapping region by the user.

[0067] That is to say, in an embodiment, after obtaining the orthographic image corresponding to the surveying area, the user can select the weak texture area in the orthographic image and draw the boundary line of the weak texture area. In another embodiment, in order to avoid the complexity of manually drawing or importing other related software to identify the boundary of the weak texture area, the electronic device can extract the weak texture area in the orthographic image according to the pixel value in the orthographic image corresponding to the surveying area or a pre-trained convolutional neural network.

[0068] In the process of extracting the weak texture area by using the pixel value in the orthographic image, the pixel in the orthographic image is traversed, and when the current pixel value is greater than a preset value, the pixel value of the neighborhood pixel is judged by using the region growing method. If the pixel value of the neighborhood pixel is greater than the preset value, the outward diffusion is continued until the pixel value is less than the preset value. By this method, the weak texture area in the orthographic image can be obtained. The preset value can be set according to actual needs, for example, set to 240.

[0069] In the process of extracting the weak texture area in the orthographic image by using the pre-trained convolutional neural network, the orthographic image corresponding to the surveying area can be input into the pre-trained convolutional neural network, and the weak texture area in the orthographic image can be extracted by using the convolutional neural network.

[0070] In this embodiment, a deep learning convolutional neural network can be pre-trained for extracting the weak texture area in the orthographic image. The trained convolutional neural network is stored in the electronic device. When the weak texture area needs to be extracted, the orthographic image corresponding to the surveying area is input into the convolutional neural network for processing. Finally, the weak texture area in the orthographic image and the category to which the weak texture area belongs are output by the convolutional neural network.

[0071] In an embodiment, the training and application process of the deep learning convolutional neural network can include:

[0072] (1) The sample labeling of the weak texture area is performed by using Labelme or other labeling tools. Not only the position but also the type such as water area, strong light road surface or weak texture roof is labeled.

[0073] (2) The data augmentation is performed on the original data and the label data to increase the data amount for training the convolutional neural network. The augmentation methods include cropping, rotating, flipping, scaling and adding noise.

[0074] (3) The prepared data set is divided into three parts: training set, validation set and test set according to a certain ratio, for example, 6:2:2. Firstly, the training set data is sent into the convolutional neural network, which generally includes an input layer, a convolutional layer, a pooling layer and a full connection layer. The convolutional network is used to assign an initial class label to each pixel. The convolutional layer can effectively capture the local features in the image and nest many such modules together in a hierarchical manner. After the training of the convolutional neural network is completed, the validation set is used to evaluate the convolutional neural network, and the best set of hyperparameters is selected for testing on the test set. In the process of continuously adjusting the hyperparameters, the convolutional neural network is optimized on the test set.

[0075] (4) The optimal convolutional neural network is used to complete the classification of the weak texture area in the orthographic image, so as to obtain the weak texture area in the orthographic image and the category to which the weak texture area belongs, and the boundary line of the weak texture area in the orthographic image is obtained by vectorizing the classification result (as shown in FIG. 6). Figure 4

[0076] It can be seen that the point cloud processing method provided in the embodiments of the present application can extract the weak texture area in the orthographic image by using different methods, and the weak texture area in the orthographic image can be extracted by using the pixel value in the orthographic image or by using the convolutional neural network of deep learning, so that the tedious process of manually drawing or importing other related software to identify the boundary of the weak texture area is effectively avoided.

[0077] It should be noted that in actual application, considering that the weak texture area has different types, different methods can be used to extract the weak texture area of different types. For example, the pixel intensity value can be used to extract the strong light reflection area. For the water area, roof and solar reflector and the like, the convolutional neural network of deep learning is used to complete the extraction.

[0078] In an implementation manner, the electronic device can fit a plane equation according to the point cloud data at the boundary of the weak texture area, and complete the point cloud filling of the weak texture area based on the fitted plane equation. Based on this, the electronic device fills the point cloud of the weak texture area according to the point cloud data at the boundary of the weak texture area in the step S203, which can specifically include:

[0079] According to the point cloud data at the boundary of the weak texture area, the vertical direction of the weak texture area is calculated, the plane equation of the weak texture area is obtained according to the vertical direction of the weak texture area and the point cloud data at the boundary of the weak texture area, and the point cloud interpolation processing of the weak texture area is performed according to the plane equation of the weak texture area, so as to complete the point cloud filling of the weak texture area.

[0080] ​In this embodiment, the electronic device can calculate the vertical direction of the weak-texture region according to the point cloud data at the boundary of the weak-texture region According to the point cloud data at the boundary of the weak-texture region (i.e., the coordinates of each point cloud around the weak-texture region) and the vertical direction of the weak-texture region Perform plane fitting to obtain the plane equation ax+by+cz+d=0 of the weak-texture region, and perform point cloud interpolation processing on the weak-texture region based on the plane equation to complete the point cloud filling of the weak-texture region.

[0081] In the point cloud interpolation processing on the weak-texture region, the electronic device can perform point cloud interpolation processing on the weak-texture region according to a preset point cloud interpolation density or the resolution of the original point cloud data.

[0082] For example, when the point cloud interpolation density is not preset in the electronic device, the interpolation can be performed according to the resolution of the original point cloud data to obtain a suitable point cloud density.

[0083] Optionally, when calculating the vertical direction of the weak-texture region according to the point cloud data at the boundary of the weak-texture region, the electronic device can specifically include: performing distribution statistics on the point cloud data at the boundary of the weak-texture region, and selecting point cloud data within a preset range according to the statistical result; calculating the normal vector of each point cloud in the point cloud data within the preset range; calculating the average value of the normal vectors of the point clouds in the point cloud data within the preset range to obtain the vertical direction of the weak-texture region.

[0084] For example, the electronic device can perform normal distribution statistics on the point cloud data at the boundary of the weak-texture region, extract point cloud data within a two-standard-deviation range (or other number of standard deviation ranges) as a reference according to the statistical result, calculate the normal vector of each point cloud in the point cloud data within the two-standard-deviation range, and calculate the average value of the normal vectors of the point clouds to obtain the vertical direction of the weak-texture region.

[0085] Optionally, referring to Figure 5 After completing the point cloud filling of the weak-texture region, the electronic device can further perform the following steps:

[0086] In step S501, each point cloud filled in the weak-texture region is colored.

[0087] The electronic device can use the pixel value of the corresponding position in the DOM to color the point cloud in the weak-texture region. For example, for each point cloud filled in the weak-texture region, the point cloud can be colored according to the pixel value at the corresponding position of the point cloud in the orthographic image.

[0088] That is to say, when coloring each point cloud in the weak texture region, the pixel value of the corresponding position of the point cloud in the orthographic image is obtained, and the pixel value is taken as the color value of each point cloud, so as to complete the coloring of the point cloud in the weak texture region.

[0089] It can be seen that in the point cloud processing method provided by the embodiment of the application, the electronic device finds the point cloud data at the boundary of the weak texture region in the original point cloud data, performs distribution statistics on the point cloud data at the boundary of the weak texture region, obtains the point cloud data within the two standard deviation ranges as a reference, calculates the normal vector of the point cloud data within the two standard deviation ranges, obtains the average value of the normal vector as the vertical direction of the weak texture region, fits the plane equation according to the point cloud coordinates around the weak texture region and the vertical direction, and then performs point cloud interpolation according to the pre-set point cloud interpolation density or the resolution of the original point cloud data, and the color value of the point cloud is taken as the pixel value of the corresponding position of the point cloud in the orthographic image, so as to complete the point cloud filling in the weak texture region (as shown in Figure 6 , effectively solving the problem that a large number of noise points and holes appear in the weak texture region of the point cloud data, and improving the recovery effect of the three-dimensional product.

[0090] In order to perform the corresponding steps in the above-mentioned embodiments and various possible manners, an implementation manner of a point cloud processing device is given below. Please refer to Figure 7 A functional module diagram of the point cloud processing device 600 provided by the embodiment of the application is shown in the figure. It should be noted that the basic principle and the generated technical effect of the point cloud processing device 600 provided by the embodiment are the same as those of the above-mentioned embodiments, and for brief description, the part not mentioned in the embodiment can be referred to the corresponding content in the above-mentioned embodiments. The point cloud processing device 600 includes a weak texture region extraction module 610, a point cloud selection module 620 and a point cloud filling module 630.

[0091] The weak texture region extraction module 610 is configured to extract the weak texture region in the orthographic image corresponding to the surveying and mapping region.

[0092] It can be understood that the weak texture region extraction module 610 can perform the above-mentioned step S201.

[0093] The point cloud selection module 620 is configured to obtain the point cloud data at the boundary of the weak texture region according to the boundary line of the weak texture region and the original point cloud data corresponding to the surveying and mapping region.

[0094] It can be understood that the point cloud selection module 620 can perform the above-mentioned step S202.

[0095] The point cloud filling module 630 is configured to perform point cloud filling on the weak texture region according to the point cloud data at the boundary of the weak texture region.

[0096] It can be understood that the point cloud filling module 630 can perform the above step S203.

[0097] Optionally, the point cloud filling module 630 is specifically configured to calculate a vertical direction of the weak texture region according to the point cloud data at the boundary of the weak texture region; obtain a plane equation of the weak texture region according to the vertical direction of the weak texture region and the point cloud data at the boundary of the weak texture region; and perform point cloud interpolation processing on the weak texture region according to the plane equation of the weak texture region, so as to complete the point cloud filling of the weak texture region.

[0098] Optionally, the point cloud filling module 630 is further specifically configured to perform the point cloud interpolation processing on the weak texture region according to a preset point cloud interpolation density or a resolution of the original point cloud data.

[0099] Optionally, the point cloud filling module 630 is further specifically configured to perform distribution statistics on the point cloud data at the boundary of the weak texture region, and select point cloud data within a preset range according to the statistical result; calculate a normal vector of each point cloud in the point cloud data within the preset range; and calculate an average value of the normal vectors of the point clouds in the point cloud data within the preset range to obtain the vertical direction of the weak texture region.

[0100] Optionally, the weak texture region extraction module 610 is specifically configured to extract the weak texture region in the orthographic image corresponding to the surveying region according to a pixel value in the orthographic image corresponding to the surveying region or a pre-trained convolutional neural network, or extract the weak texture region in the orthographic image corresponding to the surveying region by a user.

[0101] Optionally, referring to Figure 8 The point cloud processing apparatus 600 provided by the embodiment of the present application can further include an orthographic image generation module 640 and a point cloud coloring module 650, the orthographic image generation module 640 being configured to generate an orthographic image corresponding to a surveying region according to original point cloud data or original image data corresponding to the surveying region.

[0102] It can be understood that the orthographic image generation module 640 can perform the above step S301.

[0103] The point cloud coloring module 650 is configured to color each point cloud filled in the weak texture region. Specifically, the point cloud coloring module 650 is configured to color each point cloud filled in the weak texture region according to a pixel value at a corresponding position of the point cloud in the orthographic image.

[0104] It can be understood that the point cloud coloring module 650 can perform the above step S501.

[0105] The point cloud processing device provided by the embodiment of the present application comprises a weak texture area extraction module, a point cloud selection module and a point cloud filling module, the weak texture area extraction module is used to extract a weak texture area in an orthographic image corresponding to a surveying and mapping area, the point cloud selection module is used to obtain point cloud data at a boundary of the weak texture area according to a boundary line of the weak texture area and original point cloud data corresponding to the surveying and mapping area, and the point cloud filling module is used to fill the point cloud in the weak texture area according to the point cloud data at the boundary of the weak texture area. In this way, the point cloud filling in the weak texture area is performed by using the point cloud data at the boundary of the weak texture area, the holes in the original point cloud data can be filled, and thus the problem that a large number of noise points and holes may appear in the point cloud data in the weak texture area is effectively solved, and the recovery effect of a three-dimensional product is improved.

[0106] Please refer to Figure 9 A block schematic diagram of the electronic device 700 provided by the embodiment of the present application is shown in FIG. 1. The electronic device can be a PC (Personal Computer), a server, or a device such as an aircraft or an unmanned aerial vehicle, and the embodiment is not limited thereto. The electronic device 700 comprises a memory 710, a processor 720 and a communication module 730. The memory 710, the processor 720 and the communication module 730 are electrically connected to each other directly or indirectly to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0107] The memory 710 is used to store programs or data. The memory 710 can be, but is not limited to, a RAM (Random Access Memory), a ROM (Read Only Memory), a PROM (Programmable Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electric Erasable Programmable Read-Only Memory) and the like.

[0108] The processor 720 is used to read / write the data or programs stored in the memory 710 and perform corresponding functions. For example, when the computer program stored in the memory 710 is executed by the processor 720, the point cloud processing method disclosed in the above embodiments can be realized.

[0109] The communication module 730 is used to establish a communication connection between the electronic device 700 and other communication terminals through a network and is used to receive and transmit data through the network.

[0110] It should be understood that, Figure 9 The structure shown is only a schematic diagram of the electronic device 700. The electronic device 700 may also include components that are larger than those shown. Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown. Figure 9 The components shown can be implemented using hardware, software, or a combination thereof.

[0111] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor 720, implements the point cloud processing method disclosed in the above embodiments.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0113] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0114] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0115] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A point cloud processing method, characterized in that, The method includes: Extract the weak texture regions from the orthophoto map corresponding to the surveyed area; Based on the boundary line of the weak texture region and the original point cloud data corresponding to the surveying area, the point cloud data at the boundary of the weak texture region is obtained; The weak texture region is filled with point cloud data based on the point cloud data at its boundary. This filling process includes: calculating the vertical direction of the weak texture region based on the point cloud data at its boundary; obtaining the plane equation of the weak texture region based on its vertical direction and the point cloud data at its boundary; and performing point cloud interpolation processing on the weak texture region based on its plane equation to complete the point cloud filling.

2. The method according to claim 1, characterized in that, The point cloud interpolation processing of the weakly textured region includes: Point cloud interpolation processing is performed on the weak texture region according to the preset point cloud interpolation density or the resolution of the original point cloud data.

3. The method according to claim 1, characterized in that, The step of calculating the vertical direction of the weakly textured region based on the point cloud data at the boundary of the weakly textured region includes: The distribution statistics of point cloud data at the boundary of the weak texture region are performed, and point cloud data within a preset range are selected based on the statistical results. Calculate the normal vector of each point cloud in the point cloud data within the preset range; The average value of the normal vectors of each point cloud in the point cloud data within the preset range is calculated to obtain the vertical direction of the weak texture region.

4. The method according to claim 1, characterized in that, The method further includes: Color each point cloud filled in the weakly textured region.

5. The method according to claim 1, characterized in that, The extraction of weak texture regions from the orthophoto map corresponding to the surveyed area includes: Weak texture regions in the orthophoto map can be extracted based on pixel values ​​in the orthophoto map corresponding to the surveyed area or by a pre-trained convolutional neural network, or by the user selecting weak texture regions in the orthophoto map corresponding to the surveyed area.

6. The method according to any one of claims 1-5, characterized in that, Before extracting the weak texture regions from the orthophoto map corresponding to the surveyed area, the method further includes: Based on the original point cloud data or original image data corresponding to the surveyed area, an orthophoto map corresponding to the surveyed area is generated.

7. A point cloud processing device, characterized in that, The device includes: The weak texture region extraction module is used to extract weak texture regions in the orthophoto map corresponding to the surveyed area; The point cloud selection module is used to obtain point cloud data at the boundary of the weak texture region based on the boundary line of the weak texture region and the original point cloud data corresponding to the surveying area; The point cloud filling module is used to fill the weak texture region with point cloud data at the boundary of the weak texture region; the point cloud filling module is used to calculate the vertical direction of the weak texture region based on the point cloud data at the boundary of the weak texture region; obtain the plane equation of the weak texture region based on the vertical direction of the weak texture region and the point cloud data at the boundary of the weak texture region; and perform point cloud interpolation processing on the weak texture region based on the plane equation of the weak texture region, thereby completing the point cloud filling of the weak texture region.

8. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the point cloud processing method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the point cloud processing method as described in any one of claims 1-6.

Images