A 3D model construction method based on deep learning

Abstract

The present invention provides a three-dimensional model construction method based on deep learning, which belongs to the field of three-dimensional modeling. The method comprises acquiring point cloud data of an object to be modeled, wherein the point cloud data of the object to be modeled comprises sparse point cloud data and dense point cloud data; performing voxel segmentation on the object to be modeled to obtain at least one submodule and the sparse point cloud data and dense point cloud data belonging to the submodule; inputting all submodules into a pre-constructed classification neural network model for feature matching, and outputting matching results; based on the matching results, selecting the sparse point cloud data or the dense point cloud data of the submodule for three-dimensional model construction. The present invention divides the object to be modeled to form submodules, and matches them with the classification neural network, and intelligently selects appropriate point cloud data types for the submodules when constructing the three-dimensional model, so as to fully utilize the advantages of different point cloud data in the modeling process and improve the efficiency of modeling.

Description

Technical Field

[0001] The present invention relates to the technical field of model construction, and in particular to a three-dimensional model construction method based on deep learning. Background Art

[0002] In the process of building three-dimensional models, point cloud technology is often used. It is a technology for collecting, processing and analyzing three-dimensional spatial data. Through the data obtained by laser scanning, photogrammetry or other sensors, the surface geometry of objects in the real world is converted into a large number of discrete point coordinates, thereby creating a three-dimensional point cloud model; the data obtained by point cloud technology includes dense point cloud and sparse point cloud. Among them, dense point cloud can provide more sampling points, which can more accurately capture details such as buildings, roads, green areas, and the geometric relationships between them, which is very important for creating accurate three-dimensional models and planning, analysis and visualization. However, dense point clouds usually require powerful computing resources and algorithms for processing and analysis, with low efficiency and requiring large storage space; sparse point clouds have fewer sampling points and only have sampling points on the main features or boundaries of objects. Therefore, sparse point clouds cannot provide enough details and the modeling accuracy is insufficient, but sparse point clouds have relatively few data points in computing and storage.

[0003] Therefore, it is necessary to provide a three-dimensional model construction method based on deep learning to solve the above technical problems. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a three-dimensional model construction method based on deep learning, which divides the object to be modeled into sub-modules and matches them with a classification neural network. When constructing a three-dimensional model for the sub-module, the appropriate point cloud data type is intelligently selected to fully utilize the advantages of different point cloud data in the modeling process, thereby obtaining a more accurate and complete three-dimensional model and improving the efficiency of modeling.

[0005] The present invention provides a three-dimensional model construction method based on deep learning, and the construction method comprises the following steps:

[0006] Acquire point cloud data of the object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data;

[0007] Performing voxel segmentation on the object to be modeled to obtain at least one submodule and sparse point cloud data and dense point cloud data belonging to the submodule;

[0008] Input all submodules into the pre-built classification neural network model for feature matching and output the matching results;

[0009] Based on the matching result, the sparse point cloud data or the dense point cloud data of the submodule is selected to construct the three-dimensional model.

[0010] Preferably, the step of acquiring point cloud data of the object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data, comprises:

[0011] Collecting original point cloud data of the object to be modeled, wherein the original point cloud data includes original sparse point cloud data and original dense point cloud data;

[0012] Preprocessing the original point cloud data of the object to be modeled, wherein the preprocessing includes cleaning, registering, filtering and correcting the point cloud data;

[0013] Output preprocessed point cloud data.

[0014] Preferably, the voxel segmentation of the object to be modeled to obtain at least one submodule and sparse point cloud data and dense point cloud data belonging to the submodule includes:

[0015] Discretizing the object to be modeled into voxels of uniform size, wherein each voxel represents a cubic unit of the object to be modeled;

[0016] Classify all voxels based on the relationship between them;

[0017] The boundaries of the classified voxels are determined and the boundaries are segmented to obtain submodules belonging to the object to be modeled and sparse point cloud data and dense point cloud data belonging to the submodules.

[0018] Preferably, the step of inputting all submodules into a pre-built classification neural network model for feature matching and outputting matching results includes:

[0019] Constructing a classification neural network model, wherein the classification neural network is trained by using sample submodules and their corresponding point cloud labels;

[0020] Perform feature extraction on all sub-modules to obtain feature information corresponding to each sub-module;

[0021] The feature information of all submodules is used as input into the trained classification neural network model, and the classification neural network model performs matching based on the input feature information to obtain a matching result, wherein the matching result is represented by a matching probability, a matching score or a matching label.

[0022] Preferably, the selecting the sparse point cloud data or the dense point cloud data of the submodule to construct the three-dimensional model based on the matching result includes:

[0023] Obtain matching results, select the best matching sample submodule from the matching results of each submodule, and read the point cloud label of the sample submodule;

[0024] Based on the point cloud label of the sample submodule that best matches the submodule, select sparse point cloud data or dense point cloud data with the same point cloud label;

[0025] The three-dimensional model is constructed based on the sparse point cloud data or dense point cloud data selected by each sub-module.

[0026] Compared with the related art, the three-dimensional model construction method based on deep learning provided by the present invention has the following beneficial effects:

[0027] The present invention obtains sparse and dense point cloud data of the object to be modeled and performs voxel segmentation on it to obtain multiple sub-modules and the point cloud data belonging to them. After inputting a pre-built classification neural network model for feature matching, the present invention can intelligently select the appropriate point cloud data type for three-dimensional model construction, effectively utilizing the advantages of different point cloud data types, improving modeling efficiency, and making the final three-dimensional model more accurate and complete. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of a three-dimensional model construction method based on deep learning provided by the present invention;

[0029] Figure 2 S1 flow chart of a three-dimensional model construction method based on deep learning provided by the present invention;

[0030] Figure 3 S2 flow chart of a three-dimensional model construction method based on deep learning provided by the present invention;

[0031] Figure 4 An S3 flow chart of a three-dimensional model construction method based on deep learning provided by the present invention;

[0032] Figure 5 An S4 flow chart of a three-dimensional model construction method based on deep learning provided by the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0034] Embodiment 1

[0035] refer to Figure 1 As shown, a three-dimensional model construction method based on deep learning includes the following steps:

[0036] S1: Acquire point cloud data of an object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data.

[0037] In this embodiment, the point cloud data of the object to be modeled is mainly acquired by laser scanning. During the acquisition process, a laser and a receiver are used to measure the distance and position information of the object to be modeled. Specifically, a laser beam is emitted to the object to be modeled, and then the time difference and angle of the reflected signal are calculated to determine the position of each point in the point cloud, thereby finally generating point cloud data.

[0038] It is also necessary to obtain point cloud data of two different densities, sparse and dense, which can more comprehensively reflect the shape and surface features of the object to be modeled. Acquiring two types of point cloud data at the same time can better meet diverse modeling needs.

[0039] In subsequent modeling, you can choose to use sparse point cloud data or dense point cloud data for modeling according to actual needs. Sparse point cloud data is suitable for scenes that require rapid construction of rough shapes, while dense point cloud data is suitable for scenes that require high-precision surface details.

[0040] S2: performing voxel segmentation on the object to be modeled to obtain at least one submodule and sparse point cloud data and dense point cloud data belonging to the submodule.

[0041] In this embodiment, the object to be modeled is voxelized and divided into a series of small cubic units, and then the point cloud data (including dense point cloud data and sparse point cloud data) is mapped to these voxels, so that each voxel contains dense point cloud data and sparse point cloud data, that is, the object to be modeled includes voxels of dense point cloud data and voxels of sparse point cloud data, and then these voxels are combined according to the relationship to obtain sub-modules, then the dense point cloud data and sparse point cloud data configured in the voxels will be correspondingly summarized into the sub-modules, and finally, the same sub-module of the object to be modeled is divided into two categories, that is, the sub-module of dense point cloud data and the sub-module of sparse point cloud data.

[0042] S3: Input all submodules into the pre-built classification neural network model for feature matching and output the matching results.

[0043] In this embodiment, all sub-modules obtained after voxel segmentation are used as input data and input into a pre-built classification neural network model. Each sub-module carries its corresponding sparse point cloud data and dense point cloud data. Inside the classification neural network model, it will perform feature extraction on each sub-module, extract representative feature vectors, and use the feature vectors of the sub-modules to perform feature matching with the classification neural network model to obtain matching results.

[0044] S4: Based on the matching result, the sparse point cloud data or the dense point cloud data of the submodule is selected to construct a three-dimensional model.

[0045] In this embodiment, a matching result is obtained in step S3. By analyzing the matching result, it can be decided whether each submodule chooses to use sparse point cloud data or dense point cloud data to construct a three-dimensional model. This is because in practical applications, different submodules may have different modeling requirements. Some submodules may pay more attention to the expression of the overall shape and structure, while other submodules may require finer surface details. In this way, the focus of each submodule can be obtained according to the classification neural network model, and appropriate point cloud data can be selected for the submodule.

[0046] In this embodiment, reference Figure 2 As shown, step S1 specifically includes:

[0047] S101: Collecting original point cloud data of the object to be modeled, wherein the original point cloud data includes original sparse point cloud data and original dense point cloud data.

[0048] In this embodiment, laser scanning is used to acquire the original point cloud data of the object to be modeled, and the object to be modeled needs to be scanned based on the original sparse point cloud data and the original dense point cloud data to meet the requirements for different point cloud data categories in subsequent modeling.

[0049] S102: Preprocessing the original point cloud data of the object to be modeled, wherein the preprocessing includes cleaning, registering, filtering and correcting the point cloud data.

[0050] In this embodiment, data cleaning mainly includes removing isolated points, noise filtering and surface smoothing; for point clouds collected from multiple positions, they need to be initially aligned, and based on the initial alignment results, an iterative optimization algorithm is used to minimize the distance between corresponding points to achieve the purpose of registration; filtering is achieved by reducing the number of sampling points in the point cloud data to reduce the amount of data, which can then reduce the amount of data and improve computing efficiency while maintaining the structural features of the point cloud; correction is achieved through normal vector estimation, which calculates the normal vector of each point for subsequent operations such as lighting calculation, surface reconstruction, etc.

[0051] S103: Output the pre-processed point cloud data.

[0052] In this embodiment, after the original point cloud data is cleaned, registered, filtered and corrected, the required point cloud data can be obtained.

[0053] In this embodiment, reference Figure 3 As shown, step S2 specifically includes:

[0054] S201: discretize the object to be modeled into voxels of uniform size, wherein each voxel represents a cubic unit of the object to be modeled.

[0055] In this embodiment, the space of the object to be modeled is discretized into a series of small cubic units of uniform size, namely voxels. This process divides the space evenly and ensures that each voxel has the same size and shape.

[0056] S202: Classify all voxels based on the relationship between voxels.

[0057] In this embodiment, all voxels are classified according to factors such as the spatial relationship between voxels and the distribution characteristics of point cloud data, and the classification is based on the density, shape, position and other characteristics of the voxels. Through the classification operation, voxels with similar characteristics can be grouped into one category, providing a basis for subsequent sub-module division.

[0058] S203: Determine the boundaries of the classified voxels and perform boundary segmentation to obtain submodules belonging to the object to be modeled and sparse point cloud data and dense point cloud data belonging to the submodules.

[0059] In this embodiment, after the classification is completed, the boundaries of voxels of each category are determined, and the boundary segmentation is performed by identifying the interface between adjacent voxels, so that the object to be modeled is divided into several independent sub-modules. At the same time, since each voxel contains sparse point cloud data and dense point cloud data, these point cloud data will also be divided into corresponding sub-modules according to the attribution relationship of the voxels.

[0060] In this embodiment, reference Figure 4 As shown, step S3 specifically includes:

[0061] S301: Construct a classification neural network model, wherein the classification neural network is trained through sample submodules and their corresponding point cloud labels.

[0062] In this embodiment, during construction, a certain number of sample submodules and their corresponding point cloud labels are collected. These sample submodules should be representative and can cover various possible forms and features of the object to be modeled. The point cloud labels are used to indicate the type of point cloud data (sparse or dense) that each sample submodule should use during modeling. Then, these sample submodules and point cloud labels are used to train the classification neural network model. During the training process, the classification neural network model will learn how to predict the corresponding point cloud data type based on the feature information of the submodule.

[0063] The classification neural network model obtained through training can accurately identify the characteristics of the sub-module and predict the type of point cloud data it should use based on these characteristics. This provides a reliable basis for the subsequent modeling process and ensures the accuracy and efficiency of modeling.

[0064] S302: Extract features from all sub-modules to obtain feature information corresponding to the sub-modules one by one.

[0065] In this embodiment, the feature extraction method SIFT is used to process each sub-module to extract feature information that can represent the characteristics of the sub-module. The feature information can be data such as shape, size, surface texture, etc. that can describe the properties of the sub-module. The feature information of the sub-module is the basis for subsequent classification and modeling. Only by accurately extracting the features of the sub-module can we ensure that the classification neural network model can correctly match and classify it.

[0066] S303: The feature information of all sub-modules is used as input into the trained classification neural network model, and the classification neural network model performs matching according to the input feature information to obtain a matching result, wherein the matching result is represented by a matching probability, a matching score or a matching label.

[0067] In this embodiment, the extracted feature information of all sub-modules is used as input data and input into the trained classification neural network model. The classification neural network model will match and classify the feature information according to the knowledge and rules it has learned, and finally output the matching results. The matching results can be expressed in the form of matching probability, matching score or matching label.

[0068] In this embodiment, reference Figure 5 As shown, step S4 specifically includes:

[0069] S401: Obtain matching results, select the best matching sample submodule from the matching results of each submodule, and read the point cloud label of the sample submodule.

[0070] In this embodiment, in step S3, the classification neural network model has performed feature matching on each submodule and output matching results, which include matching results between the submodule and the internal samples of the model.

[0071] For each submodule, find the sample submodule with the highest matching degree from the matching results. This sample submodule usually represents the most similar or representative features to the submodule.

[0072] Each sample submodule has a point cloud label associated with it in the classification neural network model. This label identifies whether the sample submodule is built using sparse point cloud data or dense point cloud data. By reading the point cloud label associated with the best matching sample submodule, you can obtain the point cloud data category that the submodule of the object to be modeled should use.

[0073] S402: Based on the point cloud label of the sample submodule that best matches the submodule, select sparse point cloud data or dense point cloud data with the same point cloud label.

[0074] In this embodiment, according to the point cloud label of the best matching sample submodule read in step S401, it is determined whether it is sparse point cloud data or dense point cloud data. According to the determined point cloud label, the same type of point cloud data is selected from the point cloud data corresponding to the submodule, that is, if the label is sparse point cloud data, the sparse point cloud data is selected; if the label is dense point cloud data, the dense point cloud data is selected. Selecting the same point cloud data as the best matching sample submodule can ensure the consistency of modeling, that is, the modeling method of the submodule matches its most representative sample submodule.

[0075] S403: constructing a three-dimensional model based on the sparse point cloud data or dense point cloud data selected by each submodule.

[0076] In this embodiment, according to the point cloud data selected in step S402, the point cloud data required for modeling is prepared for each sub-module, and a three-dimensional model is constructed for each sub-module based on the prepared point cloud data using suitable point cloud data processing and modeling algorithms or tools. The three-dimensional models of all sub-modules are combined according to the original spatial relationship to form a complete three-dimensional model. By selecting the most suitable point cloud data for each sub-module for modeling, it can be ensured that the final constructed three-dimensional model can meet the requirements for shape, detail and accuracy. At the same time, modeling based on the selected point cloud data can avoid frequent switching or processing of different types of data during the modeling process, thereby improving the efficiency of modeling.

[0077] The working principle of a three-dimensional model construction method based on deep learning provided by the present invention is as follows: the present invention obtains point cloud data of an object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data; then, the object to be modeled is voxelized and segmented to obtain at least one sub-module and the sparse point cloud data and dense point cloud data belonging to the sub-module; then, all sub-modules are input into a pre-constructed classification neural network model for feature matching, and the matching results are output; finally, based on the matching results, the sparse point cloud data or dense point cloud data of the sub-module is selected for three-dimensional model construction. The present invention intelligently selects the appropriate point cloud data type when the sub-module constructs the three-dimensional model, so as to make full use of the advantages of different point cloud data in the modeling process, thereby obtaining a more accurate and complete three-dimensional model and improving the efficiency of modeling.

[0078] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0080] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A three-dimensional model construction method based on deep learning, characterized in that: The construction method includes the following steps: Acquire point cloud data of the object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data; Performing voxel segmentation on the object to be modeled to obtain at least one submodule and sparse point cloud data and dense point cloud data belonging to the submodule; Input all submodules into the pre-built classification neural network model for feature matching and output the matching results; Based on the matching result, the sparse point cloud data or dense point cloud data of the submodule is selected to construct the three-dimensional model; the matching result includes the matching result between the submodule and the internal sample of the model; each sample submodule has a point cloud label associated with it in the classification neural network model, and this label identifies whether the sample submodule is constructed using sparse point cloud data or dense point cloud data. By reading the point cloud label associated with the best matching sample submodule, the point cloud data category that the submodule of the object to be modeled should use can be obtained; The step of inputting all submodules into a pre-built classification neural network model for feature matching and outputting a matching result comprises: constructing a classification neural network model, wherein the classification neural network is trained by using sample submodules and their corresponding point cloud labels; extracting features from all submodules to obtain feature information corresponding to the submodules one by one; inputting the feature information of all submodules into the trained classification neural network model as input, and matching the input feature information by the classification neural network model to obtain a matching result, wherein the matching result is represented by a matching probability, a matching score or a matching label; The method of selecting the sparse point cloud data or dense point cloud data of the submodule for three-dimensional model construction based on the matching result includes: obtaining the matching result, selecting the best matching sample submodule from the matching result of each submodule, and reading the point cloud label of the sample submodule; based on the point cloud label of the sample submodule that best matches the submodule, selecting the sparse point cloud data or dense point cloud data with the same point cloud label; and constructing the three-dimensional model based on the sparse point cloud data or dense point cloud data selected by each submodule.

2. The method for constructing a three-dimensional model based on deep learning according to claim 1, characterized in that: The step of acquiring point cloud data of the object to be modeled, wherein the point cloud data of the object to be modeled includes sparse point cloud data and dense point cloud data, comprises: Collecting original point cloud data of the object to be modeled, wherein the original point cloud data includes original sparse point cloud data and original dense point cloud data; Preprocessing the original point cloud data of the object to be modeled, wherein the preprocessing includes cleaning, registering, filtering and correcting the point cloud data; Output preprocessed point cloud data.

3. The method for constructing a three-dimensional model based on deep learning according to claim 2, characterized in that: The voxel segmentation of the object to be modeled to obtain at least one submodule and sparse point cloud data and dense point cloud data belonging to the submodule includes: Discretizing the object to be modeled into voxels of uniform size, wherein each voxel represents a cubic unit of the object to be modeled; Classify all voxels based on the relationship between them; The boundaries of the classified voxels are determined and the boundaries are segmented to obtain submodules belonging to the object to be modeled and sparse point cloud data and dense point cloud data belonging to the submodules.

Images