Indoor and outdoor integrated three-dimensional model reconstruction method, terminal, system and medium

Through Huber loss minimization learning and graph neural network processing indoor and outdoor point cloud data, an efficient construction of indoor and outdoor integrated three-dimensional models is achieved, solving the problem of low efficiency in indoor and outdoor model reconstruction in the existing technology.

CN118379455BActive Publication Date: 2024-11-08ZHONGJIAO ROAD & BRIDGE (HEBEI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410460100.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-17
Publication Date
2024-11-08
Estimated Expiration
2044-04-17

AI Technical Summary

Technical Problem

In the prior art, the reconstruction efficiency of indoor and outdoor integrated three-dimensional model is low, mainly due to the redundancy, unstructured and large quality differences in indoor point cloud data, resulting in low artificial reconstruction efficiency.

Method used

The indoor semantic segmentation model and graph neural network with Huber loss minimization learning are used to process indoor and outdoor point cloud data, build an indoor and outdoor real scene three-dimensional model, and build an integrated three-dimensional model through inter-frame matching.

Benefits of technology

The reconstruction efficiency of indoor and outdoor integrated three-dimensional models is improved, and the impact of indoor point cloud data heterogeneity on model construction is reduced through efficient and accurate semantic segmentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118379455B_ABST
    Figure CN118379455B_ABST
Patent Text Reader

Abstract

The present invention provides a method, terminal, system and medium for reconstructing an indoor and outdoor integrated three-dimensional model. Initial point cloud data is first obtained, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data; then, an indoor real scene three-dimensional model is constructed based on the indoor point cloud data and an indoor semantic segmentation model learned by Huber loss minimization; then, an outdoor real scene three-dimensional model is constructed based on the outdoor point cloud data and a graph neural network; finally, the indoor real scene three-dimensional model and the outdoor real scene three-dimensional model are matched between frames to construct an indoor and outdoor integrated three-dimensional model. By adding the indoor semantic segmentation model learned by Huber loss minimization, efficient and accurate semantic segmentation is performed on the indoor point cloud data, avoiding the influence of the inhomogeneity of the indoor point cloud data on the indoor model construction, thereby improving the reconstruction efficiency of the indoor and outdoor integrated three-dimensional model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional digital technology, and in particular relates to a method, terminal, system and medium for reconstructing an indoor and outdoor integrated three-dimensional model. Background Art

[0002] The scene 3D model is the data foundation of 3D geographic information services, building information models, virtual reality and other technologies. As the data foundation of indoor location services, the indoor 3D model is the key to building an indoor 3D GIS system for location services. Due to the different spatial structural characteristics of indoor and outdoor spaces, there are significant differences in indoor and outdoor positioning and attitude determination technologies, resulting in different ways of obtaining indoor and outdoor 3D data.

[0003] At present, the reconstruction of indoor and outdoor integrated 3D models is generally divided into two parts: one is the reconstruction of outdoor scene models, and the other is the reconstruction of indoor building structures and facilities. Indoor point cloud data has the characteristics of large data volume, redundancy, unstructured, large quality differences, and uneven sampling granularity distribution. Therefore, the reconstruction of indoor models currently mostly adopts manual reconstruction methods, which has low operation efficiency and long model production cycle. Therefore, the efficiency of indoor and outdoor integrated 3D model reconstruction in the existing technology is relatively low. Summary of the invention

[0004] In view of this, the present invention provides a method, terminal, system and medium for reconstructing an indoor and outdoor integrated three-dimensional model, aiming to solve the problem of low efficiency of indoor and outdoor integrated three-dimensional model reconstruction in the prior art.

[0005] A first aspect of an embodiment of the present invention provides a method for reconstructing an indoor and outdoor integrated three-dimensional model, comprising:

[0006] Acquire initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data;

[0007] Construct a 3D model of the indoor scene based on the indoor point cloud data and the indoor semantic segmentation model learned by minimizing the Huber loss.

[0008] Construct a 3D model of outdoor real scenes based on outdoor point cloud data and graph neural network;

[0009] The indoor real scene 3D model and the outdoor real scene 3D model are matched between frames to construct an indoor and outdoor integrated 3D model.

[0010] A second aspect of an embodiment of the present invention provides an indoor and outdoor integrated three-dimensional model reconstruction device, comprising:

[0011] A data acquisition module, used to acquire initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data;

[0012] Indoor reconstruction module, used to construct indoor real scene 3D model based on indoor point cloud data and indoor semantic segmentation model learned by Huber loss minimization;

[0013] The outdoor reconstruction module is used to construct a 3D model of outdoor real scenes based on outdoor point cloud data and graph neural network;

[0014] The model matching module is used to perform frame-to-frame matching between the indoor real-scene 3D model and the outdoor real-scene 3D model to construct an indoor-outdoor integrated 3D model.

[0015] A third aspect of an embodiment of the present invention provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the indoor and outdoor integrated three-dimensional model reconstruction method of the first aspect are implemented.

[0016] A fourth aspect of an embodiment of the present invention provides an indoor and outdoor integrated model reconstruction system, comprising a point cloud acquisition device and a terminal as described in the third aspect above.

[0017] A fifth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the indoor and outdoor integrated three-dimensional model reconstruction method of the first aspect are implemented.

[0018] The indoor-outdoor integrated 3D model reconstruction method, terminal, system and medium provided by the embodiment of the present invention first obtains initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data; then, an indoor real-scene 3D model is constructed based on the indoor point cloud data and the indoor semantic segmentation model learned by Huber loss minimization; then, an outdoor real-scene 3D model is constructed based on the outdoor point cloud data and the graph neural network; finally, the indoor real-scene 3D model and the outdoor real-scene 3D model are matched between frames to construct an indoor-outdoor integrated 3D model. By adding the indoor semantic segmentation model learned by Huber loss minimization, efficient and accurate semantic segmentation is performed on the indoor point cloud data, avoiding the influence of the inhomogeneity of the indoor point cloud data on the indoor model construction, thereby improving the reconstruction efficiency of the indoor-outdoor integrated 3D model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1is a structural schematic diagram of an indoor and outdoor integrated model reconstruction system provided by an embodiment of the present invention;

[0021] Figure 2 It is a flow chart of the implementation of the indoor and outdoor integrated three-dimensional model reconstruction method provided by an embodiment of the present invention;

[0022] Figure 3 It is a structural schematic diagram of an indoor and outdoor integrated three-dimensional model reconstruction device provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0025] Figure 1 1 is a schematic diagram of the structure of an indoor-outdoor integrated model reconstruction system provided by an embodiment of the present invention. The indoor-outdoor integrated model reconstruction system of the present invention comprises a point cloud acquisition device 11 and a terminal 12.

[0026] The point cloud acquisition device 11 can be a single-line laser scanner, a two-dimensional laser scanner, a navigation satellite system, etc., which is not limited here. The terminal 12 can be a mobile phone, a computer, etc., which is not limited here. For example, three two-dimensional laser scanners can be set on a collection robot, and the collection robot enters the interior of the building and moves continuously. The three two-dimensional laser scanners scan the building information in three different directions respectively, and the postures of the three laser scanners are synchronized. Laser scanners are usually used for the collection of indoor point cloud data, and navigation satellite systems are usually used for the collection of outdoor point cloud data.

[0027] Figure 2 FIG. 1 is a flowchart of an implementation method of an indoor and outdoor integrated three-dimensional model reconstruction method provided by an embodiment of the present invention. Figure 2 As shown, the indoor and outdoor integrated three-dimensional model reconstruction method includes:

[0028] S210, acquiring initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data;

[0029] S220, builds a 3D model of the indoor scene based on the indoor point cloud data and the indoor semantic segmentation model learned by minimizing the Huber loss;

[0030] In an embodiment of the present invention, the Huber loss function is a method for optimizing a loss function, which is a smooth, differentiable function widely used in machine learning, especially in the processing of regression and classification problems. The Huber loss function is more robust than other loss functions such as mean square error when processing outliers. It divides the loss into two parts: when the gap between the predicted value and the true value is small, the mean square error is used; when the gap is large, a linear function is used.

[0031] In some embodiments, the indoor semantic segmentation model includes a first indoor semantic segmentation model and a second indoor semantic segmentation model; S220 includes: preprocessing indoor point cloud data; semantically segmenting the preprocessed indoor point cloud data according to the first indoor semantic segmentation model learned by Huber loss minimization to obtain initial indoor semantic information; extracting the first feature point in the preprocessed indoor point cloud data according to the farthest distance sampling algorithm; wherein each feature point corresponds to the indoor point cloud data of an area; classifying the first feature points according to the initial indoor semantic information, and segmenting the indoor point cloud data corresponding to each type of first feature point using the second indoor semantic segmentation model learned by Huber loss minimization corresponding to the first feature point of the type to obtain first semantic information; and constructing a three-dimensional model of indoor real scene according to the first semantic information.

[0032] In an embodiment of the present invention, it is necessary to pre-process the indoor point cloud data by filtering out noise points and removing outliers, so as to reduce the influence of local characteristics and noise on the sampling results. Multiple groups of indoor point cloud data can be obtained, and each group of indoor point cloud data can be manually annotated to form a training set and a test set, and the first indoor semantic segmentation model is obtained by training. In the above-mentioned annotated point cloud data, the point cloud data of the same structural type are divided into a group to obtain multiple training sets and test sets, and each training set and test set corresponds to training a second indoor semantic segmentation model.

[0033] In an embodiment of the present invention, the extraction of different indoor structures has different requirements for the density of point cloud data. During the training process of the first indoor semantic model, the influence of point cloud density on segmentation is not taken into account, and the indoor point cloud data is unstructured and inhomogeneous. The model trained with single density data has poor generalization ability for density changes. Therefore, the first indoor semantic model can only roughly segment the point cloud data. After the rough segmentation, the first feature points in the point cloud data are further extracted and classified. Each type of first feature point corresponds to a type of indoor structure. For each type of indoor structure, the present invention selects a training set and a test set with appropriate point cloud density to train a second indoor semantic segmentation model for the type of indoor structure. After classifying the first feature points, the corresponding point cloud data can be input into the corresponding second semantic segmentation model, and further segmented on the basis of the rough segmentation to obtain the first semantic information, that is, the semantic annotation of the indoor structure, so as to establish a real-life three-dimensional model of the indoor scene.

[0034] In some embodiments, the first feature point in the preprocessed indoor point cloud data is extracted according to the farthest distance sampling algorithm, including: determining the starting point and the number of starting points of the farthest distance sampling algorithm according to initial semantic information; extracting the seed point in the preprocessed indoor point cloud data according to the farthest distance sampling algorithm, and determining the sampling range for searching for the seed point each time according to the initial semantic information and the sparsity of the point cloud data in the neighborhood of the seed point, until the number of seed points reaches a preset number, and all seed points are determined as the first feature points.

[0035] In the embodiment of the present invention, the farthest distance sampling is a sampling method for three-dimensional point cloud data. The basic idea of ​​this method is to select a point from the point cloud as a seed point, then find the point farthest from the seed point, add it to the sampling set, and then repeat this process until the sampling quantity requirement is met. In this process, each time the point selected is the farthest point in the current selected point set, hence the name farthest distance sampling.

[0036] In some embodiments, the starting point and the number of starting points of the farthest distance sampling algorithm are determined according to the initial semantic information, including: determining the type of each indoor target structure according to the initial semantic information; and querying the starting point and the number of starting points corresponding to each target structure.

[0037] In the embodiment of the present invention, the farthest distance sampling is very efficient and suitable for sampling large-scale point cloud data. It can quickly extract effective information from the point cloud data, but it is easy to fall into the local optimum and it is difficult to find the global optimal solution. Since the present invention has previously performed a rough segmentation to obtain initial semantic information, different starting points and the number of starting points can be set for different indoor structures. The more complex the structure, the more starting point data there is. The position of the starting point is selected according to the characteristics of the indoor structure. Thus, the diversity of sampling points can be increased and the appearance of local characteristics can be reduced.

[0038] In some embodiments, the sampling range for each seed point search is determined based on the initial semantic information and the sparsity of the point cloud data in the neighborhood of the seed point, including: determining the type of each target structure in the room based on the initial semantic information; querying the optimal sparsity corresponding to each target structure; and determining the sampling range based on the sparsity of the point cloud data in the neighborhood of the seed point of each target structure and the optimal sparsity.

[0039] In the embodiment of the present invention, in addition to considering the influence of the point cloud density at the starting point, it is also necessary to dynamically adjust the sampling radius according to the distribution of the currently selected sampling points during the sampling process. For example, if the points in a certain direction of the selected point set are relatively sparse, the sampling radius in that direction can be appropriately increased to ensure that the longest distance sampling can collect the global optimal solution.

[0040]

[0041] Among them, R is the sampling radius, that is, the sampling range, k is the sparsity of the point cloud data in the neighborhood of the seed point of each target structure, k0 is the optimal sparsity, and R0 is the initial sampling radius.

[0042] S230, constructing an outdoor real scene three-dimensional model based on outdoor point cloud data and graph neural network;

[0043] In the embodiment of the present invention, the goal of point cloud coarse segmentation is to generate high-quality three-dimensional point clusters for constructing a graph representation of the point cloud, thereby performing semantic segmentation of the point cloud. Point clusters provide a more natural and compact representation for three-dimensional point clouds, and enable point cloud processing to be performed on local areas rather than on scattered points, greatly reducing the complexity of point cloud processing.

[0044] In some embodiments, S230 includes: preprocessing outdoor point cloud data; locally clustering the preprocessed outdoor point cloud data according to an energy function to obtain multiple point clusters; inputting the point clusters and outdoor point cloud data into a graph neural network to obtain an outdoor real-scene three-dimensional model.

[0045] In the embodiment of the present invention, the clustering method based on the energy function can be based on a naive point cloud clustering algorithm, a local k-means clustering method, etc., which is not limited here. The present invention uses the spatial relationship of the point cloud within the point cluster to encode the point cluster to obtain the initial state of the node, and uses the spatial relationship between the point clusters to synthesize the state information of adjacent nodes through the edge convolution method to generate the input of the graph.

[0046] S240, performing frame matching on the indoor real scene 3D model and the outdoor real scene 3D model to construct an indoor and outdoor integrated 3D model.

[0047] In some embodiments, S240 includes: determining the indoor matching points corresponding to the preset outdoor contour points based on the first semantic information; determining the coordinate transformation relationship based on the outdoor contour points and the indoor matching points; and performing inter-frame matching of the indoor real scene three-dimensional model and the outdoor real scene three-dimensional model based on the coordinate transformation relationship to construct an integrated indoor and outdoor three-dimensional model.

[0048] In an embodiment of the present invention, the preset outdoor contour points can be contour points on specific indoor and outdoor connecting structures such as doors and windows. After the semantic segmentation is completed, the corresponding parts of the doors and windows in the indoor point cloud data can be identified, that is, the corresponding indoor matching points are obtained. According to the correspondence between the outdoor contour points and the indoor matching points, the coordinate transformation relationship of the indoor and outdoor coordinate systems is solved, thereby realizing the matching of indoor and outdoor models and constructing an integrated indoor and outdoor three-dimensional model.

[0049] In summary, the beneficial effects of the present invention are specifically to perform efficient and accurate semantic segmentation of indoor point cloud data by adding the indoor semantic segmentation model learned by Huber loss minimization, avoiding the influence of the inhomogeneity of indoor point cloud data on the construction of indoor models, thereby improving the reconstruction efficiency of the integrated indoor and outdoor three-dimensional model.

[0050] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0051] Figure 3 Schematic diagram of the structure of the indoor and outdoor integrated three-dimensional model reconstruction device provided by the embodiment of the present invention. Figure 3 As shown, in some embodiments, the indoor and outdoor integrated three-dimensional model reconstruction device 3 includes:

[0052] A data acquisition module 310 is used to acquire initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data;

[0053] An indoor reconstruction module 320 is used to construct an indoor real scene 3D model based on indoor point cloud data and an indoor semantic segmentation model learned by minimizing Huber loss;

[0054] An outdoor reconstruction module 330 is used to construct an outdoor real scene 3D model based on outdoor point cloud data and graph neural network;

[0055] The model matching module 340 is used to perform frame matching between the indoor real scene 3D model and the outdoor real scene 3D model to construct an indoor and outdoor integrated 3D model.

[0056] Optionally, the indoor semantic segmentation model includes a first indoor semantic segmentation model and a second indoor semantic segmentation model; an indoor reconstruction module 320 is used to preprocess indoor point cloud data; semantic segmentation is performed on the preprocessed indoor point cloud data according to the first indoor semantic segmentation model learned by Huber loss minimization to obtain initial indoor semantic information; the first feature points in the preprocessed indoor point cloud data are extracted according to the farthest distance sampling algorithm; wherein each feature point corresponds to indoor point cloud data of an area; the first feature points are classified according to the initial indoor semantic information, and the indoor point cloud data corresponding to each type of first feature point is segmented using the second indoor semantic segmentation model learned by Huber loss minimization corresponding to the first feature point of the type to obtain first semantic information; and a real-scene three-dimensional model of the indoor scene is constructed according to the first semantic information.

[0057] Optionally, the indoor reconstruction module 320 is used to determine the starting point and the number of starting points of the farthest distance sampling algorithm based on the initial semantic information; extract the seed points in the preprocessed indoor point cloud data according to the farthest distance sampling algorithm, and determine the sampling range for searching for seed points each time according to the initial semantic information and the sparsity of the point cloud data in the neighborhood of the seed points, until the number of seed points reaches a preset number, and all seed points are determined as the first feature points.

[0058] Optionally, the indoor reconstruction module 320 is used to determine the type of each indoor target structure according to the initial semantic information; and query the starting point and the number of starting points corresponding to each target structure.

[0059] Optionally, the indoor reconstruction module 320 is used to determine the type of each indoor target structure based on the initial semantic information; query the optimal sparsity corresponding to each target structure; and determine the sampling range based on the sparsity and optimal sparsity of the point cloud data in the neighborhood of the seed point of each target structure.

[0060] Optionally, the outdoor reconstruction module 330 is used to: pre-process the outdoor point cloud data; locally cluster the pre-processed outdoor point cloud data according to the energy function to obtain multiple point clusters; input the point clusters and outdoor point cloud data into the graph neural network to obtain an outdoor real-scene three-dimensional model.

[0061] Optionally, the model matching module 340 is used to determine the indoor matching points corresponding to the preset outdoor contour points according to the first semantic information; determine the coordinate transformation relationship according to the outdoor contour points and the indoor matching points; and perform frame-to-frame matching on the indoor real scene three-dimensional model and the outdoor real scene three-dimensional model according to the coordinate transformation relationship to construct an integrated indoor and outdoor three-dimensional model.

[0062] The indoor and outdoor integrated three-dimensional model reconstruction device provided in this embodiment can be used to execute the above method embodiments. Its implementation principles and technical effects are similar, and this embodiment will not be repeated here.

[0063] Figure 4 Schematic diagram of the structure of the terminal provided by the embodiment of the present invention. Figure 4 As shown, a terminal 4 provided by an embodiment of the present invention includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer program 42, the steps in the above-mentioned embodiments of the indoor and outdoor integrated three-dimensional model reconstruction method are implemented, for example Figure 2 Alternatively, when the processor 40 executes the computer program 42, the functions of each module / unit in the above-mentioned system embodiments are realized, for example Figure 3 The functions of each module are shown.

[0064] Exemplarily, the computer program 42 may be divided into one or more modules / units, one or more modules / units are stored in the memory 41, and are executed by the processor 40 to implement the present invention. One or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 42 in the terminal 4.

[0065] The terminal 4 may be a terminal or a server. The terminal may be a mobile phone, MCU, ECU, industrial computer, etc., which are not limited here. The server may be a physical server, a cloud server, etc., which are not limited here. The terminal 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will understand that Figure 4 It is only an example of terminal 4 and does not constitute a limitation on terminal 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.

[0066] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0067] The memory 41 may be an internal storage unit of the terminal 4, such as a hard disk or memory of the terminal 4. The memory 41 may also be an external storage device of the terminal 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal 4. The memory 41 is used to store computer programs and other programs and data required by the terminal. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0068] An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned indoor and outdoor integrated three-dimensional model reconstruction method embodiment are implemented.

[0069] The computer readable storage medium stores a computer program 42, which includes program instructions. When the program instructions are executed by the processor 40, all or part of the processes in the above-mentioned embodiment method are implemented. The computer program 42 can also be used to instruct related hardware to complete. The computer program 42 can be stored in a computer readable storage medium. When the computer program 42 is executed by the processor 40, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program 42 includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium.

[0070] The computer-readable storage medium may be an internal storage unit of the terminal of any of the foregoing embodiments, such as a hard disk or memory of the terminal. The computer-readable storage medium may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit of the terminal and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0071] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0072] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0073] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0075] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0076] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0077] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0078] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media can include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A method for reconstructing an indoor and outdoor integrated three-dimensional model, characterized in that: include: Acquire initial point cloud data, wherein the initial point cloud data includes indoor point cloud data and outdoor point cloud data; Constructing an indoor real scene three-dimensional model based on the indoor point cloud data and the indoor semantic segmentation model learned by Huber loss minimization; Constructing an outdoor real scene three-dimensional model according to the outdoor point cloud data and the graph neural network; Performing frame matching on the indoor real scene 3D model and the outdoor real scene 3D model to construct an indoor and outdoor integrated 3D model; The indoor semantic segmentation model includes a first indoor semantic segmentation model and a second indoor semantic segmentation model; and constructing an indoor real scene three-dimensional model according to the indoor point cloud data and the indoor semantic segmentation model learned by Huber loss minimization, including: Preprocessing the indoor point cloud data; The preprocessed indoor point cloud data is semantically segmented according to the first indoor semantic segmentation model learned by Huber loss minimization to obtain the initial indoor semantic information; Extracting the first feature point from the preprocessed indoor point cloud data according to the farthest distance sampling algorithm; wherein each feature point corresponds to the indoor point cloud data of a region; The first feature points are classified according to the initial indoor semantic information, and the indoor point cloud data corresponding to each type of first feature point is segmented by using a second indoor semantic segmentation model learned by minimizing the Huber loss corresponding to the type of first feature point to obtain first semantic information; Constructing a three-dimensional model of indoor real scene according to the first semantic information; According to the outdoor point cloud data and the graph neural network, a three-dimensional model of an outdoor real scene is constructed, including: Preprocessing the outdoor point cloud data; The pre-processed outdoor point cloud data is locally clustered according to the energy function to obtain multiple point clusters; The point cluster and the outdoor point cloud data are input into the graph neural network to obtain an outdoor real-scene three-dimensional model.

2. The indoor and outdoor integrated three-dimensional model reconstruction method according to claim 1, characterized in that: The first feature point in the preprocessed indoor point cloud data is extracted according to the farthest distance sampling algorithm, including: Determining the starting point and the number of starting points of the farthest distance sampling algorithm according to the initial semantic information; The seed points in the preprocessed indoor point cloud data are extracted according to the farthest distance sampling algorithm, and the sampling range for searching the seed points each time is determined according to the initial semantic information and the sparsity of the point cloud data in the neighborhood of the seed points, until the number of seed points reaches a preset number, and all the seed points are determined as the first feature points.

3. The indoor-outdoor integrated three-dimensional model reconstruction method according to claim 2, characterized in that: Determining the starting point and the number of starting points of the farthest distance sampling algorithm according to the initial semantic information includes: Determining the types of each indoor target structure according to the initial semantic information; Query the starting point and the number of starting points corresponding to each target structure.

4. The indoor-outdoor integrated three-dimensional model reconstruction method according to claim 2, characterized in that: According to the initial semantic information and the sparsity of the point cloud data in the neighborhood of the seed point, the sampling range for searching the seed point each time is determined, including: Determining the types of each indoor target structure according to the initial semantic information; Query the optimal sparsity corresponding to each target structure; The sampling range is determined according to the sparsity and optimal sparsity of the point cloud data in the neighborhood of the seed point of each target structure.

5. The indoor-outdoor integrated three-dimensional model reconstruction method according to claim 1, characterized in that: Performing frame matching on the indoor real scene 3D model and the outdoor real scene 3D model to construct an indoor and outdoor integrated 3D model, including: Determining, according to the first semantic information, an indoor matching point corresponding to a preset outdoor contour point; Determine a coordinate transformation relationship according to the outdoor contour points and the indoor matching points; According to the coordinate conversion relationship, the indoor real scene three-dimensional model and the outdoor real scene three-dimensional model are matched between frames to construct an indoor and outdoor integrated three-dimensional model.

6. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the indoor and outdoor integrated three-dimensional model reconstruction method as described in any one of claims 1 to 5 are implemented.

7. An indoor and outdoor integrated model reconstruction system, comprising a point cloud acquisition device and the terminal as described in claim 6 above.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the indoor and outdoor integrated three-dimensional model reconstruction method as described in any one of claims 1 to 5 are implemented.