Panorama, point cloud and BIM-based model construction method and device, equipment and medium

Abstract

The application relates to the technical field of three-dimensional modeling, and discloses a model construction method based on panoramas, point clouds and BIM, which comprises the following steps: acquiring multi-view point cloud data of a building, and obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; acquiring multiple panorama images of the building, and splicing the multiple panorama images according to feature points of the panorama images to construct three-dimensional panorama image data; performing standardization processing on the three-dimensional point cloud data and the panorama image data to obtain standard point cloud data and standard panorama image data; acquiring building information model data from a preset BIM software, and fusing the standard point cloud data, the standard panorama image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model. The application also provides a model construction device based on panoramas, point clouds and BIM, equipment and a storage medium. The application can improve the accuracy of the constructed three-dimensional model.

Description

Technical Field

[0001] This invention relates to the field of 3D modeling technology, and in particular to a model construction method, apparatus, equipment and medium based on panorama, point cloud and BIM. Background Technology

[0002] In recent years, with the rapid development of the construction industry and the advancement of digital transformation, a large amount of data has been accumulated during the design, construction, and operation and maintenance of construction projects, including Building Information Modeling (BIM), point cloud data, and panoramic images. These data each provide different perspectives and information layers of construction projects, but the lack of effective integration and collaboration among them has led to serious information silos, affecting the efficiency and quality of construction projects.

[0003] However, existing technologies often only process single types of data, failing to achieve deep integration and comprehensive utilization of multi-source data. For example, while BIM data provides precise information on building structures and components, it lacks a direct view of the actual construction site; point cloud data, while capturing the precise three-dimensional shape of a building, lacks color and texture information; and panoramic images, while providing a panoramic view of the site, cannot be combined with the building's precise dimensions and structural information. This data isolation limits the decision support and problem-solving capabilities of building projects during the design, construction, and operation phases.

[0004] Furthermore, construction projects involve numerous stakeholders, including designers, construction workers, maintenance teams, and owners, who require information from different angles and levels to support their respective work. Existing technologies cannot meet these diverse and personalized information needs, leading to high communication costs, low collaboration efficiency, and even potential project delays or quality issues due to misunderstandings. Therefore, there is an urgent need for a model building methodology based on panoramic views, point clouds, and BIM, focusing on the digital reconstruction and visualization of construction projects. This methodology should deeply integrate and comprehensively utilize multi-source building data to address the problems of information silos and low collaboration efficiency in construction projects. Summary of the Invention

[0005] This invention provides a model building method, apparatus, equipment, and medium based on panorama, point cloud, and BIM, with the main purpose of improving the accuracy of the constructed three-dimensional model.

[0006] To achieve the above objectives, this invention provides a model construction method based on panorama, point cloud, and BIM, comprising:

[0007] Multi-view point cloud data of a building is acquired by intelligent scanning equipment, and three-dimensional point cloud data is obtained by aligning the multi-view point cloud data to the same coordinate system.

[0008] Multiple panoramic images of the building are acquired using a panoramic camera device, and the multiple panoramic images are stitched together based on the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0009] The three-dimensional point cloud data and the panoramic image data are standardized to obtain standard point cloud data and standard panoramic image data.

[0010] Building information model data is obtained from the preset BIM software, and the standard point cloud data, standard panoramic image data and building information model data are fused together based on the preset spatial coordinate system to construct a unified three-dimensional building model.

[0011] Optionally, obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system includes:

[0012] The first-view point cloud data is obtained from the multi-view point cloud data as the first source point cloud, and the second-view point cloud data corresponding to the source point cloud is used as the target point cloud.

[0013] By setting an initial transformation matrix for each point in the first source point cloud, querying the nearest neighbor in the target point cloud, and constructing a corresponding point set;

[0014] The rotation matrix and translation vector of the corresponding point set are calculated by minimizing the objective function, and the point cloud position of the source point cloud is updated by the rotation matrix and translation vector to obtain the transformation matrix;

[0015] If the change in the transformation matrix is ​​less than a preset threshold or the maximum number of iterations is reached, the iteration is stopped and the final transformation matrix is ​​obtained.

[0016] The source point cloud data is registered to the target point cloud according to the final transformation matrix to obtain the first three-dimensional point cloud data.

[0017] Select a third-view point cloud from the multi-view point cloud data as the second source point cloud, and register the second source point cloud to the first three-dimensional point cloud data to obtain the second three-dimensional point cloud data. Continue until the point clouds from all views in the multi-view point cloud data are registered to obtain the three-dimensional point cloud data.

[0018] Optionally, the step of stitching together the multiple panoramic images based on the feature points of the panoramic images to construct three-dimensional panoramic image data includes:

[0019] The multiple panoramic images are compared, and the same feature points are obtained in the overlapping area of ​​the multiple panoramic images by the feature point detection algorithm to obtain matching feature points;

[0020] The multiple panoramic images are stitched together using the matching feature points to construct a stitched panoramic image;

[0021] The boundaries of the stitched panoramic image are smoothed to obtain three-dimensional panoramic image data.

[0022] Optionally, obtaining matching feature points by acquiring identical feature points in the overlapping areas of the multiple panoramic images using a feature point detection algorithm includes:

[0023] The same feature points are found in the overlapping region using a feature detection algorithm, and descriptors are generated for the feature points.

[0024] The descriptors of feature points in different images are compared using a preset matching algorithm to obtain matching feature points.

[0025] Optionally, the step of stitching together the multiple panoramic images using the matching feature points to construct a stitched panoramic image includes:

[0026] The transformation matrix between panoramic images is calculated using the matching feature points, and robust registration is performed using the RANSAC algorithm.

[0027] The image is transformed to a unified coordinate system using a transformation matrix. The transformed images are then stitched together in a specific order to obtain a stitched panoramic image.

[0028] Optionally, the standardization process for the 3D point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data includes:

[0029] Exposure processing is performed on the stitched panoramic image to reduce its brightness;

[0030] Perform color balancing on the stitched panoramic image to ensure accurate color reproduction, and then apply the color-balanced panoramic image accordingly.

[0031] Noise is removed from the color-balanced panoramic image, and then sharpened to obtain standard panoramic image data.

[0032] Optionally, the step of comparing the descriptors of feature points in different images using a preset matching algorithm to obtain matching feature points includes:

[0033] Feature points in the different images are filtered to remove bad feature points, resulting in filtered feature points. The homography matrix between the filtered feature point pairs is then calculated using the RANSAC algorithm.

[0034] Based on the homography matrix, the positions of vertices in the second image in the first image are calculated, as well as the vertices covered by the vertices in the second image, and these vertices are used as initial feature points.

[0035] The first and second images are connected and drawn based on the initial feature points and OpenCV's drawMatches function to obtain the initial matching result;

[0036] Verify whether the initial feature points in the initial matching result correctly identify the overlapping areas in the image. If they are correctly identified, the matching feature points are obtained.

[0037] To address the aforementioned problems, the present invention also provides a model building device based on panorama, point cloud, and BIM, the device comprising:

[0038] The point cloud data acquisition module is used to acquire multi-view point cloud data of buildings through intelligent scanning equipment, and to obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system.

[0039] A panoramic image construction module is used to acquire multiple panoramic images of the building through a panoramic camera device, and to stitch the multiple panoramic images together based on the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0040] The data standardization module is used to standardize the 3D point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data.

[0041] The data fusion module is used to acquire building information model data from preset BIM software, and to fuse the standard point cloud data, standard panoramic image data and building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0042] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0043] At least one processor; and,

[0044] A memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the model building method based on panorama, point cloud and BIM as described above.

[0046] To address the aforementioned problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area. The data storage area stores created data, and the program storage area stores a computer program. When the computer program is executed by a processor, it implements the model construction method based on panorama, point cloud, and BIM as described above.

[0047] This invention acquires multi-view point cloud data of a building using an intelligent scanning device, and obtains 3D point cloud data by aligning the multi-view point cloud data to the same coordinate system. Multiple panoramic images of the building are acquired using a panoramic camera, and these panoramic images are stitched together based on their feature points to construct 3D panoramic image data. The 3D point cloud data and the panoramic image data are standardized to obtain standard point cloud data and standard panoramic image data. Building information model data is acquired from preset BIM software, and the standard point cloud data, standard panoramic image data, and building information model data are fused based on a preset spatial coordinate system to construct a unified 3D building model. Therefore, the model construction method, device, electronic device, and computer-readable storage medium proposed in this invention, based on panorama, point cloud, and BIM, constructs a 3D building model by combining 3D point cloud data, panoramic image data, and building model information model data from BIM software, fully exploring the characteristics between various data types and improving the accuracy of the constructed 3D model. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a model building method based on panorama, point cloud, and BIM, provided in an embodiment of the present invention.

[0049] Figure 2 A schematic diagram of a module for a model building device based on panorama, point cloud, and BIM provided in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the internal structure of an electronic device that implements a model building method based on panorama, point cloud, and BIM, according to an embodiment of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0053] This application provides a model building method based on panorama, point cloud, and BIM. The executing entity of this panorama, point cloud, and BIM-based model building method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In other words, the panorama, point cloud, and BIM-based model building method can be executed by software or hardware installed on remote devices or server-side devices, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0054] Reference Figure 1 The diagram shown is a flowchart illustrating a model construction method based on panorama, point cloud, and BIM according to an embodiment of the present invention. In this embodiment, the model construction method based on panorama, point cloud, and BIM includes the following steps S1-S4:

[0055] S1. Obtain multi-view point cloud data of the building through intelligent scanning equipment, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system.

[0056] Understandably, acquiring multi-view point cloud data through intelligent scanning devices can avoid the possibility that a single-view scan may not capture all the details of a building. Especially for large or complex structures, multi-view scanning can provide more comprehensive coverage, ensuring that all areas are recorded. Furthermore, point cloud data acquired from different views can be mutually verified and supplemented, improving the accuracy and reliability of the overall point cloud data.

[0057] In another embodiment of the present invention, each pixel in the depth image can be converted into a point in three-dimensional space to generate a point cloud.

[0058] In this embodiment of the invention, obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system includes:

[0059] The first-view point cloud data is obtained from the multi-view point cloud data as the first source point cloud, and the second-view point cloud data corresponding to the source point cloud is used as the target point cloud.

[0060] By setting an initial transformation matrix for each point in the first source point cloud, querying the nearest neighbor in the target point cloud, and constructing a corresponding point set;

[0061] The rotation matrix and translation vector of the corresponding point set are calculated by minimizing the objective function, and the point cloud position of the source point cloud is updated by the rotation matrix and translation vector to obtain the transformation matrix;

[0062] If the change in the transformation matrix is ​​less than a preset threshold or the maximum number of iterations is reached, the iteration is stopped and the final transformation matrix is ​​obtained.

[0063] The source point cloud data is registered to the target point cloud according to the final transformation matrix to obtain the first three-dimensional point cloud data.

[0064] Select a third-view point cloud from the multi-view point cloud data as the second source point cloud, and register the second source point cloud to the first three-dimensional point cloud data to obtain the second three-dimensional point cloud data. Continue until the point clouds from all views in the multi-view point cloud data are registered to obtain the three-dimensional point cloud data.

[0065] In this embodiment of the invention, the initial transformation matrix is ​​typically an identity matrix, representing the preliminary transformation relationship from the source point cloud to the target point cloud, indicating that no transformation has been performed. The minimized objective function is typically the sum of squared distances between corresponding point sets.

[0066] Furthermore, the iteration can be stopped when the change in the error function during the transformation of the transformation matrix is ​​less than a preset threshold, thus obtaining the final transformation matrix.

[0067] In this embodiment of the invention, the target point cloud is a point cloud used as a registration reference. Point cloud data from other perspectives need to be registered to that perspective. The first source point cloud and the second source point cloud represent point cloud data that need to be registered to the target point cloud. After registration, the point cloud data of that perspective will disappear.

[0068] S2. Acquire multiple panoramic images of the building using a panoramic camera device, and stitch the multiple panoramic images together based on the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0069] In this embodiment of the invention, the panoramic camera device can be a panoramic camera, and the panoramic image is a wide-view image that creates a seamless, continuous wide-view frame by stitching together multiple individual images. Panoramic images can be horizontal, vertical, or even spherical, providing a wider field of view than traditional single images. In this invention, multiple panoramic images can be stitched together to construct three-dimensional panoramic image data.

[0070] Furthermore, when acquiring multiple panoramic images of the building using a panoramic camera device, the device, being an ultra-wide-angle lens, captures an extremely wide field of view. However, due to its unique optical characteristics, the resulting images exhibit significant distortion; objects at the image edges are stretched and distorted. Therefore, after acquiring multiple panoramic images, image processing techniques are needed to correct this distortion, restoring objects in the images to their true shapes and proportions in the real world. This ensures that the buildings or other elements in the images more realistically reflect their actual appearance and proportions.

[0071] In this embodiment of the invention, the step of stitching together the multiple panoramic images based on the feature points of the panoramic images to construct three-dimensional panoramic image data includes:

[0072] The multiple panoramic images are compared, and the same feature points are obtained in the overlapping area of ​​the multiple panoramic images by the feature point detection algorithm to obtain matching feature points;

[0073] The multiple panoramic images are stitched together using the matching feature points to construct a stitched panoramic image;

[0074] The boundaries of the stitched panoramic image are smoothed to obtain three-dimensional panoramic image data.

[0075] Specifically, the matching feature points refer to corresponding points found in different panoramic images, where these points represent the same physical location or feature. These matching feature points can be detected in overlapping areas of multiple panoramic images using feature point detection algorithms (such as SIFT, SURF, or ORB), and then corresponding point pairs are found using a feature point matching algorithm.

[0076] Furthermore, the step of obtaining matching feature points by acquiring identical feature points in the overlapping areas of the multiple panoramic images using a feature point detection algorithm includes:

[0077] The same feature points are found in the overlapping region using a feature detection algorithm, and descriptors are generated for the feature points.

[0078] The descriptors of feature points in different images are compared using a preset matching algorithm to obtain matching feature points.

[0079] Furthermore, the step of comparing the descriptors of feature points in different images using a preset matching algorithm to obtain matching feature points includes:

[0080] Feature points in the different images are filtered to remove bad feature points, resulting in filtered feature points. The homography matrix between the filtered feature point pairs is then calculated using the RANSAC algorithm.

[0081] Based on the homography matrix, the positions of vertices in the second image in the first image are calculated, as well as the vertices covered by the vertices in the second image, and these vertices are used as initial feature points.

[0082] The first and second images are connected and drawn based on the initial feature points and OpenCV's drawMatches function to obtain the initial matching result;

[0083] Verify whether the initial feature points in the initial matching result correctly identify the overlapping areas in the image. If they are correctly identified, the matching feature points are obtained.

[0084] In this embodiment of the invention, RANSAC (RANdom Sampling Consensus) represents a random sampling consensus algorithm, an iterative method for estimating mathematical model parameters from a set of data containing outliers. It is suitable for datasets with a large proportion of erroneous data or outliers, and can still robustly estimate the correct model parameters.

[0085] Furthermore, the homography matrix, also known as the perspective transformation matrix, is an important mathematical tool in computer vision and image processing used to describe the perspective transformation relationship between two images. This transformation is commonly used to map an image from one plane to another, such as mapping an image from one camera plane to another in stereo vision, or aligning images from different perspectives in panoramic image stitching.

[0086] In this embodiment of the invention, the `drawMatches` function is a function in the OpenCV library used to draw matching feature points between two images. This function is typically used after feature point detection and matching to facilitate visualization and verification of the feature point matching results.

[0087] Furthermore, after using feature detection algorithms (such as SIFT, SURF, ORB, etc.) to detect feature points in two images and using feature matching algorithms to find matching pairs, the drawMatches function can help to visually see which feature points have been matched together.

[0088] In this embodiment of the invention, the step of stitching together the multiple panoramic images by matching feature points to obtain a stitched panoramic image includes:

[0089] The transformation matrix between panoramic images is calculated using the matching feature points, and robust registration is performed using the RANSAC algorithm.

[0090] The image is transformed to a unified coordinate system using a transformation matrix. The transformed images are then stitched together in a specific order to obtain a stitched panoramic image.

[0091] In this embodiment of the invention, the transformation matrix can be 2D or 3D and is used to describe linear transformations between points, lines, and surfaces in space, including rotation, translation, scaling, and shearing. Compared with homography matrices, transformation matrices are applicable to a wider range of spatial transformations, such as 3D modeling and object localization in machine vision.

[0092] S3. Standardize the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data.

[0093] In this embodiment of the invention, the standardization of 3D point cloud data and panoramic image data is to ensure the consistency of data in terms of measurement, format and quality, improve the comparability, compatibility and accuracy of data, so that the data can be effectively stored, processed, analyzed and shared to meet the needs of different application scenarios.

[0094] In this embodiment of the invention, the standardization processing of the 3D point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data includes:

[0095] Exposure processing is performed on the stitched panoramic image to reduce its brightness;

[0096] Perform color balancing on the stitched panoramic image to ensure accurate color reproduction, and then apply the color-balanced panoramic image accordingly.

[0097] Noise is removed from the color-balanced panoramic image, and then sharpened to obtain standard panoramic image data.

[0098] The feature descriptors of the 3D point cloud are obtained, and the boundary points and discovery locations of the point cloud are estimated by combining the feature descriptors of the 3D point cloud. The coordinate information of the point cloud is adjusted to obtain standard point cloud data.

[0099] S4. Obtain building information model data from the preset BIM software, and fuse the standard point cloud data, standard panoramic image data and building information model data based on the preset spatial coordinate system to construct a unified three-dimensional building model.

[0100] In this embodiment of the invention, the BIM software, namely Building Information Modeling software, is a digital tool used for building design, construction, and operation management. It achieves a three-dimensional digital representation of a project by creating and managing the entire lifecycle information of building assets.

[0101] Furthermore, by integrating building information model data, standard point cloud data, and standard panoramic image data from BIM software to construct a unified 3D building model, it is possible to achieve efficient collaboration in project management, improve design and construction quality, optimize resource allocation, enhance decision support, improve energy efficiency, and expand application areas, thereby bringing comprehensive benefits to the construction and other industries.

[0102] In this embodiment of the invention, the process of fusing the standard point cloud data, standard panoramic image data, and the building information model based on a preset spatial coordinate system to construct a unified three-dimensional building model includes:

[0103] The feature matching algorithm is used to find matching feature point pairs between different data sources;

[0104] Owe's ratio test is used to filter matching point pairs, removing bad matching feature point pairs to obtain filtered matching point pairs;

[0105] The homography matrix between the selected matching point pairs is calculated using the RANSAC algorithm, which describes the perspective transformation from one image to another, and the perspective transformation matrix is ​​obtained.

[0106] Ensure all data are in the same preset spatial coordinate system, correct for differences in coordinate systems, and obtain a corrected coordinate system;

[0107] The standard point cloud data, standard panoramic image data, and building information model data are fused together using the perspective transformation matrix and the correction coordinate system to obtain a unified three-dimensional building model.

[0108] In another embodiment of the present invention, points of interest can also be created in the three-dimensional building model. These points of interest are represented as location-based information points and can be linked with attribute information such as text, images, and hyperlinks. When a point of interest is clicked, its location will be automatically queried and located, and the corresponding descriptive information will be displayed. Combined with the AR positioning and visualization functions of mobile terminals, multi-terminal data interoperability can be achieved, satisfying the linkage between online and offline.

[0109] In another embodiment of the present invention, the three-dimensional building model can also be imported into a management platform, which provides a variety of measurement tools, including free measurement, horizontal measurement, vertical measurement, etc., to measure the data of the three-dimensional building model and export and share the measurement results.

[0110] Furthermore, the platform also allows for the comparison of point cloud, panoramic, and BIM data from different periods through the curtain and split-screen functionality provided by the platform.

[0111] Furthermore, the platform also supports collaborative linkage of large-space positioning devices, connecting online and offline, realizing multi-segment data synchronization and collaborative editing. In addition, it has the function of customizing data, which can share the scene through QR code or web link, enabling sharing with customers and colleagues, and allowing users to view the on-site situation and data details at any time on mobile devices.

[0112] This invention acquires multi-view point cloud data of a building using an intelligent scanning device, and obtains 3D point cloud data by aligning the multi-view point cloud data to the same coordinate system. Multiple panoramic images of the building are acquired using a panoramic camera, and these panoramic images are stitched together based on their feature points to construct 3D panoramic image data. The 3D point cloud data and the panoramic image data are standardized to obtain standard point cloud data and standard panoramic image data. Building information model data is acquired from preset BIM software, and the standard point cloud data, standard panoramic image data, and building information model data are fused based on a preset spatial coordinate system to construct a unified 3D building model. Therefore, the model construction method, device, electronic device, and computer-readable storage medium proposed in this invention, based on panorama, point cloud, and BIM, constructs a 3D building model by combining 3D point cloud data, panoramic image data, and building model information model data from BIM software, fully exploring the characteristics between various data types and improving the accuracy of the constructed 3D model.

[0113] like Figure 2 The diagram shown is a schematic diagram of the module of the model building device based on panorama, point cloud and BIM of the present invention.

[0114] The model building device 100 based on panorama, point cloud, and BIM described in this invention can be installed in an electronic device. Depending on the functions implemented, the model building device may include a point cloud data acquisition module 101, a panoramic image construction module 102, a data standardization module 103, and a data fusion module 104. The module described in this invention can also be called a unit, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0115] In this embodiment, the functions of each module / unit are as follows:

[0116] The point cloud data acquisition module 101 is used to acquire multi-view point cloud data of a building through an intelligent scanning device, and to obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system.

[0117] The panoramic image construction module 102 is used to acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images together according to the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0118] The data standardization module 103 is used to standardize the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data.

[0119] The data fusion module 104 is used to acquire building information model data from preset BIM software, and to fuse the standard point cloud data, standard panoramic image data and building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0120] In detail, the modules in the model building device 100 based on panorama, point cloud, and BIM described in this embodiment of the invention employ the same methods as described above when in use. Figure 1 The model building methods based on panorama, point cloud, and BIM use the same technical means and can produce the same technical effects, so they will not be elaborated here.

[0121] like Figure 3 The diagram shown is a structural schematic of the electronic device that implements the model building method based on panorama, point cloud and BIM according to the present invention.

[0122] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a model building program based on panorama, point cloud, and BIM.

[0123] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing model building programs based on panoramas, point clouds, and BIM), and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0124] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for model building programs based on panoramas, point clouds, and BIM, but also to temporarily store data that has been output or will be output.

[0125] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0126] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or, optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0127] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0128] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0129] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0130] The model building program based on panorama, point cloud, and BIM stored in the memory 11 of the electronic device is a combination of multiple computer programs. When run in the processor 10, it can achieve the following:

[0131] Multi-view point cloud data of a building is acquired by intelligent scanning equipment, and three-dimensional point cloud data is obtained by aligning the multi-view point cloud data to the same coordinate system.

[0132] Multiple panoramic images of the building are acquired using a panoramic camera device, and the multiple panoramic images are stitched together based on the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0133] The three-dimensional point cloud data and the panoramic image data are standardized to obtain standard point cloud data and standard panoramic image data.

[0134] Building information model data is obtained from the preset BIM software, and the standard point cloud data, standard panoramic image data and building information model data are fused together based on the preset spatial coordinate system to construct a unified three-dimensional building model.

[0135] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0136] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0137] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0138] Multi-view point cloud data of a building is acquired by intelligent scanning equipment, and three-dimensional point cloud data is obtained by aligning the multi-view point cloud data to the same coordinate system.

[0139] Multiple panoramic images of the building are acquired using a panoramic camera device, and the multiple panoramic images are stitched together based on the feature points of the panoramic images to construct three-dimensional panoramic image data.

[0140] The three-dimensional point cloud data and the panoramic image data are standardized to obtain standard point cloud data and standard panoramic image data.

[0141] Building information model data is obtained from the preset BIM software, and the standard point cloud data, standard panoramic image data and building information model data are fused together based on the preset spatial coordinate system to construct a unified three-dimensional building model.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0143] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0145] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0146] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0147] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0148] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0149] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for model building based on panorama, point cloud and BIM, characterized in that, The method comprises: obtaining multi-view point cloud data of a building through an intelligent scanning device, and obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; obtaining multiple panoramic images of the building through a panoramic camera device, and splicing the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data, wherein the splicing the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data comprises: comparing the multiple panoramic images, obtaining the same feature points in the overlapping areas of the multiple panoramic images through a feature point detection algorithm to obtain matching feature points; splicing the multiple panoramic images through the matching feature points to construct spliced panoramic images, comprising: calculating a transformation matrix between the panoramic images by using the matching feature points, and performing robust registration through a RANSAC algorithm; transforming the images into a unified coordinate system according to the transformation matrix, and splicing the transformed images together in a certain order to obtain spliced panoramic images; performing a smoothing operation on the boundary of the spliced panoramic images to obtain three-dimensional panoramic image data; performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; obtaining building information model data from a preset BIM software, and fusing the standard point cloud data, the standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model, wherein the fusing the standard point cloud data, the standard panoramic image data and the building information model based on a preset spatial coordinate system to construct a unified three-dimensional building model comprises: finding matching feature point pairs between different data sources through a feature matching algorithm; screening the matching point pairs by using a lowe's ratio test to remove poor matching feature point pairs to obtain screened matching point pairs; calculating a homography matrix between the screened matching point pairs by using a RANSAC algorithm to describe the perspective transformation from one image to another image to obtain a perspective transformation matrix; ensuring that all data are in the same preset spatial coordinate system, correcting the difference in the coordinate system to obtain a corrected coordinate system; fusing the standard point cloud data, the standard panoramic image data and the building information model data according to the perspective transformation matrix and the corrected coordinate system to obtain a unified three-dimensional building model.

2. The panoramic, point cloud and BIM based model building method of claim 1, wherein, The obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system comprises: obtaining point cloud data of a first view as first source point cloud from the multi-view point cloud data, and obtaining point cloud data of a second view corresponding to the source point cloud as target point cloud; querying the nearest neighbor point in the target point cloud for each point in the first source point cloud by setting an initial transformation matrix to construct a corresponding point set; calculating a rotation matrix and a translation vector of the corresponding point set by minimizing an objective function, and updating the point cloud position of the source point cloud by using the rotation matrix and the translation vector to obtain a transformation matrix; If the change of the transformation matrix is less than a preset threshold or reaches a maximum number of iterations, the iteration is stopped, and a final transformation matrix is obtained; The source point cloud data is registered to the target point cloud according to the final transformation matrix, and first three-dimensional point cloud data is obtained; Third-view point cloud data is selected from the multi-view point cloud data as second source point cloud, and the second source point cloud is registered to the first three-dimensional point cloud data, and second three-dimensional point cloud data is obtained, until all the point clouds of all the views in the multi-view point cloud data are registered, and the three-dimensional point cloud data is obtained.

3. The panoramic, point cloud and BIM based model building method of claim 1, wherein, The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points; The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points.

4. The panoramic, point cloud and BIM based modeling method of claim 1, wherein, The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points.

5. The panoramic, point cloud and BIM based modeling method of claim 3, wherein, The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. 6.A model construction apparatus based on panorama, point cloud and BIM, characterized by, The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature points in the overlapping regions and generating descriptors for the feature points. The matching feature points are obtained by using a feature detection algorithm to search for the same feature A data fusion module is configured to acquire building information model data from a preset BIM software, and fuse the standard point cloud data, the standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

7. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the model construction method based on panoramic, point cloud and BIM as claimed in any one of claims 1 to 5.

8. A computer readable storage medium comprising a storage data area and a storage program area, the storage data area storing created data, the storage program area storing a computer program; wherein, The computer program is executed by the processor to implement the model construction method based on panoramic, point cloud and BIM as claimed in any one of claims 1 to 5.

Images