Model construction method and device based on panorama, point cloud and BIM, equipment and medium

Abstract

The invention relates to the technical field of three-dimensional modeling, and discloses a panorama, point cloud and BIM-based model construction method, which comprises the following steps: acquiring multi-view point cloud data of a building, and aligning the multi-view point cloud data to the same coordinate system to obtain three-dimensional point cloud data; acquiring a plurality of panoramic images of the building, and splicing the plurality of panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data; performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; and obtaining building information model data from preset BIM software, and fusing the standard point cloud data, the standard panoramic image data and the building information model data based on a preset space coordinate system to construct a unified three-dimensional building model. The invention further provides a model construction device and equipment based on the panorama, the point cloud and the BIM and a storage medium. According to the invention, the accuracy of the constructed three-dimensional model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and in particular to a model construction method, device, equipment and medium based on panorama, point cloud and BIM. Background Art

[0002] In recent years, with the rapid development of the construction industry and the advancement of digital transformation, a large amount of data has been accumulated in the design, construction and operation and maintenance of construction projects, including building information models (BIM), point cloud data and panoramic images. These data each provide different perspectives and information levels of construction projects, but there is a lack of effective integration and coordination between them, resulting in serious information islands, which affects the efficiency and quality of construction projects.

[0003] However, existing technical means can often only process a single type of data and cannot achieve deep integration and comprehensive utilization of multi-source data. For example, although BIM data provides accurate information on building structures and components, it lacks an intuitive view of actual on-site construction; although point cloud data can capture the precise three-dimensional form of the building, it lacks color and texture information; although panoramic images can provide a panoramic view of the site, they cannot be combined with the precise size and structural information of the building. This data isolation limits the decision support and problem-solving capabilities of building projects during the design, construction, and operation and maintenance stages.

[0004] In addition, construction projects involve many parties, including designers, construction workers, operation and maintenance teams, and owners, who need to obtain construction information from different angles and levels to support their respective work. Existing technical means cannot meet such diversified and personalized information needs, resulting in high communication costs, low collaboration efficiency, and even project delays or quality problems due to information misunderstandings. Therefore, there is an urgent need for a model construction method based on panorama, point cloud, and BIM, focusing on the digital reconstruction and visualization of construction projects, deeply integrating and comprehensively utilizing multi-source building data, and solving the problems of information islands and low collaboration efficiency in construction projects. Summary of the invention

[0005] The present invention provides a model construction method, device, equipment and medium based on panorama, point cloud and BIM, the main purpose of which is to improve the accuracy of the constructed three-dimensional model.

[0006] To achieve the above object, the present invention provides a model construction method based on panorama, point cloud and BIM, comprising:

[0007] Acquire multi-view point cloud data of the building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system;

[0008] Acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data;

[0009] Performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data;

[0010] The building information model data is obtained from the preset BIM software, and the standard point cloud data, the standard panoramic image data and the building information model data are integrated based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0011] Optionally, obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system includes:

[0012] Acquire point cloud data of a first perspective from the multi-perspective point cloud data as a first source point cloud, and obtain point cloud data of a second perspective corresponding to the source point cloud as a target point cloud;

[0013] By setting an initial transformation matrix for each point in the first source point cloud, querying the nearest neighbor point in the target point cloud, and constructing a corresponding point set;

[0014] Calculate the rotation matrix and translation vector of the corresponding point set by minimizing the objective function, and update the point cloud position of the source point cloud by the rotation matrix and translation vector to obtain a transformation matrix;

[0015] If the change amount of the transformation matrix is ​​less than a preset threshold or reaches the maximum number of iterations, the iteration is stopped to obtain the final transformation matrix;

[0016] Registering the source point cloud data to the target point cloud according to the final transformation matrix to obtain first three-dimensional point cloud data;

[0017] Select third-view point cloud data from the multi-view point cloud data as the second source point cloud, and align the second source point cloud to the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, until the point clouds of all viewpoints in the multi-view point cloud data are aligned to obtain the three-dimensional point cloud data.

[0018] Optionally, the step of stitching the multiple panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data includes:

[0019] Comparing the multiple panoramic images, acquiring the same feature points in the overlapping areas of the multiple panoramic images by a feature point detection algorithm, and obtaining matching feature points;

[0020] Stitching the multiple panoramic images through the matching feature points to construct a stitched panoramic image;

[0021] A smoothing operation is performed on the boundary of the stitched panoramic image to obtain three-dimensional panoramic image data.

[0022] Optionally, acquiring the same feature points in the overlapping areas of the multiple panoramic images by using a feature point detection algorithm to obtain matching feature points includes:

[0023] Using a feature detection algorithm to find the same feature points in the overlapping area, and generating descriptors for the feature points;

[0024] The descriptors of feature points in different images are compared using a preset matching algorithm to obtain matching feature points.

[0025] Optionally, stitching the multiple panoramic images by using the matching feature points to construct a stitched panoramic image includes:

[0026] The transformation matrix between the panoramic images is calculated using the matching feature points, and robust registration is performed using a RANSAC algorithm;

[0027] The images are transformed into a unified coordinate system according to the transformation matrix, and the transformed images are stitched together in a certain order to obtain a stitched panoramic image.

[0028] Optionally, the performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data includes:

[0029] Performing exposure processing on the stitched panoramic image to eliminate the brightness of the stitched panoramic image;

[0030] Performing color balancing on the stitched panoramic image to ensure that the colors are consistent, and generating the panoramic image according to the color balance;

[0031] The noise in the color-balanced panoramic image is removed and sharpening processing is performed to obtain standard panoramic image data.

[0032] Optionally, the using a preset matching algorithm to compare descriptors of feature points in different images to obtain matching feature points includes:

[0033] Screening the feature points in the different images, removing bad feature points, obtaining screened feature points, and using the RANSAC algorithm to calculate the homography matrix between the screened feature point pairs;

[0034] Based on the homography matrix, calculate the positions of the vertices of the second image in the first image and the vertices covered by the vertices of the first image in the second image in different images, and use the vertices as initial feature points;

[0035] The first image and the second image are connected and drawn according to the initial feature points and the drawMatches function of OpenCV to obtain an initial matching result;

[0036] Verify whether the initial feature points in the initial matching result correctly identify the overlapping area in the image, and if so, obtain the matching feature points.

[0037] In order to solve the above problems, the present invention also provides a model construction device based on panorama, point cloud and BIM, the device comprising:

[0038] A point cloud data acquisition module, used to acquire multi-view point cloud data of a building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system;

[0039] A panoramic image construction module, used to obtain multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data;

[0040] A data standardization module, used for performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data;

[0041] The data fusion module is used to obtain building information model data from the preset BIM software, and fuse the standard point cloud data, standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0042] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0043] at least one processor; and,

[0044] a memory communicatively connected to the at least one processor; wherein,

[0045] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model building method based on panorama, point cloud and BIM as described above.

[0046] In order to solve the above problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area, the data storage area stores created data, and the program storage area stores a computer program; wherein, when the computer program is executed by a processor, the model building method based on panorama, point cloud and BIM as described above is implemented.

[0047] The embodiment of the present invention obtains multi-view point cloud data of a building through an intelligent scanning device, and obtains three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; obtains multiple panoramic images of the building through a panoramic camera device, and splices the multiple panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data; standardizes the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; obtains building information model data from a preset BIM software, and fuses the standard point cloud data, standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model. Therefore, the model construction method, device, electronic device and computer-readable storage medium based on panorama, point cloud and BIM proposed in the present invention construct a three-dimensional building model by combining three-dimensional point cloud data, panoramic image data and building model information model data in BIM software, fully exploits the characteristics between various data, and improves the accuracy of the constructed three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a model building method based on panorama, point cloud and BIM provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of a module of a model building device based on panorama, point cloud and BIM provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a model building method based on panorama, point cloud and BIM provided by an embodiment of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0053] The embodiment of the present application provides a model building method based on panorama, point cloud and BIM. The execution subject of the model building method based on panorama, point cloud and BIM includes but is not limited to at least one of the electronic devices such as the server, terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. Among them, the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. In other words, the model building method based on panorama, point cloud and BIM can be executed by software or hardware installed on a remote device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0054] Reference Figure 1 FIG. 1 is a flow chart of a model building method based on panorama, point cloud and BIM provided by an embodiment of the present invention. In this embodiment, the model building method based on panorama, point cloud and BIM includes the following steps S1-S4:

[0055] S1. Acquire multi-view point cloud data of a building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system.

[0056] It is understandable that acquiring multi-view point cloud data through intelligent scanning equipment can avoid the situation that scanning from a single perspective may not capture all the details of the building. Especially for large or complex structures, multi-view scanning can provide more comprehensive coverage, ensuring that all areas are recorded, and point cloud data acquired from different perspectives can verify and complement each other, improving the accuracy and reliability of the overall point cloud data.

[0057] In another embodiment of the present invention, each pixel in the depth image may be converted into a point in three-dimensional space to generate a point cloud.

[0058] In an embodiment of the present invention, aligning the multi-view point cloud data to the same coordinate system to obtain three-dimensional point cloud data includes:

[0059] Acquire point cloud data of a first perspective from the multi-perspective point cloud data as a first source point cloud, and obtain point cloud data of a second perspective corresponding to the source point cloud as a target point cloud;

[0060] By setting an initial transformation matrix for each point in the first source point cloud, querying the nearest neighbor point in the target point cloud, and constructing a corresponding point set;

[0061] Calculate the rotation matrix and translation vector of the corresponding point set by minimizing the objective function, and update the point cloud position of the source point cloud by the rotation matrix and translation vector to obtain a transformation matrix;

[0062] If the change amount of the transformation matrix is ​​less than a preset threshold or reaches the maximum number of iterations, the iteration is stopped to obtain the final transformation matrix;

[0063] Registering the source point cloud data to the target point cloud according to the final transformation matrix to obtain first three-dimensional point cloud data;

[0064] Select third-view point cloud data from the multi-view point cloud data as the second source point cloud, and align the second source point cloud to the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, until the point clouds of all viewpoints in the multi-view point cloud data are aligned to obtain the three-dimensional point cloud data.

[0065] In the embodiment of the present invention, the initial transformation matrix is ​​usually a unit matrix, which represents the initial transformation relationship from the source point cloud to the target point cloud, indicating that no transformation is performed. The minimization objective function is usually the sum of the squares of the distances between the corresponding point sets.

[0066] Furthermore, when the variation of the error function during transformation of the transformation matrix is ​​less than a preset threshold, the iteration is stopped to obtain the final transformation matrix.

[0067] In an embodiment of the present invention, the target point cloud is a point cloud used as a registration reference, and point cloud data from other perspectives need to be registered to this perspective. The first source point cloud and the second source point cloud represent point cloud data that need to be registered to the target point cloud. After registration, the point cloud data of this perspective will disappear.

[0068] S2. Acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data.

[0069] In an embodiment of the present invention, the panoramic imaging device may be a panoramic camera, and the panoramic image is an image with a wide viewing angle, which creates a seamless, continuous wide viewing angle picture by stitching together multiple individual images. Panoramic images are horizontal, vertical, or even spherical, and they can provide a wider field of view than traditional single images. In the present invention, multiple panoramic images may be stitched together to construct three-dimensional panoramic image data.

[0070] Furthermore, when a plurality of panoramic images of the building are acquired through a panoramic camera device, since the panoramic camera device has an ultra-wide-angle lens capable of capturing an extremely wide viewing angle, the captured image will be obviously distorted due to its special optical characteristics, that is, objects at the edge of the image will be stretched and distorted. Therefore, after acquiring a plurality of panoramic images, it is necessary to use image processing technology to correct this distortion so that the objects in the image are restored to their true shape and proportion in the real world, so that the buildings or other elements in the image will more realistically reflect their actual appearance and proportion.

[0071] In the embodiment of the present invention, the step of stitching the multiple panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data includes:

[0072] Comparing the multiple panoramic images, acquiring the same feature points in the overlapping areas of the multiple panoramic images by a feature point detection algorithm, and obtaining matching feature points;

[0073] Stitching the multiple panoramic images through the matching feature points to construct a stitched panoramic image;

[0074] A smoothing operation is performed on the boundary of the stitched panoramic image to obtain three-dimensional panoramic image data.

[0075] Specifically, the matching feature points refer to corresponding points found in different panoramic images, which represent the same physical location or feature in different images. The matching feature points can be detected by a feature point detection algorithm (such as SIFT, SURF or ORB) in the overlapping area of ​​multiple panoramic images, and then the corresponding point pairs are found by the feature point matching algorithm.

[0076] Furthermore, the method of acquiring the same feature points in the overlapping areas of the plurality of panoramic images by using a feature point detection algorithm to obtain matching feature points includes:

[0077] Using a feature detection algorithm to find the same feature points in the overlapping area, and generating descriptors for the feature points;

[0078] The descriptors of feature points in different images are compared using a preset matching algorithm to obtain matching feature points.

[0079] Furthermore, the use of a preset matching algorithm to compare descriptors of feature points in different images to obtain matching feature points includes:

[0080] Screening the feature points in the different images, removing bad feature points, obtaining screened feature points, and using the RANSAC algorithm to calculate the homography matrix between the screened feature point pairs;

[0081] Based on the homography matrix, calculate the positions of the vertices of the second image in the first image and the vertices covered by the vertices of the first image in the second image in different images, and use the vertices as initial feature points;

[0082] The first image and the second image are connected and drawn according to the initial feature points and the drawMatches function of OpenCV to obtain an initial matching result;

[0083] Verify whether the initial feature points in the initial matching result correctly identify the overlapping area in the image, and if so, obtain the matching feature points.

[0084] In the embodiment of the present invention, the RANSAC (RANdom SAmple Consensus) represents a random sampling consensus algorithm, which is an iterative method for estimating mathematical model parameters from a set of data containing outliers. It is suitable for situations where there is a large proportion of erroneous data or outliers in the data set, and the correct model parameters can still be robustly estimated.

[0085] Furthermore, the homography matrix, also known as the homography matrix or perspective transformation matrix, is an important mathematical tool used in computer vision and image processing to describe the perspective transformation relationship between two images. This transformation is usually used to map an image from one plane to another, such as mapping an image from one camera plane to another in stereo vision, or aligning images of different perspectives in panoramic image stitching.

[0086] In the embodiment of the present invention, the drawMatches function is a function in the OpenCV library, which is used to draw matching feature points between two images. This function is usually used after feature point detection and matching to facilitate visualization and verification of the results of feature point matching.

[0087] Furthermore, after using a feature detection algorithm (such as SIFT, SURF, ORB, etc.) to detect feature points in two images and using a feature matching algorithm to find matching pairs, the drawMatches function can help visually see which feature points are matched together.

[0088] In the embodiment of the present invention, the step of stitching the multiple panoramic images by matching feature points to obtain a stitched panoramic image includes:

[0089] The transformation matrix between the panoramic images is calculated using the matching feature points, and robust registration is performed using a RANSAC algorithm;

[0090] The images are transformed into a unified coordinate system according to the transformation matrix, and the transformed images are stitched together in a certain order to obtain a stitched panoramic image.

[0091] In an embodiment of the present invention, the transformation matrix can be 2D or 3D, and is used to describe linear transformations between points, lines, and surfaces in space, including rotation, translation, scaling, and shearing. Compared with the homography matrix, the transformation matrix is ​​suitable for a wider range of spatial transformations, such as three-dimensional modeling, object positioning in machine vision, etc.

[0092] S3. Standardize the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data.

[0093] In an embodiment of the present invention, the three-dimensional point cloud data and panoramic image data are standardized to ensure the consistency of the data in terms of measurement, format and quality, and to improve the comparability, compatibility and accuracy of the data, so that the data can be effectively stored, processed, analyzed and shared to meet the needs of different application scenarios.

[0094] In the embodiment of the present invention, the step of performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data includes:

[0095] Performing exposure processing on the stitched panoramic image to eliminate the brightness of the stitched panoramic image;

[0096] Performing color balancing on the stitched panoramic image to ensure that the colors are consistent, and generating the panoramic image according to the color balance;

[0097] removing noise from the color-balanced panoramic image and performing sharpening processing to obtain standard panoramic image data;

[0098] The feature descriptor of the three-dimensional point cloud is obtained, and the boundary points and discovery positions of the point cloud are estimated in combination with the feature descriptor of the three-dimensional point cloud, and the coordinate information of the point cloud is adjusted to obtain standard point cloud data.

[0099] S4. Acquire building information model data from a preset BIM software, and fuse the standard point cloud data, the standard panoramic image data, and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0100] In the embodiment of the present invention, the BIM software, i.e. Building Information Modeling software, is a digital tool for building design, construction and operation management. It realizes the three-dimensional digital representation of the project by creating and managing the full life cycle information of building assets.

[0101] Furthermore, integrating the building information model data, standard point cloud data and standard panoramic image data in BIM software to construct a unified three-dimensional building model can achieve efficient collaboration in project management, improve design and construction quality, optimize resource allocation, enhance decision-making support, improve energy efficiency, and expand application areas, thereby bringing comprehensive benefit improvements to industries such as construction.

[0102] In an embodiment of the present invention, the step of fusing the standard point cloud data, the standard panoramic image data and the building information model based on a preset spatial coordinate system to construct a unified three-dimensional building model includes:

[0103] Find matching feature point pairs between different data sources through feature matching algorithms;

[0104] Owe's ratio test is used to screen the matching point pairs, remove the poorly matched feature point pairs, and obtain the screened matching point pairs;

[0105] Using the RANSAC algorithm to calculate the homography matrix between the screened matching point pairs, describing the perspective transformation from one image to another image, and obtaining a perspective transformation matrix;

[0106] Ensure that all data are in the same preset spatial coordinate system, correct the differences in the coordinate systems, and obtain the corrected coordinate system;

[0107] The standard point cloud data, the standard panoramic image data and the building information model data are fused according to the perspective transformation matrix and the correction coordinate system to obtain a unified three-dimensional building model.

[0108] In another embodiment of the present invention, a point of interest can also be created in the three-dimensional building model. The point of interest is represented as an information point with a location, and can be connected with attribute information such as text, pictures, hyperlinks, etc. When the point of interest is clicked, the location of the point of interest will be automatically queried and located, and the corresponding introduction information will be displayed. In conjunction with the AR positioning and visualization function of the mobile terminal, multi-terminal data intercommunication can be achieved to meet the linkage between online and offline.

[0109] In another embodiment of the present invention, the three-dimensional building model can also be imported into a management platform, which provides a wealth of measurement tools, including free measurement, horizontal measurement, vertical measurement, etc., to measure the data of the three-dimensional building model, and export and share the measurement results.

[0110] Furthermore, the platform can also use the curtains and split screens provided by the platform to compare point cloud, panorama, and BIM data from different periods.

[0111] Furthermore, the platform also supports the coordinated linkage of large-space positioning devices, connects online and offline, realizes the synchronization and collaborative editing of multiple data segments, and also has the function of customizing data. It can share scenes through QR codes or web links, share with customers and colleagues, and view on-site conditions and data details at any time on the mobile terminal.

[0112] The embodiment of the present invention obtains multi-view point cloud data of a building through an intelligent scanning device, and obtains three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; obtains multiple panoramic images of the building through a panoramic camera device, and splices the multiple panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data; standardizes the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; obtains building information model data from a preset BIM software, and fuses the standard point cloud data, standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model. Therefore, the model construction method, device, electronic device and computer-readable storage medium based on panorama, point cloud and BIM proposed in the present invention construct a three-dimensional building model by combining three-dimensional point cloud data, panoramic image data and building model information model data in BIM software, fully exploits the characteristics between various data, and improves the accuracy of the constructed three-dimensional model.

[0113] like Figure 2 The figure shows a module schematic diagram of a model building device based on panorama, point cloud and BIM of the present invention.

[0114] The model building device 100 based on panorama, point cloud and BIM of the present invention can be installed in an electronic device. According to the functions implemented, the model building device based on panorama, point cloud and BIM can include a point cloud data acquisition module 101, a panoramic image construction module 102, a data standardization module 103 and a data fusion module 104. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0115] In this embodiment, the functions of each module / unit are as follows:

[0116] The point cloud data acquisition module 101 is used to acquire multi-view point cloud data of a building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system;

[0117] A panoramic image construction module 102 is used to obtain multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data;

[0118] A data standardization module 103 is used to perform standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data;

[0119] The data fusion module 104 is used to obtain building information model data from the preset BIM software, and fuse the standard point cloud data, the standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0120] In detail, each module described in the model construction device 100 based on panorama, point cloud and BIM in the embodiment of the present invention is used in the same manner as described above. Figure 1 The above-mentioned model building method based on panorama, point cloud and BIM has the same technical means and can produce the same technical effects, so it will not be repeated here.

[0121] like Figure 3 As shown, it is a structural schematic diagram of an electronic device for realizing a model building method based on panorama, point cloud and BIM according to the present invention.

[0122] The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a model building program based on panorama, point cloud, and BIM.

[0123] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing model building programs based on panorama, point cloud, and BIM, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0124] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the model building program based on panorama, point cloud and BIM, but also can be used to temporarily store data that has been output or is to be output.

[0125] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0126] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0127] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0128] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0129] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0130] The model building program based on panorama, point cloud and BIM stored in the memory 11 of the electronic device is a combination of multiple computer programs. When running in the processor 10, it can achieve:

[0131] Acquire multi-view point cloud data of the building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system;

[0132] Acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data;

[0133] Performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data;

[0134] The building information model data is obtained from the preset BIM software, and the standard point cloud data, the standard panoramic image data and the building information model data are integrated based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0135] Specifically, the specific implementation method of the processor 10 for the above computer program can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0136] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0137] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0138] Acquire multi-view point cloud data of the building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system;

[0139] Acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data;

[0140] Performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data;

[0141] The building information model data is obtained from the preset BIM software, and the standard point cloud data, the standard panoramic image data and the building information model data are integrated based on a preset spatial coordinate system to construct a unified three-dimensional building model.

[0142] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0143] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0145] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0146] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0147] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0148] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0149] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A model construction method based on panorama, point cloud and BIM, characterized in that: The method comprises: Acquire multi-view point cloud data of the building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; Acquire multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data; Performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; The building information model data is obtained from the preset BIM software, and the standard point cloud data, the standard panoramic image data and the building information model data are integrated based on a preset spatial coordinate system to construct a unified three-dimensional building model.

2. The model construction method based on panorama, point cloud and BIM according to claim 1, characterized in that: The step of obtaining three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system includes: Acquire point cloud data of a first perspective from the multi-perspective point cloud data as a first source point cloud, and obtain point cloud data of a second perspective corresponding to the source point cloud as a target point cloud; By setting an initial transformation matrix for each point in the first source point cloud, querying the nearest neighbor point in the target point cloud, and constructing a corresponding point set; Calculate the rotation matrix and translation vector of the corresponding point set by minimizing the objective function, and update the point cloud position of the source point cloud by the rotation matrix and translation vector to obtain a transformation matrix; If the change amount of the transformation matrix is ​​less than a preset threshold or reaches the maximum number of iterations, the iteration is stopped to obtain the final transformation matrix; Registering the source point cloud data to the target point cloud according to the final transformation matrix to obtain first three-dimensional point cloud data; Select third-view point cloud data from the multi-view point cloud data as the second source point cloud, and align the second source point cloud to the first three-dimensional point cloud data to obtain second three-dimensional point cloud data, until the point clouds of all viewpoints in the multi-view point cloud data are aligned to obtain the three-dimensional point cloud data.

3. The model construction method based on panorama, point cloud and BIM according to claim 1, characterized in that: The step of stitching the plurality of panoramic images according to the feature points of the panoramic images to construct three-dimensional panoramic image data includes: Comparing the multiple panoramic images, acquiring the same feature points in the overlapping areas of the multiple panoramic images by a feature point detection algorithm, and obtaining matching feature points; Stitching the multiple panoramic images through the matching feature points to construct a stitched panoramic image; A smoothing operation is performed on the boundary of the stitched panoramic image to obtain three-dimensional panoramic image data.

4. The model construction method based on panorama, point cloud and BIM according to claim 3, characterized in that: The method of acquiring the same feature points in the overlapping areas of the plurality of panoramic images by using a feature point detection algorithm to obtain matching feature points includes: Using a feature detection algorithm to find the same feature points in the overlapping area, and generating descriptors for the feature points; The descriptors of feature points in different images are compared using a preset matching algorithm to obtain matching feature points.

5. The model construction method based on panorama, point cloud and BIM according to claim 3, characterized in that: The step of stitching the plurality of panoramic images by using the matching feature points to construct a stitched panoramic image includes: The transformation matrix between the panoramic images is calculated using the matching feature points, and robust registration is performed using a RANSAC algorithm; The images are transformed into a unified coordinate system according to the transformation matrix, and the transformed images are stitched together in a certain order to obtain a stitched panoramic image.

6. The model construction method based on panorama, point cloud and BIM according to claim 1, characterized in that: The step of performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data includes: Performing exposure processing on the stitched panoramic image to eliminate the brightness of the stitched panoramic image; Performing color balancing on the stitched panoramic image to ensure that the colors are consistent, and generating the panoramic image according to the color balance; The noise in the color-balanced panoramic image is removed and sharpening processing is performed to obtain standard panoramic image data.

7. The model construction method based on panorama, point cloud and BIM according to any one of claims 1 to 6, characterized in that: The method of using a preset matching algorithm to compare the descriptors of feature points in different images to obtain matching feature points includes: Screening the feature points in the different images, removing bad feature points, obtaining screened feature points, and using the RANSAC algorithm to calculate the homography matrix between the screened feature point pairs; Based on the homography matrix, calculate the positions of the vertices of the second image in the first image and the vertices covered by the vertices of the first image in the second image in different images, and use the vertices as initial feature points; The first image and the second image are connected and drawn according to the initial feature points and the drawMatches function of OpenCV to obtain an initial matching result; Verify whether the initial feature points in the initial matching result correctly identify the overlapping area in the image, and if so, obtain the matching feature points.

8. A model building device based on panorama, point cloud and BIM, characterized in that: The device comprises: A point cloud data acquisition module, used to acquire multi-view point cloud data of a building through an intelligent scanning device, and obtain three-dimensional point cloud data by aligning the multi-view point cloud data to the same coordinate system; A panoramic image construction module, used to obtain multiple panoramic images of the building through a panoramic camera device, and stitch the multiple panoramic images according to feature points of the panoramic images to construct three-dimensional panoramic image data; A data standardization module, used for performing standardization processing on the three-dimensional point cloud data and the panoramic image data to obtain standard point cloud data and standard panoramic image data; The data fusion module is used to obtain building information model data from the preset BIM software, and fuse the standard point cloud data, standard panoramic image data and the building information model data based on a preset spatial coordinate system to construct a unified three-dimensional building model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model building method based on panorama, point cloud and BIM as described in any one of claims 1 to 7.

10. A computer-readable storage medium, comprising a data storage area and a program storage area, wherein the data storage area stores created data and the program storage area stores a computer program; wherein: When the computer program is executed by a processor, the model building method based on panorama, point cloud and BIM is implemented as described in any one of claims 1 to 7.

Images