A positioning method based on monocular camera and building BIM model

Abstract

The application relates to the technical field of computer vision, in particular to a positioning method based on a monocular camera and a building BIM model. The positioning method comprises the following steps: converting a building BIM model into a four-dimensional point cloud data set containing spatial coordinates and component type labels; calibrating the internal parameters and external parameters of a monocular camera; collecting an image and carrying out denoising, light correction and distortion correction; identifying the building component type in the image by using a classification neural network and extracting feature points with the component type labels; and solving the position coordinates and attitude angles of the camera relative to the BIM model by using an iterative optimization algorithm based on the component type label consistency constraint. The positioning method based on the monocular camera and the building BIM model reduces the positioning cost, improves the positioning accuracy and success rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a positioning method based on a monocular camera and a building BIM model. BACKGROUND

[0002] With the development of smart buildings, indoor navigation and industrial 4.0, high-precision indoor positioning technology has become a key infrastructure. The positioning methods in the industry can be roughly divided into hardware-dependent, hardware-free vision type and point cloud matching type.

[0003] Among them, the hardware-dependent type, such as the method based on UWB ultra-wideband, Bluetooth beacon or WiFi fingerprint, needs to deploy special base stations or signal tags, which has high implementation cost and complex maintenance; the hardware-free vision type avoids hardware cost, but in the repeated texture area of building structure (such as long corridor white wall, standardized office), the matching failure rate is more than 40% due to the lack of feature points, and the recognition effect is poor; the point cloud matching type uses the geometric coordinate information of the BIM model, but in the complex scene of dense doors and windows, the mismatching rate is more than 30%. Therefore, there is an urgent need in the art for a positioning method that can simultaneously meet the low hardware deployment cost and high positioning accuracy. SUMMARY

[0004] In order to solve the problems existing in the prior art, the purpose of the present application is to provide a positioning method based on a monocular camera and a building BIM model, which reduces the cost of positioning and improves the positioning accuracy.

[0005] To achieve the above purpose, the present application provides a positioning method based on a monocular camera and a building BIM model, comprising:

[0006] Converting the building BIM model into a four-dimensional point cloud data set containing spatial coordinates and component type labels;

[0007] Calibrating the intrinsic and extrinsic parameters of the monocular camera;

[0008] Collecting images and performing denoising, illumination correction and distortion correction;

[0009] Identifying the building component type in the image with a classification neural network, and extracting feature points with component type labels;

[0010] Based on the consistency constraint of the component type label, the position coordinates and attitude angle of the camera relative to the BIM model are solved by an iterative optimization algorithm.

[0011] Further, the component type label includes at least one of a wall, a door, a window, a column, a beam, a floor and a staircase.

[0012] Further, the specific steps of converting the building BIM model into a four-dimensional point cloud dataset containing spatial coordinates and component type labels include:

[0013] Point cloud sampling is performed on the BIM model geometric surface according to a preset sampling density;

[0014] A corresponding component type label is attached to each sampling point;

[0015] A mapping relationship between point cloud coordinates and component type labels is established to generate a four-dimensional point cloud dataset.

[0016] Further, the intrinsic calibration adopts Zhang Zhengyou calibration method, and the extrinsic calibration adopts PnP algorithm.

[0017] Further, the specific steps of collecting images and performing denoising, illumination correction and distortion correction include:

[0018] Gaussian filter kernel is used for image denoising;

[0019] Limiting contrast adaptive histogram equalization is used for illumination correction;

[0020] Radial distortion coefficient and tangential distortion coefficient of the monocular camera are used for distortion correction based on reverse mapping.

[0021] Further, the classification neural network includes any one of ResNet-50 and EfficientNet.

[0022] Further, the specific steps of extracting feature points with component type labels include:

[0023] SIFT, SURF and ORB algorithms are used simultaneously to extract feature points with component type labels, and each algorithm extracts up to 500 feature points;

[0024] Based on feature response intensity and spatial distribution, 1,000 optimal feature points are selected.

[0025] Further, the iterative optimization algorithm is an improved iterative closest point algorithm, and the specific steps include:

[0026] Image feature points are back-projected to three-dimensional space to form an observation point cloud;

[0027] In the four-dimensional point cloud dataset, only points with the same component type label as the observation points are matched;

[0028] A rotation matrix and a translation vector are solved by least squares method to minimize the distance square sum of the matched point pairs;

[0029] Iterative optimization is performed until the pose change between adjacent iterations is less than a threshold value or the maximum number of iterations is reached.

[0030] Further, KD-Tree is used in the matching process to accelerate the search, the search radius is set to 0.5 meters, and the feature points of different component types are assigned weights.

[0031] Further, the method further comprises:

[0032] The Kalman filter or particle filter algorithm is used for time domain smoothing of the pose estimation results of the continuous frames, so as to reduce the positioning jitter.

[0033] Further, the method further comprises:

[0034] A hierarchical positioning strategy is used for large building spaces, including:

[0035] In the coarse positioning stage, the low-resolution BIM point cloud is used to determine the region;

[0036] In the fine positioning stage, the high-resolution BIM point cloud of the region is loaded to solve the centimeter-level pose.

[0037] To achieve the above purpose, the application also provides a positioning system based on a monocular camera and a building BIM model, comprising: a BIM preprocessing module, used to convert the building BIM model into a four-dimensional point cloud data set with component type labels;

[0038] A camera calibration module is used to calibrate the intrinsic and extrinsic parameters of the monocular camera.

[0039] An image processing module is connected with the camera calibration module, used to receive the image collected by the monocular camera, and to perform denoising, illumination correction and distortion correction on the image based on the intrinsic and extrinsic parameters.

[0040] A feature recognition module is connected with the image processing module, comprising a classification neural network and a feature detection unit, used to perform component type identification and feature extraction on the processed image, and to generate a feature point set with component types.

[0041] A pose estimation module is connected with the feature recognition module and the BIM preprocessing module, used to execute the ICP algorithm with component type label constraint according to the feature points with component types and the four-dimensional point cloud data set with component type labels, and to output the three-dimensional position coordinates and attitude angles of the target.

[0042] The positioning method based on the monocular camera and the building BIM model provided by the application does not need to deploy a special base station or a signal tag, and has low deployment cost.

[0043] The positioning method based on the monocular camera and the building BIM model provided in the application can reach a positioning accuracy of 5-10 cm by fusing the structural prior knowledge of the BIM model and the visual feature information, and can improve the accuracy by 30-60 times compared with the traditional WiFi positioning method (3-5 m) and by 2-10 times compared with the pure visual SLAM method (20-50 cm).

[0044] The positioning method based on the monocular camera and the building BIM model provided in the application can effectively solve the positioning failure problem of the traditional visual method in the environment with repeated texture, light change and few features by using the feature matching strategy based on the building component type constraint, and can reach a positioning success rate of more than 95% under different light conditions, and can improve the positioning success rate by 15-25% compared with the pure visual method.

[0045] The positioning method based on the monocular camera and the building BIM model provided in the application can significantly reduce the search space by using the optimized ICP algorithm combined with the component type constraint, and the processing time of a single image is less than 2 seconds, which can improve the calculation efficiency by 3-5 times compared with the traditional ICP algorithm, and can meet the real-time navigation application requirements.

[0046] Other features and advantages of the application will be described in the following description, and some will become apparent from the description, or will be understood by those skilled in the art through implementation of the application. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings are intended to provide a further understanding of the application, and constitute a part of the specification, and together with the embodiments of the application, are used to explain the application, and do not constitute a limitation of the application. In the drawings:

[0048] Figure 1 The flowchart of the positioning method based on the monocular camera and the building BIM model of the embodiment 1 of the application is shown in FIG. 1.

[0049] Figure 2 The flowchart of the positioning method based on the monocular camera and the building BIM model of the embodiment 2 of the application is shown in FIG. 2.

[0050] Figure 3 The flowchart of the positioning method based on the monocular camera and the building BIM model of the embodiment 3 of the application is shown in FIG. 3.

[0051] Figure 4 The structural diagram of the positioning system based on the monocular camera and the building BIM model of the embodiment 7 of the application is shown in FIG. 7. DETAILED DESCRIPTION

[0052] Embodiments of the present application will be described below in greater detail. While certain embodiments of the present application are shown in the drawings, it is understood that the present application can be practiced by many forms that will not be exactly as shown in the drawings. It is also understood that the specific embodiments disclosed are illustrative in nature and are not intended to limit the scope of the present application, which is defined only by the claims. It is understood that the drawings and detailed description thereto are not intended to limit the scope of the present application.

[0053] It is understood that the various steps of the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0054] The term "comprising" and variations thereof as used herein are open-ended, and mean "including but not limited to". The term "based on" means "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related terms are defined in the description that follows.

[0055] It is to be noted that the terms "a" and "an" and "the" and "said" and "the" used in the present application are used generically and not in their exclusive sense unless otherwise expressly provided. It is to be understood that, where "one," "the," and / or "said" are used herein, that there is always a "one" or "the" or "said" for each and every reference and / or claim term that is used herein. It is also to be understood that "multiple" is understood to mean two or more.

[0056] Embodiment 1

[0057] In one embodiment of the present application, a positioning method based on a monocular camera and a building BIM model is provided, and the positioning method is applied to an office building, Figure 1 The flowchart of the positioning method based on a monocular camera and a building BIM model of Embodiment 1 of the present application will be described below with reference to Figure 1 The positioning method based on a monocular camera and a building BIM model of the present application will be described in detail, including:

[0058] Step S101: Convert the building BIM model into a four-dimensional point cloud data set containing spatial coordinates and component type labels.

[0059] In the embodiment, first, a BIM (Building Information Modeling, a digital representation containing architectural geometry and semantic information) model file (in IFC or Revit format) of an office building is acquired, which contains 3 floors and a total building area of 2000 square meters. Then, the BIM model is point clouded using an open source library Open3D or PCL, and a sampling density of 200 sampling points per square meter is set to uniformly sample the inner surface of the building (including walls, floors and ceilings) to generate about 400,000 three-dimensional coordinate points. Then, according to the component classification information in the BIM model, a component type label is attached to each point cloud point, including: wall (Wal, label number 1), door (Door, label number 2), window (Window, label number 3), column (Column, label number 4), floor (Slab, label number 5), etc. The finally constructed four-dimensional point cloud dataset is stored in the format: each row contains 4 values (x, y, z, label), x, y and z are the three-dimensional coordinates of the point cloud point, and label is the component type label of the point cloud point. The file size is about 50MB.

[0060] In some other embodiments, the sampling density can also be 100-500 sampling points per square meter. For example, for large building spaces, a hierarchical positioning strategy is used, in the coarse positioning stage, a sampling density of 50-100 sampling points per square meter is used, and in the fine positioning stage, a sampling density of 300-500 sampling points per square meter is used.

[0061] Step S102: calibrate the intrinsic and extrinsic parameters of the monocular camera.

[0062] In the embodiment, the monocular camera used is a monocular lens of a smart phone, with a resolution of 1920x1080 and a field of view of about 70 degrees. Zhang Zhengyou calibration method is used for intrinsic parameter calibration, and a 9x6 checkerboard calibration board is made. 20 groups of calibration images at different angles and distances are taken. The camera intrinsic parameters are calculated by the calibrateCamera function of the OpenCV library: focal length fx=1420 pixels, fy=1425 pixels; principal point coordinates cx=960 pixels, cy=540 pixels; radial distortion coefficients k1=-0.15, k2=0.08, k3=-0.02; tangential distortion coefficients p1=0.001, p2=-0.002. The re-projection error of calibration is 0.35 pixels, which meets the accuracy requirement. The extrinsic parameter calibration determines the spatial pose of the camera relative to the world coordinate system, including a 3x3 rotation matrix R and a 3x1 translation vector t. The camera extrinsic parameters are solved by the PnP (Perspective-n-Point) algorithm, which provides accurate geometric constraints for subsequent three-dimensional reconstruction and pose estimation.

[0063] Step S103: Collect images and perform denoising, illumination correction and distortion correction.

[0064] In this embodiment, for each frame of image, a 5x5 Gaussian filter kernel is used for filtering processing with a standard deviation σ = 1.0 to remove CCD sensor noise.

[0065] In this embodiment, the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm is used to enhance the local contrast of the image for illumination correction, with a clipping limit of 2.0 and a grid size of 8x8.

[0066] In this embodiment, based on the distortion coefficients obtained by calibration, the undistort function is used to eliminate radial and tangential distortion, ensuring the geometric accuracy of the image.

[0067] In some other embodiments, a bilateral filtering algorithm can also be used for filtering, with a filter kernel size of 5x5 or 7x7. In some other embodiments, a gamma correction algorithm can also be used for illumination correction, with a gamma value set between 0.8 and 1.2.

[0068] In this embodiment, based on the classification of building components, the ORB feature detection algorithm is used to extract image feature points, with a maximum number of feature points set to 1000 and a feature descriptor dimension of 256. Each feature point is assigned a component type label corresponding to its position to form a labeled feature point set. Typically, 800-1000 valid feature points can be extracted per image.

[0069] It can be understood that the classification neural network includes a feature extraction layer, a spatial attention mechanism, a global average pooling layer and a classification output layer.

[0070] In some other embodiments, a pre-trained EfficientNet classification neural network can also be used for building component classification. In some other embodiments, a SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features) or ORB (Oriented FAST and Rotated BRIEF) feature detection algorithm can also be used to extract image feature points, each of which contains a two-dimensional image coordinate (u, v), a 128-dimensional or 256-dimensional feature descriptor, and a corresponding component type label, forming a labeled feature point set {(ui, vi, di, labeli) | i = 1, 2,.., M}, where M is the number of extracted feature points, and di is the dimension.

[0071] Step S105: Based on the component type label consistency constraint, the position coordinates and attitude angles of the camera relative to the BIM model are solved by an iterative optimization algorithm.

[0072] In the present embodiment, a modified ICP (Iterative Closest Point) algorithm is used for pose estimation. First, the image feature points are back-projected to a three-dimensional space with a depth of 1 meter using the camera intrinsic parameters, forming an observed point cloud. Then, matching points are searched in the four-dimensional BIM point cloud dataset, with the matching strategy being: only considering point pairs with the same component type label, using KD-Tree to accelerate the nearest neighbor search, and setting the search radius to 0.5 meters. After establishing the correspondence between the point pairs, a least squares method is used to solve the rotation matrix R and the translation vector t, with the optimization objective function being to minimize the sum of the squares of the distances between the point pairs. The iteration process is set to have a maximum number of iterations of 50 times and a convergence threshold of a position change of less than 0.01 meters and an angle change of less than 0.1 degrees. Finally, the three-dimensional position coordinates (xyz) and attitude angles (roll, pitch, yaw) of the observer are output.

[0073] In some other embodiments, the maximum number of iterations is 50-100 times.

[0074] In the present embodiment, performance verification and detection are performed in five different types of buildings.

[0075] The five types include: an office building (2000m 2 ), a hospital (8000m 2 ), a shopping mall (12000m 2 ), a residential building (500m 2 ), and a factory (5000m 2). For each building, 100 test points were selected, totaling 50 test points, and the true position was measured as a benchmark using a high-precision total station.

[0076] Table 1 is a positioning performance comparison table:

[0077] Table 1

[0078] Positioning method Average error (cm) Standard deviation (cm) Maximum error (cm) Success rate (%) WiFi fingerprint 342 156 680 78 Bluetooth Beacon 185 89 420 85 UWB 28 12 65 92 Visual SLAM 35 18 85 88 This embodiment 7.2 3.1 15 96

[0079] As shown in Table 1, the positioning method of Embodiment 1 has significantly reduced average error, standard deviation, and maximum error compared to several positioning methods in the prior art, and the positioning success rate reaches 96%.

[0080] Table 2 is a calculation performance comparison table:

[0081] Performance indicators WiFi fingerprint Bluetooth Beacon UWB Visual SLAM This embodiment Processing time (s) 0.5 0.3 0.1 3.2 1.8 Memory usage (MB) 50 30 20 200 120 Deployment cost (ten thousand yuan) 15 25 50 0 0 Maintenance cost (ten thousand yuan / year) 3 5 8 0 0

[0082] As shown in Table 2, the processing time of the positioning method of Embodiment 1 is only 1.8s, the memory usage is only 120MB, and the deployment cost and maintenance cost are less than 1W yuan.

[0083] Table 3 is a qualitative performance comparison table under different lighting environments:

[0084]

[0085]

[0086] As shown in Table 3, the positioning success rate of the positioning method of Embodiment 1 under sunny conditions is as high as 98%, and the average error is only 6.5cm. The positioning success rate under dim environment can also be maintained at 92%, and the average error is only 9.1cm. That is, the positioning method of Embodiment 1 has strong positioning robustness under different lighting environments and is less affected by lighting.

[0087] Embodiment 2

[0088] In an embodiment of the present application, a positioning method based on a monocular camera and a building BIM model is provided, Figure 2 FIG. 2 is a flowchart of the positioning method based on a monocular camera and a building BIM model of Embodiment 2 of the present application. Referring to FIG. 2, Figure 2 The positioning method based on a monocular camera and a building BIM model of Embodiment 2 of the present application differs from Embodiment 1 in that:

[0089] In this embodiment, adaptive BIM point cloud sampling is adopted. According to the geometric complexity and component density of the building space, an adaptive sampling strategy is implemented. In areas with dramatic geometric changes (such as the connection between columns and beams around doors and windows), the sampling density is increased to 500 points per square meter, and in flat wall areas, the sampling density is reduced to 100 points per square meter. This strategy ensures the richness of key area features while reducing the overall data volume and improving matching efficiency.

[0090] In this embodiment, a multi-scale feature fusion strategy is adopted. In the feature extraction stage, three feature detection algorithms, SIFT, SURF, and ORB, are used simultaneously to take full advantage of the complementary advantages of different algorithms. The SIFT algorithm extracts scale-invariant features, which is suitable for handling distance changes; the SURF algorithm has high computational efficiency, which is suitable for real-time applications; and the ORB algorithm is robust to rotation changes, which is suitable for mobile scenarios. The upper limit of the number of feature points extracted by each algorithm is set to 500, and a total of 1500 candidate feature points can be obtained. Feature response strength and spatial distribution uniformity are used for feature point screening, and finally 1000 optimal feature points are retained.

[0091] In this embodiment, a weighted ICP optimization algorithm is used. In the ICP algorithm of Embodiment 1, feature importance weights are introduced. Different weights are assigned according to the response strength, component type rarity, and spatial distribution characteristics of the feature points. For example, the weight of structural feature points such as corners and door frames is set to 1.0, and the weight of ordinary wall feature points is set to 0.5. The optimization objective function is modified to a weighted least squares form to enhance the influence of key features.

[0092] The same 50 test points are used to verify the method of Embodiment 2 of the present application. It is found that the average positioning error is reduced to 5.8 cm, the standard deviation is reduced to 2.4 cm, and the maximum error is controlled within 11.2 cm. The calculation time increases slightly, with an average of 2.1 seconds, but the positioning accuracy is improved by about 20%. Therefore, the positioning method of Embodiment 2 of the present application has obvious advantages in high-precision requirement scenarios.

[0093] Embodiment 3

[0094] One embodiment of the present application provides a positioning method based on a monocular camera and a building BIM model, Figure 3 FIG. 1 is a flowchart of the positioning method based on a monocular camera and a building BIM model of Embodiment 3 of the present application. For a detailed description of the positioning method based on a monocular camera and a building BIM model of Embodiment 3 of the present application, please refer to Figure 3 The positioning method based on a monocular camera and a building BIM model of Embodiment 3 of the present application differs from Embodiment 1 in that:

[0095] In this embodiment, the positioning method is applied to a large commercial building, and a layered positioning and region switching strategy is adopted.

[0096] In this embodiment, the large building is divided into layers by floor and functional area, and each area maintains an independent four-dimensional BIM point cloud dataset. For example, a large shopping mall contains 3 floors, each with an area of 5000 square meters, and is divided into 16 sub-areas according to the functional area, each with an area of about 300 square meters. This division strategy effectively reduces the amount of data matched at a time and improves the calculation efficiency.

[0097] In this embodiment, a two-stage positioning mechanism is adopted: first, a coarse positioning stage is used to quickly determine the approximate area and floor where the observer is located using a low-resolution BIM model (sampling density of 50 points per square meter) and fewer feature points (200). After coarse positioning, the high-resolution BIM model (sampling density of 300 points per square meter) of the corresponding area is loaded for fine positioning. This two-stage mechanism controls the positioning time within 1.5 seconds while maintaining high accuracy.

[0098] In this embodiment, area switching and continuity are also ensured: when the observer moves between different areas, an overlapping area design and smooth transition algorithm are used to ensure positioning continuity. A 10% overlap area is set between adjacent areas, and when the observer enters the overlap area, the BIM data of both areas is loaded for positioning, and a Kalman filter algorithm is used to smooth the pose estimation results of consecutive frames in the time domain to avoid positioning jumps.

[0099] In some other embodiments, a particle filter algorithm can also be used to smooth the pose estimation results of consecutive frames in the time domain to reduce positioning jitter.

[0100] In this embodiment, multi-sensor data fusion is used: IMU (Inertial Measurement Unit) data from a smartphone, including three-axis accelerometer, three-axis gyroscope and three-axis magnetometer information, are combined to build a visual-inertial fusion positioning system. IMU data is used to predict the monocular camera motion trajectory to provide an initial estimate for the ICP algorithm, significantly improving the convergence speed and accuracy of the algorithm.

[0101] The method of Example 3 of the present application was tested in a large shopping center, with a building area of 15,000 square meters, including 3 floors and an underground parking lot. The total length of the test path is 1.2 kilometers, including floor switching, area conversion and other complex scenarios. The experimental results show that the average positioning error is 8.5 cm, with a slight increase in error at the area switching point but remaining within 15 cm, and the overall positioning success rate reaches 96%.

[0102] Example 4

[0103] One embodiment of the present application provides a positioning method based on a monocular camera and a building BIM model. The positioning method of embodiment 4 of the present application is different from that of embodiment 1 in that:

[0104] In the present embodiment, the positioning method is applied to an autonomous navigation system of an indoor service robot. The robot is equipped with a monocular camera and a laser radar sensor, and precise path planning and obstacle avoidance are realized in combination with a BIM model. On the basis of the depth information provided by the laser radar, the visual-BIM fusion positioning method provides an absolute position reference for the robot, solving the cumulative error problem existing in the pure SLAM method. Experimental results show that after the robot works continuously for 8 hours, the positioning error remains within 10 cm, which is reduced by 80% compared with the cumulative error of the pure visual SLAM method.

[0105] Embodiment 5

[0106] One embodiment of the present application provides a positioning method based on a monocular camera and a building BIM model. The positioning method of embodiment 5 of the present application is different from that of embodiment 1 in that:

[0107] In the present embodiment, the positioning method is applied to a construction and maintenance scene, and precise superimposed display of a BIM model and a real scene is realized in combination with AR technology. Construction personnel use AR glasses or a tablet device to obtain accurate position and attitude information through the positioning method of the present application, and the system superimposes pipeline, structure and other information in the BIM model on the real field of view. The positioning accuracy reaches 5 cm, meeting the precise AR superimposed display requirements and improving the construction efficiency and quality control level.

[0108] Embodiment 6

[0109] One embodiment of the present application provides a positioning method based on a monocular camera and a building BIM model. The positioning method of embodiment 6 of the present application is different from that of embodiment 1 in that:

[0110] In the present embodiment, the positioning method is applied to a fire, earthquake and other emergency rescue scene, and rescue personnel use the present application to perform indoor positioning and navigation in a smoky, dusty and other harsh environment. The system is pre-installed with a building BIM model, and rescue personnel obtain image information through a helmet camera. Even in an environment with extremely low visibility, reliable positioning can still be realized based on the structural features of the building. In combination with infrared imaging technology, the positioning success rate reaches 85% in an environment with a visibility of less than 2 meters, providing an important position reference for rescue personnel.

[0111] Embodiment 7

[0112] One embodiment of the present application provides a monocular camera and building BIM model based positioning system, Figure 4 FIG. 7 is a structural schematic diagram of a monocular camera and building BIM model based positioning system according to an embodiment of the present application, Figure 4 As shown in the figure, the positioning system comprises:

[0113] a BIM preprocessing module 100 configured to convert the building BIM model into a four-dimensional point cloud data set with component type labels;

[0114] a camera calibration module 200 configured to calibrate the intrinsic and extrinsic parameters of the monocular camera 800;

[0115] an image processing module 300 connected to the camera calibration module 200 and configured to receive images collected by the monocular camera 800, and perform denoising, illumination correction and distortion correction on the images based on the intrinsic and extrinsic parameters;

[0116] a feature recognition module 400 connected to the image processing module 300 and comprising a classification neural network and a feature detection unit, and configured to perform component type identification and feature extraction on the processed images to generate a feature point set with component types;

[0117] a pose estimation module 500 connected to the feature recognition module 400 and the BIM preprocessing module, and configured to execute a component type label constrained ICP algorithm based on the feature point set with component types and the four-dimensional point cloud data set with component type labels, and output the three-dimensional position coordinates and attitude angles of the target.

[0118] The above description is only some embodiments of the present application and a description of the principles of the technology applied. Those skilled in the art should understand that the disclosure range involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present application (but not limited to) having similar functions.

[0119] In addition, although each operation is described in a specific order, this should not be understood as requiring the operations to be performed in the specific order shown or in a sequential order. In certain circumstances, multi-tasking and parallel processing can be advantageous. Similarly, although the above discussion contains many specific implementation details, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0120] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A positioning method based on a monocular camera and a building BIM model, characterized in that, include: The specific steps to convert a building BIM model into a four-dimensional point cloud dataset containing spatial coordinates and component type labels include: sampling the geometric surface of the BIM model at a preset sampling density; attaching a corresponding component type label to each sampling point; establishing a mapping relationship between point cloud coordinates and component type labels to generate a four-dimensional point cloud dataset. Calibrate the intrinsic and extrinsic parameters of a monocular camera; Acquire images and perform noise reduction, illumination correction, and distortion correction; Use a classification neural network to identify the types of building components in images and extract feature points labeled with component types; Based on the component type label consistency constraint, the position coordinates and attitude angle of the camera relative to the BIM model are solved by an iterative optimization algorithm.

2. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The component type label includes at least one of the following: wall, door, window, column, beam, floor slab, and staircase.

3. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The internal parameter calibration uses the Zhang Zhengyou calibration method, and the external parameter calibration uses the PnP algorithm.

4. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The specific steps for acquiring images and performing noise reduction, illumination correction, and distortion correction include: Image denoising is performed using a Gaussian filter kernel; Illumination correction is performed using contrast-limited adaptive histogram equalization. Distortion correction is performed by inverse mapping of the radial and tangential distortion coefficients of a monocular camera.

5. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The classification neural network includes either ResNet-50 or EfficientNet.

6. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The specific steps for extracting feature points with component type labels include: Simultaneously, SIFT, SURF, and ORB algorithms are used to extract feature points with component type labels, with each algorithm extracting a maximum of 500 feature points; 1,000 optimal feature points were selected based on feature response intensity and spatial distribution.

7. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The iterative optimization algorithm is an improved iterative closest point algorithm, and the specific steps include: Image feature points are back-projected into three-dimensional space to form an observation point cloud; In the 4D point cloud dataset, only points with the same component type label as the observation point are matched; The rotation matrix and translation vector that minimize the sum of squared distances between matching point pairs are obtained by using the least squares method. Iterative optimization continues until the pose change between adjacent iterations is less than a threshold or the maximum number of iterations is reached.

8. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The KD-Tree was used to accelerate the search during the matching process, with the search radius set to 0.5 meters, and weights were assigned to feature points of different component types.

9. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The method further includes: Kalman filtering or particle filtering algorithms are used to smooth the pose estimation results of consecutive frames in the temporal domain, thereby reducing positioning jitter.

10. The positioning method based on a monocular camera and a building BIM model according to claim 1, characterized in that, The method further includes: A layered positioning strategy is adopted for large architectural spaces, including: In the coarse positioning stage, low-resolution BIM point cloud is used to determine the location area; In the fine positioning stage, the high-resolution BIM point cloud of the loaded area is used to solve the pose at the centimeter level.

11. A positioning system based on a monocular camera and a building BIM model, characterized in that, include: The BIM preprocessing module is used to sample point clouds on the geometric surface of the BIM model according to a preset sampling density, attach a corresponding component type label to each sampling point, establish a mapping relationship between point cloud coordinates and component type labels, and generate a four-dimensional point cloud dataset. Camera calibration module, used to calibrate the intrinsic and extrinsic parameters of a monocular camera; An image processing module, connected to the camera calibration module, is used to receive images acquired by a monocular camera and perform noise reduction, illumination correction, and distortion correction on the images based on intrinsic and extrinsic parameters. The feature recognition module, connected to the image processing module, includes a classification neural network and a feature detection unit, used to perform component class recognition and feature extraction on the processed image, and generate a set of feature points with component types; The pose estimation module is connected to the feature recognition module and the BIM preprocessing module. It is used to execute the ICP algorithm with component type label constraints based on feature points with component type and a four-dimensional point cloud dataset with component type labels, and output the three-dimensional position coordinates and pose angle of the target.

Images