A mirror platform data processing method based on a BIM model

Abstract

The application discloses a mirror platform data processing method based on a BIM model, which comprises the following steps: synchronously connecting a panoramic camera and a mobile device on a site, establishing a reference coordinate system based on a starting point of shooting, selecting a marker, recording a marker coordinate and a reference direction of the panoramic camera, storing a path point coordinate and a coordinate generation time in a mobile shooting process, outputting the path point coordinate and a panoramic video frame after the mobile shooting is completed, making the path point obtain a coordinate position of a corresponding space point in the BIM model through coordinate transformation, and outputting a photo with the same camera visual angle direction as the BIM model according to a transformation angle of the camera visual angle relative to the reference direction at any path point in the BIM model.

Description

Technical Field

[0001] This application relates to the field of construction supervision, and in particular to a data processing method for a BIM model-based visualization platform. Background Technology

[0002] In the traditional construction industry, managers typically need to be physically present on-site to assess construction progress and resolve issues, significantly increasing management time costs. With technological advancements, Building Information Modeling (BIM) technology has emerged. By transmitting relevant information collected on-site to remote managers and comparing it with the BIM model, the current construction progress and existing problems can be clearly identified. Managers can then immediately propose solutions, greatly saving time. Building upon this, mirroring platforms have emerged that collect construction site photos and match them within the BIM model, allowing for the display of photos of the construction site at specified locations within the BIM model.

[0003] Currently, there are two main methods for capturing on-site photos: mobile acquisition and static acquisition. Mobile acquisition involves shooting video and recording waypoints during movement, and then extracting video frames as photos of those waypoints for display. Static acquisition involves fixing the camera at designated locations within the building. The mobile acquisition method requires addressing the issue of binding photos and waypoints obtained with spatial points in the BIM model, while the static acquisition method needs to address the problem of blind spots that result in incomplete capture. Summary of the Invention

[0004] This application addresses the problem in existing technologies where photos and path points acquired through mobile acquisition methods cannot be bound one-to-one with spatial points in a BIM model. It proposes a BIM model-based mirroring platform data processing method, which ensures that the acquired video frames and path points can not only be bound to spatial points within the BIM model, but also output photos with the same viewing direction as the camera in the BIM model.

[0005] This application is achieved through the following technical solution: A data processing method for a BIM model-based mirroring platform includes the following steps: S1. Initialize the panoramic camera and mobile device on site, complete the synchronous connection between the panoramic camera and the mobile device, and establish a reference coordinate system based on the shooting starting point. S2. Before starting the mobile acquisition, select a landmark, record the coordinates of the landmark in the reference coordinate system, and record the direction of the landmark in the panoramic camera and define it as the reference direction of the panoramic camera. S3. The panoramic camera starts to move and collect data under the drive of the mobile device. During the moving and collecting process, it stores the coordinates of the path points generated by the mobile device in the reference coordinate system and the coordinate generation time. During the moving process, the panoramic camera synchronously captures panoramic video. S4. End the moving shot and output the coordinates of the path points in the reference coordinate system and the panoramic video frames at the coordinate generation time. S5. Based on the coordinates of the marker in the reference coordinate system and its coordinates in the BIM model, calculate the relative relationship between the reference coordinate system and the BIM model coordinate system, and use coordinate transformation to enable the path points to obtain the coordinate positions of the corresponding spatial points in the BIM model. S6. Based on the change angle of the BIM model camera viewpoint relative to the reference direction at any path point in the BIM model, adjust the panoramic video frame corresponding to that path point and output a photo with the same viewpoint direction as the BIM model camera.

[0006] As a further improvement of the present invention, step S1 specifically includes the following steps: S11. Control the panoramic camera via HTTP protocol; S12. Based on the AR framework, establish an AR coordinate system with the shooting starting point as the origin as the reference coordinate system.

[0007] As a further improvement of the present invention, step S5 specifically includes the following steps: S51. Define the coordinates of the location of the marker in the reference coordinate system as [X...]. ref_0 ,Y ref_0 Z ref_0 The coordinates of the location of the marker in the BIM model coordinate system are [X...]. bim_0 ,Y bim_0 Z bim_0 The coordinates of any path point in the reference frame are [X]. ref_n ,Y ref_n Z ref_n ]; S52. Establish a first coordinate system with the location of the marker as the origin. The X, Y, and Z axes of the first coordinate system are parallel to and in the same direction as the X, Y, and Z axes of the reference coordinate system. Calculate the coordinates [X, Y, Z] of any path point in the reference system using the coordinate translation formula. ref_n ,Y ref_n Z ref_n After translation, the coordinates in the first coordinate system are [X]. no1_n ,Y no1_ n Z no1_n ]; S53. Establish a second coordinate system with the location of the landmark as the origin. The X, Y, and Z axes of the second coordinate system are parallel to and have the same direction as the X, Y, and Z axes of the BIM model coordinate system. Calculate the coordinates [X, Y, Z] of any path point in the first coordinate system according to the coordinate system rotation formula. no1_n ,Y no1_n Z no1_n The coordinates [X] in the second coordinate system after rotation no2_n ,Y no2_n Z no2_n ]; S54. Again, calculate the coordinates [X] of any path point in the second coordinate system using the coordinate translation formula. no2_n ,Y no2_n Z no2_n The coordinates [X] within the BIM model after translation bim_n ,Y bim_n Z bim_n ].

[0008] As a further improvement of the present invention, step S6 specifically includes the following steps: S61. Position the camera view of the BIM model at the spatial point corresponding to any path point, extract the panoramic video frame corresponding to the path point, and import the panoramic video frame into the sky sphere model to generate a sky sphere with panoramic photo texture. S62. Based on the angle transformation of the BIM model camera viewpoint relative to the reference direction and the observation range, output the corresponding panoramic photo texture in the sky sphere as a field photo from the BIM model camera viewpoint.

[0009] As a further improvement of the present invention, the mobile device further includes an IMU unit and / or an RTK module and / or a lidar.

[0010] As a further improvement of the invention, an auxiliary camera is also included, which includes one or more cameras.

[0011] As a further improvement of the present invention, the mobile device is equipped with a lidar, which marks points around the device with lasers during the mobile acquisition process and forms a temporary point cloud in the reference coordinate system.

[0012] As a further improvement of the present invention, the panoramic camera includes multiple cameras, the lenses of which are oriented at different angles to form a panoramic shooting range.

[0013] Implementing this invention has the following beneficial effects: it realizes the synchronous recording of path coordinates and video in time and space during the mobile acquisition process, binds the path points and panoramic video frames in the mobile acquisition process to the spatial points in the BIM model one by one, and can output on-site photos with the same viewing direction and range as the camera on the mirroring platform according to the different camera viewing angles in the BIM model. Attached Figure Description

[0014] To more clearly illustrate the technical solution of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the data processing method for the BIM model-based mirroring platform of the present invention; Figure 2 This invention relates to the coordinate transformation sequence and method of the BIM model-based mirroring platform data processing method. Figure 3 This is a schematic diagram of a BIM model in an embodiment of the data processing method for a mirroring platform based on a BIM model according to the present invention; Figure 4 This is the correspondence between panoramic video frames output at each path point and BIM spatial points in an embodiment of the BIM model-based mirroring platform data processing method of the present invention. Figure 5 This is a schematic diagram of observing a panoramic photo through a sky sphere model in an embodiment of the data processing method of the mirroring platform based on the BIM model of the present invention. Figure 6 This is an embodiment of the BIM model-based mirroring platform data processing method of the present invention, showing on-site photos and BIM model diagrams viewed on the mirroring platform; Figure 7 This is an embodiment of the BIM model-based mirroring platform data processing method of the present invention, showing on-site photos and BIM model diagrams viewed on the mirroring platform. Detailed Implementation

[0016] The technical solutions in the embodiments of the invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In this embodiment, the panoramic camera is a 360° panoramic camera, and the mobile device is a programmable mobile robot. The panoramic camera is fixed on the top of the mobile robot. The mobile robot can establish an AR reference coordinate system with the starting point of the movement as the origin and record the path coordinates during the movement. After initializing the panoramic camera and the mobile robot, the data acquisition functions of the panoramic camera and the mobile robot are synchronized. The panoramic camera is controlled to start shooting at the same time as the mobile robot starts moving via the HTTP protocol, and the panoramic camera stops shooting when the mobile robot stops moving, so as to ensure that the movement time of the mobile robot and the shooting time of the panoramic camera are consistent.

[0018] Before starting the moving shot, designate a marker on the wall at the location and record the initial position of the marker within the panoramic camera lens and the coordinates of the marker in the AR reference coordinate system [X]. ref_0 Y ref_0 Z ref_0 The direction in which the surface of the marker faces outward is denoted as the marker's orientation, and its Euler angle is denoted as the AR Euler angle (RX). ref RY ref RZ ref In this embodiment, the marker is initially located in the positive N direction of the panoramic camera. Therefore, the positive N direction is defined as the reference direction of the panoramic camera. In the BIM model coordinate system, the Euler angle towards the positive N direction is defined as (0,0,0). The orientation of the marker in the BIM model is denoted as the marker's BIM Euler angle (RX). beam RY beam, RZ beam ).

[0019] The mobile robot and panoramic camera are activated to begin data acquisition. During this process, the coordinates of the path points generated by the mobile robot in the AR reference coordinate system and the time of coordinate generation are stored. Simultaneously, the panoramic camera captures panoramic video during the movement. The coordinates of any path point in the AR reference coordinate system are [X...]. ref_n ,Y ref_n Z ref_n ].

[0020] After the mobile shooting ends, the coordinates of the path points in the AR reference coordinate system are output. The panoramic video captured is then output according to the coordinate generation time to correspond to the panoramic video frame in that direction. Since the movement time and shooting time are the same during the mobile acquisition process, the path points traversed during the movement at any coordinate generation time and the panoramic video frames output at that time are in one-to-one correspondence.

[0021] Establish a first coordinate system with the landmark as the origin, sharing the same coordinate axes as the AR reference coordinate system. According to the coordinate translation formula, the coordinates of any path point in the first coordinate system are [X...]. no1_n ,Y no1_n Z no1_n ], where the coordinate translation matrix T1 is: The matrix calculation relationship of the coordinates of any path point in the first coordinate system and the AR reference coordinate system is as follows: A second coordinate system is established with the marker's location as the origin, sharing the same coordinate axis directions as the BIM model coordinate system. Both the first and second coordinate systems originate from the marker's location. The marker has Euler angles identical to the AR Euler angles in the first coordinate system and identical to the BIM Euler angles in the second coordinate system. Therefore, the first coordinate system can be transformed into the second coordinate system by rotating it around its origin. To obtain the corresponding coordinate rotation matrix, the marker's AR Euler angles and BIM Euler angles are converted into matrix form using the following formula. → After the above transformation, the matrix after the AR Euler angle transformation is denoted as Rref, and the matrix after the BIM Euler angle transformation is denoted as Rbim. Since both the AR reference coordinate system and the BIM model coordinate system use Euler angles pointing to the positive N direction as (0,0,0), the following equation can be obtained through the transformation matrix Rchange: Rbim=Rchange*Rref Given Rref and Rbim, the matrix Rchange can be obtained through matrix calculation. The matrix calculation relationship between the coordinates of any path point in the second and first coordinate systems is as follows: Find the coordinates [X] of the spatial point where the landmark is located in the BIM model. bim_0 ,Y bim_0 Z bim_0 Then, using the coordinate translation formula, the coordinates [X] of the corresponding spatial point in the BIM model for any path point can be calculated. bim_n ,Y bim_ n Z bim_n This completes the binding of path coordinate points from the AR reference coordinate system to spatial points within the BIM model, where the coordinate translation matrix T2 is: The matrix calculation relationship of the coordinates of any path point in the BIM model coordinate system and the second coordinate system is as follows: The coordinates of the path points in the AR reference coordinate system are bound to the spatial points in the BIM model after translation → rotation → translation.

[0022] Within the BIM model, the model's camera is positioned at a spatial point corresponding to any path point. Panoramic video frames corresponding to these path points are extracted and imported into the sky sphere model to generate a sky sphere with panoramic photo textures. The sky sphere's viewing angle originates from its center and can be adjusted. Depending on the angle, the observed panoramic photo textures will differ. Viewing panoramic photos within the sky sphere model is as follows: Figure 5 As shown, and in this embodiment, the sky sphere still uses the positive N direction as the reference direction.

[0023] Adjust the camera viewpoint in the BIM model and calculate the angular transformation and viewing range of the camera viewpoint relative to the reference direction. Output the corresponding panoramic photo texture observed in the sky sphere as a site photo corresponding to the camera viewpoint of the BIM model, such as... Figure 6 , 7 As shown, the output site photos will also change accordingly as the camera viewpoint of the BIM model changes.

[0024] In other embodiments, mobile devices can employ drones to achieve a more three-dimensional mobile data collection process.

[0025] In other embodiments, the mobile robot is equipped with an RTK module, which can generate path point coordinates more accurately.

[0026] In other embodiments, the mobile robot is equipped with a lidar to emit lasers to mark points in the surrounding area during the mobile data acquisition process, thereby generating a temporary point cloud in the AR reference coordinate system to further refine the coordinates of the path points.

[0027] In some other embodiments, a secondary camera is also provided during mobile acquisition. This secondary camera is an additional panoramic camera used to further refine the orientation of the output photos to achieve binocular recognition similar to human eyes.

[0028] In other embodiments, the panoramic camera employs multiple high-resolution cameras, which are combined at certain angles to enable panoramic framing.

[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method for a mirroring platform based on a BIM model, characterized in that, Includes the following steps: S1. Initialize the panoramic camera and mobile device on site, complete the synchronous connection between the panoramic camera and the mobile device, and establish a reference coordinate system with the shooting starting point as the reference; the panoramic camera includes multiple cameras. S2. Before starting the mobile acquisition, select a landmark, record the coordinates of the landmark in the reference coordinate system, and record the direction of the landmark in the panoramic camera and define it as the reference direction of the panoramic camera. S3. The panoramic camera starts to move and collect data under the drive of the mobile device. During the moving and collecting process, it stores the coordinates of the path points generated by the mobile device in the reference coordinate system and the coordinate generation time. During the moving process, the panoramic camera synchronously captures panoramic video. S4. End the moving shot and output the coordinates of the path points in the reference coordinate system and the panoramic video frames of the panoramic video at the coordinate generation time. S5. Based on the coordinates of the marker in the reference coordinate system and the coordinates of its corresponding spatial point in the BIM model, calculate the relative relationship between the reference coordinate system and the coordinate system of the BIM model, and use coordinate transformation to enable the path point to obtain the coordinate position of the corresponding spatial point in the BIM model. S6. Based on the change angle of the BIM model camera viewpoint relative to the reference direction at any path point in the BIM model, adjust the panoramic video frame corresponding to that path point and output a photo with the same viewpoint direction as the BIM model camera. Step S5 specifically includes the following steps: S51. Define the coordinates of the location of the marker in the reference coordinate system as [X...]. ref_0 ,Y ref_0 Z ref_0 The coordinates of the location of the marker in the BIM model coordinate system are [X...]. bim_0 ,Y bim_0 Z bim_0 The coordinates of any path point in the reference frame are [X]. ref_n ,Y ref_n Z ref_n ]; S52. Establish a first coordinate system with the location of the marker as the origin. The X, Y, and Z axes of the first coordinate system are in the same direction as the X, Y, and Z axes of the reference coordinate system. Calculate the coordinates [X, Y, Z] of any path point in the reference coordinate system using the coordinate translation formula. ref_n ,Y ref_n Z ref_n After translation, the coordinates in the first coordinate system are [X]. no1_n ,Y no1_n Z no1_ n ]; S53. Establish a second coordinate system with the location of the landmark as the origin. The X, Y, and Z axes of the second coordinate system are in the same direction as the X, Y, and Z axes of the BIM model coordinate system. Calculate the coordinates [X, Y, Z] of any path point in the first coordinate system using the coordinate rotation formula. no1_n ,Y no1_n Z no1_n The coordinates [X] in the second coordinate system after rotation no2_n ,Y no2_n Z no2_ n ]; S54. Again, calculate the coordinates [X] of any path point in the second coordinate system using the coordinate translation formula. no2_n ,Y no2_ n Z no2_n The coordinates [X] within the BIM model after translation bim_n ,Y bim_n Z bim_n ]; Step S6 specifically includes the following steps: S61. Position the camera view of the BIM model at the spatial point corresponding to any path point, extract the panoramic video frame corresponding to the path point, and import the panoramic video frame into the sky sphere model to generate a sky sphere with panoramic photo texture. S62. Based on the angle transformation of the BIM model camera viewpoint relative to the reference direction and the observation range, output the corresponding panoramic photo texture in the sky sphere as a field photo from the BIM model camera viewpoint.

2. The data processing method for a BIM model-based mirroring platform according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Control the panoramic camera via HTTP protocol; S12. Based on the AR framework, establish an AR coordinate system with the shooting starting point as the origin as the reference coordinate system.

3. The data processing method for a BIM model-based mirroring platform according to any one of claims 1-2, characterized in that: The mobile device also includes an IMU unit and / or an RTK module.

4. The data processing method for a BIM model-based mirroring platform according to claim 1, characterized in that: It also includes auxiliary cameras, which include one or more of the panoramic cameras.

5. The data processing method for a BIM model-based mirroring platform according to claim 1, characterized in that: The mobile device is equipped with a lidar. During the mobile data acquisition process, the lidar marks points around the device with lasers and forms a temporary point cloud in the reference coordinate system.

6. The data processing method for a BIM model-based mirroring platform according to claim 1, characterized in that... The lenses of the multiple cameras are pointed at different angles to form a panoramic shooting range.

Images