System and method for checking building structure using augmented reality

Abstract

A checking system of a building structure includes a digital environment integration platform, an entity checking picture, a server and a mobile device, the digital environment integration platform is based on a general data environment (CDE), and the entity checking picture corresponds to a structural body component model of a building information model (BIM). The checking method comprises the following steps: the server is connected with the digital environment integration platform; the mobile device identifies feature data of the entity check picture and transmits the feature data to the server; the server obtains the corresponding structural body component model from the digital environment integration platform and transmits a connection between the structural body component model and a check table to the mobile device, and the check table conforms to the ISO19650 standard; the mobile device displays an environment image and an augmented reality (AR) element of the structural body component model on the screen; the mobile device is connected to the checking table.

Description

Technical Field

[0001] The present invention relates to the verification of building structures; in particular, it relates to a verification system and a verification method for building structures using augmented reality; more specifically, it relates to a verification system and a verification method for building structures using augmented reality based on a building information model. Background Art

[0002] Before the construction of buildings today, most of them will produce a virtual 3D model of the building, Building Information Modeling (BIM). In order to ensure that the constructed building structure is consistent with the building information model, the structural components of the building structure (such as beams, columns, windows, doorways, openings, etc.) must be checked at each stage of the construction process so that any errors can be corrected in time.

[0003] The existing building structure verification method is that the verification personnel carry a laptop computer storing a building information model and a paper verification form to the location of the structural components to be verified on the construction site, open the building information model with the laptop computer, search for the parts of the structural components to be verified one by one in the entire building information model, and then compare them with the actual structural components on the construction site. After the comparison is completed, the paper verification form is filled out.

[0004] However, it takes a considerable amount of time to find a specific location in a huge building information model, which will delay the verification work and lead to poor efficiency. In addition, it is easy to make omissions or errors when filling out many paper verification forms on the construction site.

[0005] Therefore, the existing inspection method for building structures is still not perfect and needs to be improved. Summary of the invention

[0006] In view of this, an object of the present invention is to provide a building structure inspection system and inspection method, which can effectively simplify the inspection process of structural components.

[0007] In order to achieve the above-mentioned purpose, the present invention provides a building structure verification system, comprising a digital environment integration platform, at least one physical verification picture, a server and a mobile device, wherein the digital environment integration platform stores at least one structural component model of a building structure; the at least one physical verification picture is a picture of the at least one structural component model; the server is connected to the digital environment integration platform; the server records a correspondence between a feature data of the at least one physical verification picture and the at least one structural component model, and the correspondence also includes a link to a verification table; the mobile device has a camera module and a screen, and the mobile device is connected to the server;

[0008] The mobile device uses the camera module to take a picture of the at least one entity verification picture and identifies feature data of the at least one entity verification picture; the mobile device transmits the feature data to the server;

[0009] wherein the server obtains the at least one structural component model from the digital environment integration platform according to the characteristic data and the corresponding relationship, and transmits the link between the at least one structural component model and the checklist to the mobile device;

[0010] Among them, the mobile device captures an environmental image through the photographic module and displays the environmental image on the screen, and the mobile device superimposes the obtained structural component model on the environmental image displayed on the screen as an augmented reality element; the mobile device connects to the checklist through the obtained link to edit the checklist.

[0011] The present invention provides a building structure verification method, comprising the following steps:

[0012] A digital environment integration platform is provided, wherein the digital environment integration platform stores at least one structural component model of a building structure;

[0013] Providing at least one physical verification picture, wherein the at least one physical verification picture is a picture of the at least one structural component model;

[0014] A server is provided, connected to the digital environment integration platform; the server records a correspondence between a feature data of the at least one entity verification image and the at least one structural component model, and the correspondence also includes a link to a verification table;

[0015] Connecting to the server through a mobile device;

[0016] The mobile device uses a camera module to take a picture of the at least one entity verification picture and recognizes feature data of the at least one entity verification picture; the mobile device transmits the feature data to the server;

[0017] The server obtains the at least one structural component model from the digital environment integration platform according to the feature data and the corresponding relationship, and transmits the link between the at least one structural component model and the checklist to the mobile device;

[0018] The mobile device captures an environment image through the camera module and displays the environment image on a screen of the mobile device, and the mobile device superimposes the obtained structural component model on the environment image displayed on the screen as an augmented reality element;

[0019] The mobile device connects to the checklist via the obtained link to edit the checklist.

[0020] The effect of the present invention is that users can check the structural components of a building structure by carrying a mobile device and a physical check picture, without having to search for the structural components to be checked one by one in the entire building information model of the building structure, and without having to carry a paper checklist. This makes it more convenient for users to check the building structure and effectively simplifies the check process of the structural components. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 FIG. 1 is a schematic diagram of a building structure verification system according to a first preferred embodiment of the present invention.

[0022] Figure 2 Flow chart of a building structure verification method according to a first preferred embodiment of the present invention.

[0023] Figure 3 It is a schematic diagram of a structural component model of the first preferred embodiment of the present invention.

[0024] Figure 4 It is a schematic diagram of a physical verification image according to a first preferred embodiment of the present invention.

[0025] Figure 5 It is a schematic diagram of a floor plan of a first preferred embodiment of the present invention.

[0026] Figure 6 Schematic diagram of an augmented reality element and an environment image displayed on a screen according to a first preferred embodiment of the present invention.

[0027] Figure 7 Schematic diagram of a field photo displayed on a screen according to a first preferred embodiment of the present invention.

[0028] Figure 8 Schematic diagram of an augmented reality element and an environment image displayed on a screen according to a first preferred embodiment of the present invention.

[0029] Fig. 9 It is a schematic diagram of an augmented reality element and an environment image displayed on a screen according to a second preferred embodiment of the present invention.

[0030] Description of reference numerals:

[0031] 1, 2, 3: Checkpoints

[0032] 10: Digital environment integration platform

[0033] 20: Entity verification image

[0034] 30: Server

[0035] 40: Mobile Devices

[0036] 42: Screen

[0037] 44: Photography Module

[0038] 46: Positioning module

[0039] 50: Structural component model

[0040] 502: Main inspection parts

[0041] 504: Secondary checkpoints

[0042] 52: Paper

[0043] 53: Floor Plan

[0044] 54: Location data

[0045] 56: Environmental Imagery

[0046] 58: Augmented reality components

[0047] 60: Operation interface

[0048] 62a~62e:Operation image

[0049] 64: Structural components

[0050] 66: Site Photos

[0051] 68: Prompt message

[0052] 70: Manipulating Images

[0053] S11~S18: Steps DETAILED DESCRIPTION

[0054] In order to more clearly illustrate the present invention, preferred embodiments are described in detail below with reference to the accompanying drawings. Figure 1 FIG. 1 is a first preferred embodiment of a building structure verification system of the present invention, comprising a digital environment integration platform 10, at least one physical verification image 20, a server 30 and a mobile device 40. The verification system is used to execute the verification method of the present embodiment, which includes Figure 2 The steps S11 to S18 are shown. The verification system and the verification method are described together below.

[0055] The digital environment integration platform 10 is based on a common data environment (CDE). The digital environment integration platform 10 can be, for example, a database or cloud space, which can be accessed online by relevant participants. The digital environment integration platform 10 stores at least one structural component model 50 of a building structure, and the structural component model 50 corresponds to a structural component of the building structure (such as a beam, a column, a window, a doorway, an opening, etc.). In this embodiment, there are multiple at least one structural component model 50. In implementation, the digital environment integration platform can be developed using Unity software.

[0056] like Figure 3 A structural component model 50 is shown. Each structural component model 50 is one of a plurality of structural component components of a building information model (BIM) of the building structure. In this embodiment, at least a portion of the structural component models 50 are defined with a primary check location 502 and a secondary check location 504. The primary check location 502 has a different color from the secondary check location 504, for example Figure 3 The middle opening is the main inspection part 502, and its color is different from other parts. The main inspection part 502 is the part where the size tolerance needs to be smaller, such as openings, windows, and doorways. The main inspection part on the actual structural component is located where other components, such as doors, frames, etc., need to be installed later, so the tolerance needs to be smaller. In practice, the building information model can be developed using Revit software.

[0057] In this embodiment, the digital environment integration platform 10 further stores at least one field photo corresponding to the structural component model 50. The at least one field photo is an electronic file of a 360-degree photo taken by a 360-degree camera. The field photo is a historical construction photo of the location of the structural component, which can be used for subsequent comparison. The at least one field photo can also be multiple, and each field photo is a historical construction photo of each stage of the location of the structural component.

[0058] In this embodiment, there are multiple at least one physical verification picture 20, each of which is printed on a paper 52, and each of which corresponds to a structural component model 50 of the building information model. Figure 4 Shown Figure 3 The physical verification picture 20 of the structural component model 50 is printed by a staff member on at least a portion of the structural component model 50 on a paper 52. In addition, the paper 52 further prints the structural component model 50 on a floor plan 53 ( Figure 5 Reference) on a position data 54. Figure 5The floor plan 53 in the figure has a plurality of checkpoints 1 to 3, each of which has a corresponding structural component model 50. Taking the structural component model 50 of checkpoint 1 as an example, the position data 54 includes, for example, the grid (line) position "Line: F / 5", and the position data 54 may also include other data indicating the position of the structural component model 50, such as the construction site name "OO Company Factory 1", the building type and floor "Building B 2F", and the checkpoint name "Checkpoint 1". The structural component model 50 and the physical check picture 20 are further marked with dimensions associated with the main checkpoint 502 for users to compare.

[0059] The server 30 is connected to the digital environment integration platform 10, and the server 30 is used to read the data in the digital environment integration platform 10. In practice, the digital environment integration platform 10 can also be integrated with the server 30 in a host. In practice, Github development server can be used. The server 30 records a correspondence between a feature data of each entity verification image 20 and each structural component model 50, and the correspondence also includes a link to a checklist corresponding to each structural component model 50. Each checklist is in the form of an electronic file and stored in the cloud space, and each link is used to link to the checklist located in the cloud space, but it is not limited to this, and each checklist can also be stored in the digital environment integration platform 10. Each checklist complies with the audit specifications of ISO19650. In practice, Vuforia software can be used to generate the feature data of the entity verification image 20, using the technology of Marker Augmented Reality (Marker AR).

[0060] The mobile device 40 is connected to the server 30. In this embodiment, the mobile device 40 may be, for example, a smart phone, a tablet computer, a laptop computer, or other device that is convenient for a user (such as an inspector) to carry and can be connected to the server 30 via the Internet. The mobile device 40 has a screen 42, a camera module 44, and a positioning module 46. The screen 42 may be, for example, a touch screen.

[0061] The mobile device 40 executes an application to connect to the server 30, and the user enters the account and password on the mobile device 40 to log in to the server 30. The positioning module 46 of the mobile device is used to obtain positioning data. For example, the positioning module 46 can obtain the positioning data through an indoor positioning device (not shown) set at the current floor of the construction site of the building structure. The indoor positioning device can, for example, use wireless signal positioning, such as WiFi or Bluetooth signals. The mobile device 40 obtains the positioning data based on the wireless signal received by the positioning module 46.

[0062] When the user walks to a check point to be checked at the construction site of the building structure, he takes out the paper physical check picture 20 for use.

[0063] Then, the user controls the mobile device 40 to scan the entity verification picture 20. That is, the mobile device 40 takes a picture of the entity verification picture 20 through the camera module 44 and identifies the feature data of the entity verification picture 20. The mobile device 40 transmits the feature data to the server 30.

[0064] The server 30 obtains a corresponding structural component model 50 from the digital environment integration platform 10 according to the feature data and the corresponding relationship. The server 30 transmits the link between the structural component model 50 and the corresponding checklist to the mobile device 40 .

[0065] Then, if Figure 6 As shown, the mobile device 40 continuously captures an environmental image 56 of the construction site through the photography module 44 and displays the environmental image 56 on the screen 42 of the mobile device 40 , and the mobile device 40 superimposes the obtained structural component model 50 as an augmented reality element 58 on the environmental image 56 displayed on the screen 42 .

[0066] In this embodiment, the corresponding relationship recorded by the server 30 further includes the position data 54 corresponding to each of the structural component models 50. In the step of the server 30 transmitting the link between the structural component model 50 and the corresponding checklist to the mobile device 40, the server further transmits the position data 54 corresponding to the structural component model 50 to the mobile device 40, and the mobile device 40 overlays the position data 54 on the screen 42.

[0067] In this embodiment, when the mobile device 40 displays the augmented reality element 58 on the screen 42, the primary check area 502 and the secondary check area 504 are displayed simultaneously, and the primary check area 502 has a different color from the secondary check area 504. Figure 6 In the embodiment, the opening of the augmented reality element 58 is shown in a darker color, thereby reminding the user that the main inspection area 502 needs to be paid attention to and needs to be compared in detail with the main inspection area (e.g., the opening) of the structural component 64. Preferably, the main inspection area 502 on the augmented reality element 58 can be marked as follows: Figure 3 and Figure 4 The dimensions associated with the main inspection location 502 are shown in the figure so that the user can measure the actual dimensions of the main inspection location of the structural component 64 with a ruler on site for comparison.

[0068] In this embodiment, the mobile device 40 further displays an operation interface 60 on the screen 42, and the operation interface 60 includes a plurality of operation images 62a to 62e. The operation image 62a is used to store the environmental image 56 as a static picture file and to store the environmental image 56 superimposed with the augmented reality element 58 as another static picture file. The operation image 62b is used to display the at least one field photo, and the operation image 62c is used to link the checklist. The operation image 62d is used to operate the rotation of the augmented reality element 58, and the operation image 62e is used to operate the translation of the augmented reality element 58. In addition, other operation images may also be included to operate the zooming in and out of the augmented reality element. Of course, the user also uses a finger to operate the rotation, translation, zooming in and out of the augmented reality element 58 on the screen 42. The user can compare the augmented reality element 58 with the structural component 64 in the environmental image 56.

[0069] When the user clicks the operation icon 62b, the mobile device sends a request command to the server 30, wherein the request command is to request to obtain the at least one scene photo. The server 30 obtains the at least one scene photo from the digital environment integration platform 10 according to the request command and sends it to the mobile device. Figure 7 As shown, the mobile device 40 displays the at least one scene photo 66 on the screen 42 , and the user can use a finger to manipulate the 360-degree scene photo 66 on the screen 42 to change the viewing angle, so as to facilitate comparison with the current situation on the construction site.

[0070] After the user clicks the operation icon 62 c , the mobile device 40 connects to the checklist of the corresponding structural component model 50 through the obtained link to edit the checklist.

[0071] Therefore, the user compares the augmented reality element 58 with the structural component 64 in the environmental image 56 to confirm whether the structural component 64 in the environmental image 56 is consistent with or consistent with the structural component model 50. The inspection result can be filled in the check list through the mobile device 40. The mobile device 40 further automatically adds the image file having the environmental image 56 and the other image file having the augmented reality element 58 and the environmental image 56 to the check list.

[0072] In addition, in this embodiment, user data of the user, such as the user's name, work number, etc., can also be established in the mobile device. When the mobile device edits the checklist, the mobile device substitutes the user data into the checklist to confirm that the checklist is filled out by this user.

[0073] Therefore, the user can carry the mobile device 40 and the physical verification image 20 to verify the structural components of the building structure without having to search for the structural components to be verified one by one in the entire building information model of the building structure, and without having to carry a paper verification sheet.

[0074] Optionally, the mobile device 40 generates a prompt message when comparing a structural component in the environment image with the shape of the augmented reality element 58 of the structural component model 50. The prompt message may be displayed on the screen 42, such as displaying "Comparison does not match".

[0075] Optionally, when the mobile device 40 compares the positioning data with the position data 54 and finds that the positioning data matches the position data 54, the augmented reality element 58 ( Figure 6 When the mobile device compares the positioning data with the position data 54 and finds that the positioning data does not match, a prompt message 68 is generated, such as Figure 8 As shown, a prompt message 68 of "the model does not match the on-site location" is displayed on the screen 42 to remind the user that the current location is incorrect. In practice, when the positioning data of the mobile device 40 does not match the location data 54, the augmented reality element 58 may not be displayed on the screen 42, but the prompt message 68 may be directly displayed until the user moves to the correct location and the mobile device 40 matches the positioning data of the location data 54. Then, the augmented reality element 58 is displayed and the prompt message 68 is canceled.

[0076] Please refer to Fig. 9 A second preferred embodiment of the present invention has substantially the same structure and steps as the first embodiment, except that when the mobile device 40 displays the augmented reality element 58 on the screen 42, an operation image 70 is further displayed on the screen, and the operation image 70 is a sliding bar for the user to adjust the transparency of the augmented reality element 58. When the user operates the operation image 70 on the screen 42, the mobile device 40 adjusts the transparency of at least a portion of the augmented reality element 58. For example, the mobile device 40 can adjust the transparency of the augmented reality element 58 as a whole, and the user can compare the semi-transparent augmented reality element 58 with the structural component 64 in the environmental image 56 on the screen 42.

[0077] Alternatively, when the user operates the operation image 70 on the screen 42, the mobile device 40 adjusts the transparency of a portion of the augmented reality element, for example, adjusting the main inspection portion 502 on the augmented reality element 58 to be semi-transparent. The mobile device 40 compares the semi-transparent main inspection portion 502 on the augmented reality element 58 with a portion of a structural component in the environmental image 56, that is, compares the semi-transparent main inspection portion 502 with the actual main inspection portion on the structural component, and generates a prompt message when the comparison does not match. The prompt message can be displayed on the screen 42, for example, displaying "Comparison does not match".

[0078] As described above, the building structure verification system and verification method of the present invention can make it more convenient for users to verify the building structure and effectively simplify the verification process of the structural components.

[0079] The above descriptions are only preferred feasible embodiments of the present invention. Any equivalent changes made by applying the description and claims of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A building structure verification system, comprising: A digital environment integration platform storing at least one structural component model of a building structure; at least one entity verification picture, which is a picture of the at least one structural component model; A server connected to the digital environment integration platform; the server records a correspondence between a feature data of the at least one entity verification image and the at least one structural component model, and the correspondence also includes a link to a verification table; a mobile device having a camera module and a screen, the mobile device being connected to the server; in, The mobile device uses the camera module to take a picture of the at least one entity verification picture and identifies feature data of the at least one entity verification picture; the mobile device transmits the feature data to the server; wherein the server obtains the at least one structural component model from the digital environment integration platform according to the characteristic data and the corresponding relationship, and transmits the link between the at least one structural component model and the checklist to the mobile device; Among them, the mobile device captures an environmental image through the photographic module and displays the environmental image on the screen, and the mobile device superimposes the obtained structural component model on the environmental image displayed on the screen as an augmented reality element; the mobile device connects to the checklist through the obtained link to edit the checklist.

2. A building structure verification system as described in claim 1, wherein the corresponding relationship also includes a position data corresponding to the at least one structural component model; the mobile device has a positioning module for obtaining a positioning data; the server further transmits the position data to the mobile device; when the mobile device compares the positioning data and the position data and finds that they match, the augmented reality element is displayed on the screen.

3. The building structure verification system as claimed in claim 2, wherein when the mobile device compares the positioning data and finds that it does not match the location data, a prompt message is generated. 4 . The building structure verification system as claimed in claim 2 , wherein when the mobile device compares the positioning data and finds that it does not match the location data, the augmented reality element is not displayed on the screen.

5. A building structure verification system as described in claim 1, wherein the digital environment integration platform includes at least one field photo corresponding to the at least one structural component model; the mobile device transmits a request instruction to the server, the request instruction is to request to obtain the at least one field photo; the server obtains the at least one field photo from the digital environment integration platform according to the request instruction and transmits it to the mobile device; the mobile device displays the at least one field photo on the screen.

6. The building structure verification system as claimed in claim 1, wherein the mobile device has a user data established therein, and when the mobile device edits the verification table, the user data is substituted into the verification table. 7 . The building structure verification system as claimed in claim 1 , wherein the mobile device generates a prompt message when comparing a structural component in the environmental image with the appearance of the augmented reality element of the at least one structural component model and finding that the appearance does not match.

8. A building structure inspection system as described in claim 1, wherein the at least one structural component model is defined with a primary inspection part and a secondary inspection part, and the color of the primary inspection part is different from that of the secondary inspection part; when the mobile device displays the augmented reality element on the screen, the primary inspection part and the secondary inspection part are displayed simultaneously.

9. A verification system for a building structure as described in claim 1, wherein the at least one structural component model is defined with a main verification part and a secondary verification part; wherein when the mobile device displays the augmented reality element on the screen, the mobile device adjusts the main verification part on the augmented reality element to be semi-transparent, and the mobile device compares the semi-transparent main verification part on the augmented reality element with a part of a structural component in the environmental image, and generates a prompt message when the comparison does not match. 10 . The architectural structure verification system of claim 1 , wherein when the mobile device displays the augmented reality element on the screen, the mobile device adjusts the transparency of at least a portion of the augmented reality element.

11. A method for checking a building structure, comprising the following steps: A digital environment integration platform is provided, wherein the digital environment integration platform stores at least one structural component model of a building structure; Providing at least one physical verification picture, wherein the at least one physical verification picture is a picture of the at least one structural component model; A server is provided, connected to the digital environment integration platform; the server records a correspondence between a feature data of the at least one entity verification image and the at least one structural component model, and the correspondence also includes a link to a verification table; Connecting to the server through a mobile device; The mobile device uses a camera module to take a picture of the at least one entity verification picture and recognizes feature data of the at least one entity verification picture; the mobile device transmits the feature data to the server; The server obtains the at least one structural component model from the digital environment integration platform according to the feature data and the corresponding relationship, and transmits the link between the at least one structural component model and the checklist to the mobile device; The mobile device captures an environment image through the camera module and displays the environment image on a screen of the mobile device, and the mobile device superimposes the obtained structural component model on the environment image displayed on the screen as an augmented reality element; The mobile device connects to the checklist via the obtained link to edit the checklist.

12. The building structure verification method according to claim 11, wherein the corresponding relationship further comprises a position data corresponding to the at least one structural component model; the mobile device has a positioning module for obtaining a positioning data; The building structure verification method further comprises: The server further transmits the location data to the mobile device; when the mobile device compares the positioning data and the location data and finds that they match, the augmented reality element is displayed on the screen.

13. The building structure verification method as claimed in claim 12, wherein when the positioning data and the location data do not match each other, the mobile device generates a prompt message.

14. The building structure verification method according to claim 12, wherein when the mobile device compares the positioning data and the location data and finds that the positioning data does not match, the augmented reality element is not displayed on the screen.

15. The building structure verification method according to claim 11, wherein the digital environment integration platform includes at least one field photo corresponding to the at least one structural component model; The building structure verification method further comprises: The mobile device transmits a request instruction to the server, wherein the request instruction is to request to obtain the at least one scene photo; the server obtains the at least one scene photo from the digital environment integration platform according to the request instruction and transmits it to the mobile device; the mobile device displays the at least one scene photo on the screen.

16. The building structure checking method as claimed in claim 11, wherein the mobile device has a user data established therein, and when the mobile device edits the checklist, the user data is substituted into the checklist.

17. The building structure verification method according to claim 11, further comprising: When the mobile device compares a structural component in the environment image and finds that the appearance of the augmented reality element of the at least one structural component model does not match, a prompt message is generated.

18. A building structure inspection method as described in claim 11, wherein the at least one structural component model is defined with a primary inspection part and a secondary inspection part, and the color of the primary inspection part is different from that of the secondary inspection part; when the mobile device displays the augmented reality element on the screen, the primary inspection part and the secondary inspection part are displayed simultaneously.

19. A building structure inspection method as described in claim 11, wherein the at least one structural component model is defined with a main inspection part and a secondary inspection part; wherein when the mobile device displays the augmented reality element on the screen, the mobile device adjusts the main inspection part on the augmented reality element to be semi-transparent, and the mobile device compares the semi-transparent main inspection part on the augmented reality element with a part of a structural component in the environmental image, and generates a prompt message when the comparison does not match. 20 . The architectural structure verification method according to claim 11 , wherein when the mobile device displays the augmented reality element on the screen, the mobile device adjusts the transparency of at least a portion of the augmented reality element.

Images