Erection of formwork and scaffolding using mobile equipment

DE502020009684D1Inactive Publication Date: 2024-12-05PERI GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
DE502020009684
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-02-27
Filing Date
2020-02-25
Publication Date
2024-12-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Construction sites face challenges in accurately setting up formwork and scaffolding due to deviations from pre-planned designs, leading to inefficiencies and potential safety hazards, as site personnel often lack awareness of the planned specifications.

Method used

A method for calibrating a mobile device's internal coordinate system against an absolute construction site coordinate system using image recordings and sensor data, allowing for precise transformation of 3D models into the site's coordinate system for augmented reality overlays, enabling real-time error detection and correction.

Benefits of technology

This method enables accurate alignment and assembly of formwork and scaffolding, reducing errors and improving site efficiency by allowing for precise visualization and correction of deviations from the planned design, enhancing safety and stability.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present application claims priority from the German

[0002] Patent application No. 10 2019 105 015.4, filed on February 27, 2019.

[0003] The invention relates to computer-implemented support for the construction and inspection of formwork and scaffolding on construction sites. State of the art

[0004] When a structure is constructed by pouring concrete, a formwork is typically used as a mold for the fresh concrete. To quickly construct the formwork and reuse the material multiple times, formwork systems made of prefabricated standard elements are used. An example of such a formwork system is known from WO 2005 / 040 525 A1. Scaffolding is also constructed from prefabricated standard elements.

[0005] With the increasing professionalization of the formwork and scaffolding market, the complexity of formwork and scaffolding has also increased. Solutions designed for efficiency and pre-planned in CAD programs are often used. Whether such a solution achieves the desired technical effect in terms of material consumption, stability, or other

[0006] Achieving the criteria considered in the planning phase requires that the plan be precisely followed during physical construction on the construction site. Construction personnel are often unaware of this, leading to deviations from the plan in practice, thus rendering the project absurd. Support during construction by representatives of the construction suppliers who possess the necessary knowledge can prevent such errors, but is comparatively expensive.

[0007] US 9 918 204 B1 discloses a method for locating a mobile device within a building. After an initial rough estimate of the room in which the mobile device is located, a point cloud of the mobile device's surroundings is recorded and compared with a reference point cloud previously recorded in the building. The position and orientation of the mobile device are evaluated from the results of this comparison.

[0008] US 2015 / 310 135 A1 discloses a method for projecting information from a Building Information Model, BIM, into photos of a construction site in order to obtain a three-dimensional point cloud model of the construction site and to visualize the construction progress.

[0009] WO 2015 / 071 940 A1 discloses a mobile device that uses a camera to photograph a construction site and superimposes a building to be constructed, which is available as a three-dimensional model, into the image of the construction site as augmented reality at the correct location.

[0010] US 2017 / 256 097 A1 discloses a system for the precise localization of a mobile device using radio beacons for the purpose of being able to overlay augmented reality content as accurately as possible into images of the mobile device's surroundings. Task and solution

[0011] It is therefore the object of the invention to make mobile devices usable for computer support during the erection of formwork and / or scaffolding on construction sites.

[0012] This object is achieved according to the invention by a method for calibration according to main claim 1 and by a method for computer-aided quality control according to dependent claim 13. Further advantageous embodiments emerge from the dependent claims which refer back thereto. Disclosure of the invention

[0013] The invention is defined by the claims. Examples and embodiments not covered by the claims are presented to illustrate the claimed invention and facilitate its understanding.

[0014] Within the scope of the invention, a method was developed for calibrating an internal coordinate system of a mobile device against an absolute coordinate system of a construction site. This absolute coordinate system can, in particular, be a world coordinate system that assigns a latitude and longitude on the Earth's surface to each location. However, it can also be spanned from any other fixed point and use other coordinates. For example, the curvature of the Earth is not relevant for most construction projects, so Cartesian coordinates can be used.

[0015] In the method, at least one image is taken with the mobile device at a plurality of positions on the construction site, and optionally additionally with a plurality of orientations of the mobile device, each image showing at least one part of the construction site.

[0016] According to the invention, the image is captured with one or more cameras on the mobile device. For example, a single camera can capture a two-dimensional image. A combination of a camera and a depth sensor, or, for example, a stereoscopic arrangement of two or more cameras, can capture a three-dimensional image. A scanner that emits a laser beam, for example, can capture a scan that, for example, starting from the location of the mobile device, indicates the distance in each spatial direction to the point where the laser beam is reflected.

[0017] For example, a satellite-based positioning system can specify the position of the mobile device in world coordinates, which include fractions of latitude and longitude. The absolute coordinate system of the construction site can, for example, be a Cartesian coordinate system, with the world coordinates of at least one fixed point of this coordinate system known.

[0018] With each recording, the position and, optionally, the orientation of the mobile device in the internal coordinate system of the mobile device are also recorded using sensors. For this purpose, acceleration sensors, inertial sensors, tilt sensors, angular rate sensors, and / or gyroscopes can be used. For example, linear accelerations and / or angular accelerations can be double-integrated to extrapolate the position of the mobile device based on a last known position.

[0019] Equations are formulated with unknowns that characterize the desired transformation between the internal coordinate system of the mobile device and the absolute coordinate system of the construction site. The unknowns are determined as solutions to a system of equations formed from the formulated equations, thus obtaining the desired transformation.

[0020] The equations can be obtained in two different ways.

[0021] Firstly, according to the invention, at least one geometric feature with a known position in the absolute coordinate system of the construction site can be detected in an image and / or in a three-dimensional actual model of the construction site determined from several images. In conjunction with the sensor-detected position and orientation of the mobile device in the internal coordinate system of the mobile device, this results in an equation whose solutions characterize those transformations between the internal coordinate system of the mobile device and the absolute coordinate system of the construction site, in which the sensor-detected position and orientation of the mobile device in its internal coordinate system, the shape, orientation and position of the geometric feature in the image and the known position of the geometric feature in the absolute coordinate system of the construction site be consistent with each other as much as possible. For example, the shape, orientation, and position of the geometric feature in the image and / or scan can be predicted from the known position of the geometric feature in the absolute coordinate system in conjunction with the transformation of this position into the internal coordinate system of the mobile device and the position of the mobile device in this internal coordinate system.

[0022] If the transformation is correct, then the geometric feature should appear in the image exactly as predicted.

[0023] Secondly (not according to the invention), the position determined by the positioning system can be related to the position determined by the sensor in the internal coordinate system of the mobile device. This yields an equation whose solutions represent the transformations that allow the coordinates of the position determined by the positioning system in the absolute coordinate system of the construction site to be mapped as consistently as possible to the position determined by the sensor in the internal coordinate system of the mobile device.

[0024] Each of these equations typically has multiple solutions, so multiple equations are necessary to unambiguously define the desired transformation. Conversely, a solution to a system of equations consisting of multiple equations will generally not be able to satisfy all equations exactly, but will only satisfy each equation up to a certain error, in which case, for example, the mean square error can be minimized across all equations. The reason for this is that the physical measurements included in the equations are subject to unavoidable uncertainties.

[0025] In the system of equations, in particular equations based on the recognition of geometric features in images and / or scans and equations based on the direct comparison of coordinates determined in different ways can be combined with each other in any way.

[0026] It was discovered that by solving the described system of equations, a transformation between the internal coordinate system of the mobile device and the absolute coordinate system of the construction site could be determined that approximates reality to within a few millimeters or better. The key to this is the great flexibility in the type and quantity of information combined in the form of equations.

[0027] Typically, different construction sites have different conditions regarding the availability of information. For example, an outdoor construction site may have a clear view of the satellites of a navigation system, allowing the localization of individual points in global coordinates with an accuracy of down to a few centimeters. On a construction site in a covered hall, however, satellite-based navigation may be available only to a limited extent or not at all, while the lack of weather influences simultaneously enables better recognition of geometric features in images. Using the system of equations, the amount of physically available information can always be taken as given and optimally evaluated with regard to the desired transformation.

[0028] The precise transformation between the coordinate systems thus obtained, in turn, opens the door to computer support for the physical realization of formwork and / or scaffolding. As previously explained, complex formwork and scaffolding are planned in advance using CAD programs. This means that a three-dimensional model of the formwork or scaffolding is available in the absolute coordinate system of the construction site. Images or scans taken by the mobile device, which are intended to facilitate or verify the physical realization of the formwork or scaffolding, are available in the internal coordinate system of the mobile device. Therefore, the image in the absolute coordinate system of the construction site and / or the three-dimensional model in the internal coordinate system of the mobile device are required. This is precisely what the transformation determined using the described method achieves.

[0029] For example, the three-dimensional model of the formwork or scaffolding can be superimposed as "augmented reality" on a current view of the construction site from the perspective of the mobile device, so that this view is supplemented by the formwork or scaffolding to be erected.

[0030] For example, a perspective-correct two-dimensional projection of the three-dimensional model can be superimposed on a two-dimensional image currently captured by the mobile device's camera and displayed on a screen on a smartphone or tablet PC. The opacity of this projection can advantageously be selected so that the user can visually determine the extent to which the physical structure of the formwork or scaffolding matches the target model. The same is also possible using virtual reality or mixed reality glasses as a mobile device, which offers the additional advantage that the user can position each individual part precisely in its designated place according to the target model using both hands.

[0031] For example, such glasses can capture images of the construction site via one or more cameras aimed at the site, and these images can be transmitted to a display inside the glasses. When displayed on the display, the images can be overlaid with a two-dimensional projection of the three-dimensional model. The images of the construction site can also be processed with a wide-angle effect, for example, so that the display covers a larger field of view than is normally visible to the human eye.

[0032] Augmented reality glasses, for example, can also be used, allowing a direct optical view of the physical scene instead of recording the scene with a camera and displaying the image on a screen. The projection of the three-dimensional model can then be partially overlaid on the direct optical view, for example.

[0033] By directly comparing the physical scene on the construction site, deviations from the plan and errors in assembling the formwork or scaffolding can be identified immediately and corrected in a timely manner. This is particularly advantageous when a building or building section is not constructed in one go, but in several work phases: The precision with which the individual work phases seamlessly connect to one another is then advantageously improved.

[0034] Detectable errors can include, in particular, those that jeopardize the stability of the formwork or scaffolding, such as forgotten, swapped, or incorrectly installed fasteners (e.g., upside down). Not all of these errors can be prevented during the design of the standard elements, for example, by ensuring that only matching parts snap together in the correct orientation.

[0035] The construction of scaffolding or formwork supported by augmented reality can, beyond simply avoiding deviations from the plan and errors, generally improve the accuracy of the final construction of the building or part of a building. The transformation between the coordinate systems can be supported by a large number of equations, so that individual measurement errors during image capture, sensor-based positioning, and positioning with the positioning system are lost in the mass of information. In contrast, if formwork or scaffolding is planned conventionally, the construction plan is often drawn based on only a few measurements in the physical world. Small inaccuracies in these measurements or in manual marking then have an unhindered impact on the final result.

[0036] Alternatively, or in combination with the display of the target model as augmented reality, an automated comparison can also be performed between the target model of the formwork and at least one image and / or scan captured by a mobile device camera. Based on this comparison, it can be verified whether at least one component and / or accessory part of the formwork has been correctly installed on the construction site.

[0037] It is up to the user whether the comparison is performed in the internal coordinate system of the mobile device or in the absolute coordinate system of the construction site. If the comparison is performed in the internal coordinate system of the mobile device, for example, a perspectively correct two-dimensional projection of the three-dimensional model can be created and compared with the image captured by the mobile device's camera. However, it can also simply be checked whether the image is consistent with the hypothesis that the component and / or accessory has been installed correctly.

[0038] The geometric feature used to create one or more equations can, for example, be a two- or three-dimensional marking code applied to the construction site. The positions of such marking codes can be precisely measured or determined in the absolute coordinate system of the construction site.

[0039] In a further advantageous embodiment, at least one component and / or accessory part is selected as a geometric feature. For example, the positions of clearly recognizable pipes or other system components can be extracted from a building model of an industrial plant.

[0040] In another particularly advantageous embodiment, detecting the geometric feature involves determining a spatial direction of the mobile device relative to the geometric feature at the time of recording from the distortion of the geometric feature in the image and / or scan. Alternatively, or in combination with this, a distance of the mobile device relative to the geometric feature at the time of recording can be determined from the size of the geometric feature in the image.

[0041] In this way, the information about which geometric features are located at which known positions can be optimally utilized. In particular, for example, one and the same geometric feature, whose position and orientation have been measured once in the absolute coordinate system of the construction site, can be used multiple times for calibration by taking images from multiple distances and perspectives.

[0042] Furthermore, information about multiple geometric features recognizable in one and the same image can be extracted. This information is then linked to the same sensor-determined position of the mobile device in the mobile device's internal coordinate system.

[0043] In another particularly advantageous embodiment, the equations in the system of equations are assigned different weights depending on the reliability of the information they contain. This allows for the fact that not all of the many possible information sources can provide the same level of accuracy. Information with a lower level of reliability does not necessarily have to be discarded, but can also contribute to calibration, while simultaneously ensuring that particularly precise information is not diluted.

[0044] In another particularly advantageous embodiment, the system of equations is solved iteratively, starting with initial values ​​obtained from a priori information about the construction site. This way, the solution converges more quickly and reduces the probability of it "getting stuck" at a secondary optimum.

[0045] In a further particularly advantageous embodiment, the recognition of the geometric feature includes recognizing at least one surface and / or at least one point in the image and / or scan and assigning at least one surface or at least one point of the geometric feature.

[0046] Mature solutions are already available for surface recognition, so the only effort required is to assign the geometric feature to the surface. For example, the assignment can be requested from the mobile device operator.

[0047] For example, prominent points of geometric features can be used as points, such as corner points or intersection points of several geometric features.

[0048] In another particularly advantageous embodiment, at least one geometric feature is selected whose position in the absolute coordinate system of the construction site is derived from a predefined three-dimensional target model of the construction site and / or a structure to be constructed or modified. Calibration can then be automated to an even greater extent, since manual measurement of the position and orientation of the geometric feature is no longer required. In particular, significantly more geometric features can be analyzed and correspondingly more equations can be formulated to increase the accuracy of the ultimately obtained calibration.

[0049] For example, an operator can request at least one assignment of a geometric feature depicted in the image and / or scan to a geometric feature contained in the three-dimensional target model of the construction site. If automatic assignment is not possible without a doubt, such an operator contribution is a useful addition. Humans tend to have good skills in making such assignments, and unlike manually measuring the position and orientation of a geometric feature, there is no risk of a deterioration in accuracy.

[0050] As previously explained, geometric features can be recognized not only in images and / or scans, but alternatively or in combination with these, also in a three-dimensional as-built model of the construction site created from multiple images. By aggregating the information from multiple images and / or scans, such a three-dimensional as-built model can reduce the influence of inaccuracies that arise when capturing individual images, including due to the limited pixel resolution of the camera used.

[0051] In another particularly advantageous embodiment, creating a three-dimensional actual model of the construction site from multiple images involves recognizing three-dimensional patterns and shapes, and / or two-dimensional projections thereof, in the images. For example, the distortion and size of the pattern or shape can be used to determine the distance to the location where the shape or pattern is located and the perspective from which the shape or pattern is visible in the image. This information can be used to correctly position the pattern or shape in the three-dimensional actual model.

[0052] Alternatively, or in combination with this, the pixels of the images can be aggregated into a point cloud in three-dimensional space, for example, using photogrammetry methods. The three-dimensional patterns or shapes can then be recognized in this point cloud.

[0053] For example, a live data stream of the mobile device's camera image can be streamed to an external server, where the live camera image is divided into individual pixels. From the composition of the individual pixels, patterns and shapes, or two-dimensional projections thereof, can then be recognized, for example using machine learning. The analysis of the composition can relate, for example, to the distances between the pixels, the color information of the individual pixels, and / or the accumulation or grouping of pixels with similar color information and the shape of this grouping. The ongoing analysis of the data transmitted with the live transmission can then lead to the three-dimensional actual model of the mobile device's environment, which is composed of points in three-dimensional space that form a point cloud.This actual model can then be compared with a three-dimensional building model, for example with regard to the extracted color information and patterns.

[0054] Color information and extracted patterns form two information layers on which the comparison can be performed, with the majority of the information being contained in the extracted patterns, i.e., the shape information. The color information can be used to pre-filter which elements of the building model are included in the shape information comparison. If, after scaling, rotating, translating, or other operations, at least a specified percentage of the points in the point cloud match parts of the three-dimensional building model, this means that the geometric features in the three-dimensional building model were recognized in the actual three-dimensional model.The positions and orientations of the mobile device recorded by sensors during image acquisition, in conjunction with the operations necessary to bring the actual model into line with the three-dimensional building model to the specified degree of agreement, result in one or more equations for the desired transformation between the internal coordinate system of the mobile device and the absolute coordinate system of the construction site.

[0055] The higher the degree of agreement between the three-dimensional actual model and the three-dimensional building model, the more restricted the solution space of the equations resulting from the comparison is. A very good agreement between the three-dimensional point cloud extracted from the stream of two-dimensional images and the three-dimensional building model can clearly, or almost clearly, define the desired transformation between the internal coordinate system and the absolute coordinate system. If, however, the agreement is less good, the equations resulting from the comparison of the point cloud with the building model can be supplemented with additional equations. For example, equations obtained from comparing a position determined with a positioning system in the absolute coordinate system of the construction site with a sensor-determined position in the internal coordinate system of the mobile device can be used.The direct recognition of geometric features in the two-dimensional images can also be used to obtain further equations.

[0056] In another particularly advantageous embodiment, image pixels with similar color information are considered to belong to the same three-dimensional pattern, the same three-dimensional shape, and / or the same point in three-dimensional space. This is based on the realization that many objects on construction sites (such as building elements or accessories) have comparatively few colored features.

[0057] Accordingly, in a further advantageous embodiment, geometric features that are recognized in the three-dimensional actual model can be preselected based on the color information of the pixels in images.

[0058] According to the above description, the invention also relates to a method for computer-assisted quality control of a given formwork and / or scaffolding on a construction site. First, an internal coordinate system of a mobile device is calibrated to an absolute coordinate system of the construction site using the above-described method.

[0059] On the basis of this calibration, a three-dimensional target model of the formwork or scaffolding created in the absolute coordinate system of the construction site and / or at least one spatially resolved state variable of the formwork or scaffolding recorded in the absolute coordinate system of the construction site can be transformed into the internal coordinate system of the mobile device. The target model and / or the state variable of the formwork or scaffolding can then be overlaid with a current view of the construction site from the perspective of the mobile device. As previously described, this overlay can be created from a camera image recorded by the mobile device and shown on a display of the mobile device. The overlay can also, for example, be partially transparent to a direct optical view of the physical scenery. This allows a visual check to be made as to whether the actual physical structure of the formwork or scaffolding is correct.of the framework, corresponds to the three-dimensional target model.

[0060] Alternatively, or in combination with this, a comparison between the target model of the formwork or scaffolding, on the one hand, and at least one image taken by the mobile device and / or at least one actual model of the construction site determined from several such images, on the other hand, can be used to check whether at least one component and / or accessory part of the formwork or scaffolding has been correctly installed on the construction site. This check can be performed fully automatically and does not even require an image to be output to the mobile device user.

[0061] The state variable can, in particular, be a variable that is important with regard to the quality and / or safety of the formwork or scaffolding, but cannot be perceived in the physical world by a human without technical aids. An example of this is a mechanical load to which the formwork, scaffolding, or a part thereof is exposed.

[0062] For example, a formwork filled with concrete is subjected to internal pressure due to the gravitational force of the concrete. Scaffolding components, for example, can be subjected to compressive forces, tensile forces, and / or torque.

[0063] The state variable can, for example, comprise output data from at least one sensor designed to directly or indirectly measure the state variable at one or more locations on the formwork or scaffolding. For example, strain gauges or other sensors for measuring loads can be attached to the formwork or scaffolding. However, the state variable can also be calculated, for example, on an external server. For example, given the internal geometry of the formwork and the amount of concrete poured, the gravitational pressure acting on each point on the formwork is known.

[0064] By visualizing mechanical loads, it's possible to identify assembly errors that could lead to uneven loading of the formwork or scaffolding and / or overloading in certain areas. For example, a missing component in one location may result in components in another location having to compensate for this missing component and therefore being subjected to greater loads. This can lead to sudden component failure, with the associated risk of accidents.

[0065] A component doesn't even have to be completely missing to cause such consequences. Improper assembly of the component can be enough. If, for example, four supports supporting a scaffold are unevenly long or incorrectly adjusted to the uneven ground, this can result in the weight of the scaffold being supported by only three supports instead of four. Visually, it's not obvious that the fourth support is only loosely resting on the ground and isn't transferring any force to the ground.

[0066] The method can be carried out at least partially by one or more computers or mobile devices and thus, in particular, can be implemented in software. Therefore, the invention also relates to a computer program with machine-readable instructions that, when executed on at least one computer and / or at least one mobile device, cause the computer or mobile device, respectively, to carry out one of the described methods. Likewise, the invention also relates to a machine-readable data carrier and / or a downloadable product containing the computer program. Special description section

[0067] The subject matter of the invention is explained below with reference to figures, without limiting the subject matter of the invention. It is shown: Figure 1 : Flowchart of an embodiment of the method 100; Figure 2: Exemplary extraction of equations 51-53 by recognition of 131 geometric features 22; Figure 3 : Exemplary generation of equations 54-56 by comparing positions 4a-4c determined by a positioning system with sensor-determined positions 12a-12c and / or orientations 13a-13c; Figure 4 : Example of obtaining equations 51-56 using a three-dimensional actual model 23 of construction site 2; Figure 5 : Flowchart of an embodiment of the method 200.

[0068] Figure 1is an exemplary flow diagram of an embodiment of the method 100. In step 110, at least one image scan 3a-3c is recorded at a plurality of positions on the construction site 2, and optionally additionally with a plurality of orientations of the mobile device 1, which shows at least a part of the construction site 2. The user of the mobile device 1 can therefore, for example, walk across the construction site 2 with the mobile device and optionally also stand at a fixed position and rotate around its own axis. In addition, with each recording 110 according to block 111, the position 12a-12c and optionally additionally the orientation 13a-13c of the mobile device 1 are also recorded by sensors in the internal coordinate system 11 of the mobile device 1. The data recording in this way is in Figure 2 explained in more detail.

[0069] In combination with this, in step 120, at the plurality of positions, or with the plurality of orientations of the mobile device 1, the position 4a-4c of the mobile device 1 can be determined using a positioning system in a coordinate system 4 that is in a known, fixed relation 41 to the absolute coordinate system 21 of the construction site 2. In this case, analogous to block 111, according to block 121, the position 12a-12c and / or orientation 13a-13c of the mobile device 1 in the internal coordinate system 11 of the mobile device 1 is also recorded by sensors. The data acquisition in this way is in Figure 3 explained in more detail.

[0070] In step 130, equations 51-56 are set up in unknowns 5a-5c, which characterize the desired transformation between the internal coordinate system 11 of the mobile device 1 and the absolute coordinate system 21 of the construction site 2. This can be done individually or in combination in two ways. According to block 131, at least one geometric feature 22 with a known position 22a in the absolute coordinate system 21 of the construction site 2 is recognized in an image 3a-3c and / or in a three-dimensional actual model 23 of the construction site 2 determined from several such images. According to block 132, the position 4a-4c determined with the positioning system is related to the position 12a-12c and / or orientation 13a-13c determined by sensors in the internal coordinate system 11 of the mobile device 1.For the recognition of 131 geometric features 22, there are again several exemplary options, which are shown in more detail within box 131 and can be used individually or in combination.

[0071] According to block 131a, a spatial direction of the mobile device 1 relative to the geometric feature 22 at the time of image capture can be determined from the distortion of the geometric feature 22 in the image 3a-3c. This means that the distortion provides an indication of the perspective from which the geometric feature 22 was captured, and this perspective, in turn, indicates the spatial direction.

[0072] According to block 131b, a distance of the mobile device 1 to the geometric feature 22 at the time of recording can be determined from the size of the geometric feature 22 in the image 3a-3c. According to block 131c, at least one surface and / or at least one point can be detected in order to subsequently assign this surface or point to a corresponding surface or point of the geometric feature 22 in block 131d.

[0073] According to block 131e, an operator can request at least one assignment of a geometric feature 22 depicted in image 3a-3c to a geometric feature 22' contained in a three-dimensional target model 24 of the construction site 2. In this way, the known position 22a of the geometric feature 22' stored in the target model 24 can be assigned to the depicted geometric feature 22a.

[0074] Optionally, equations 51-56 can be weighted differently according to block 133 depending on the reliability of the information they contain.

[0075] Equations 51-56, obtained by whatever method, form a system of equations 5 in the unknowns 5a-5c. In step 140, this system of equations 5 is solved for the unknowns 5a-5c. According to block 141, an iterative approach can optionally be taken starting from initial values ​​obtained from a priori information about construction site 2. The values ​​of the unknowns 5a-5c obtained as solutions characterize the desired transformation between the internal coordinate system 11 of mobile device 1 and the absolute coordinate system 21 of construction site 2.

[0076] Figure 2shows an example of the derivation of equations 51-53 using two-dimensional images 3a-3c, which show different sections of the same construction site 2. In this example, as well as in Figure 3 , it is only required that the mobile device 1 can capture two-dimensional images 3a-3c. This capability is common to most mobile devices out of the box, while capturing three-dimensional images or scans requires advanced hardware or additional programs (apps).

[0077] Images 3a-3c each reside in the internal coordinate system 11 of mobile device 1, and during image capture, the respective position 12a-12c, as well as optionally the respective orientation 13a-13c, of mobile device 1 are recorded by sensors. In images 3a-3c, geometric features 22 with a known position 22a in the absolute coordinate system 21 of construction site 2 are recognized according to block 131. This known position 22a, with the fact that geometric feature 22 was recognized in images 3a-3c, in conjunction with the respective position 12a-12c and / or orientation 13a-13c of mobile device 1, is only consistent for certain transformations between the internal coordinate system 11 of mobile device 1 and the absolute coordinate system 21 of construction site 2. For example, the detection of a crane in an upright position is inconsistent with a transformation in which it should actually be upside down.On the other hand, for a single detection of feature 22 in an image 3a-3c, there are usually multiple transformations consistent with that detection. By combining several equations 51-53 into a system of equations 5, such ambiguities are resolved, and the influence of errors in physical detection is reduced.

[0078] Figure 3 clearly illustrates the derivation (not according to the invention) of equations 54-56 based on the comparison 132 between positions 4a-4c of the mobile device 1 determined by a positioning system, on the one hand, and the sensor-determined positions 12a-12c and / or orientations 13a-13c, on the other hand. In this example, the positions 4a-4c exist in a world coordinate system 4, which is in a known fixed relation 41 to the absolute coordinate system 21 of the construction site 2. In the Figure 3In the example shown, the absolute coordinate system 21 of construction site 2 is a Cartesian coordinate system that is "suspended" from a fixed point in world coordinates characterized by longitude and latitude. Any comparison of the positions in the different coordinate systems 4 and 11 yields at least one equation 54-56, where the world coordinates 4 can be expressed in coordinates of the absolute coordinate system 21 of construction site 2 using the known relation 41.

[0079] Figure 4 illustrates various possibilities of how a three-dimensional actual model 23 of construction site 2 can be obtained from several images 3a-3c according to block 135.

[0080] According to block 136, three-dimensional patterns and shapes, and / or two-dimensional projections thereof, can be recognized in images 3a-3c. The pattern or shape can then be directly inserted into a three-dimensional actual model 23 of construction site 2. This actual model refers to what can be seen at construction site 2, but resides in the internal coordinate system 11 of mobile device 1, in which the underlying images 3a-3c reside.

[0081] According to block 137, the pixels of images 3a-3c can be aggregated into a point cloud 23a in three-dimensional space, again still in the internal coordinate system 11 of the mobile device 1. The three-dimensional actual model 23 of the construction site 2 can then be generated according to block 138 by recognizing three-dimensional patterns or shapes in this point cloud 23a. This means that the point cloud provides clues as to which patterns or shapes should be used where in three-dimensional space, and the patterns or shapes then collectively form the three-dimensional actual model 23 of the construction site 2, still in the internal coordinate system 11.

[0082] The comparison with the absolute coordinate system 21 of construction site 2 only takes place in block 131, when geometric features 22 with known positions 22a in the absolute coordinate system 21 of construction site 2 are recognized in the three-dimensional actual model 23. Equations 51-56 are derived from this.

[0083] Optionally, for this comparison according to block 139, those geometric features 22 that can be recognized in the actual model 23 can be preselected based on the color information of pixels in images 3a-3c.

[0084] Optionally, according to blocks 136a, 137a, or 138a, pixels whose color information is similar can be considered to belong to the same pattern, the same shape, or the same point in three-dimensional space.

[0085] Figure 5shows an embodiment of the method 200 for computer-assisted quality control. In step 210, the internal coordinate system 11 of the mobile device 1 is calibrated to the absolute coordinate system 21 of the construction site 2 using the previously described method 100. The resulting solutions 5a-5c of the system of equations 5 characterize both a transformation from the internal coordinate system 11 to the absolute coordinate system 21 and a reverse transformation from the absolute coordinate system 21 to the internal coordinate system 11.

[0086] In step 220, a three-dimensional target model 24 created in the absolute coordinate system 21 of the construction site 2 can thus be transformed into the internal coordinate system 11 of the mobile device 1. In this internal coordinate system 11, the target model 24 can then be superimposed on a current view of the construction site 2 from the perspective of the mobile device 1 in step 230.

[0087] In step 240, the images and / or scans 3a-3c, and / or a three-dimensional actual model 23 of construction site 2 created therefrom, can be transformed from the internal coordinate system 11 into the absolute coordinate system 21 and compared there with the target model 24 of the formwork or scaffolding. Based on this comparison, it can be checked in step 250 whether at least one component and / or accessory part of the formwork or scaffolding has been correctly installed on construction site 2. List of reference symbols

[0088] 1Mobile device 11Internal coordinate system of mobile device 1 12a-12cPositions of mobile device 1 13a-13cOrientations of mobile device 1 2Construction site 21Absolute coordinate system of construction site 2 22, 22'Geometric feature 22aPosition of geometric feature 22 23Actual three-dimensional model of construction site 2 24Target three-dimensional model of scaffolding and / or formwork 3a-3cImages or scans 4Coordinate system of the positioning system 41Relation between coordinate systems 4 and 21 4a-4cPositions determined in coordinate system 4 5System of equations 5a-5cUnknowns of system of equations 5 51-56Equations forming system of equations 5 100Method for calibrating coordinate systems 11 and 21 110Taking images and / or scans 3a-3c 111Sensory detection of positions 12a-12c, orientations 13a-13c 120Determination of positions 4a-4c with positioning system 121Sensory detection of positions 12a-12c,Orientations 13a-13c 130Setting up equations 51-56 131Detecting geometric features 22 with known positions 22a 131aDetecting perspective from distortions in the image / scan 3a-3c 131bDetecting distances from the size in the image / scan 3a-3c 131cDetecting surfaces and / or points 131dMapping surfaces and / or points 131eRequesting an assignment from an operator 132Comparing positions 4a-4c with sensor measurements 12a-12c,13a-13c 133Weighting the equations 51-57 135Creating a three-dimensional actual model 23 136Recognizing the projections of shapes and patterns 136aConsidering color information 137Aggregating the images / scans 3a-3c into a point cloud 23a 137aConsidering color information 138Recognizing shapes and patterns in the point cloud 23a 138aConsidering color information 139Preselecting geometric features 22 based on color information 140Solving the system of equations 5 with respect to the unknowns 5a-5c 141Iterative approach to solving 140 200Quality control procedures 210Calibrating the coordinate systems 11, 21 using method 100 220Transforming the target model 24 into the coordinate system 11 230Overlaying the target model 24 with the current view of the construction site 2 240Comparing the target model with images / scans 3a-3c, actual model 23 250Checking for correct installation of component / accessory,

Claims

1. Method (100) for calibrating an internal coordinate system (11) of a mobile device (1) against an absolute coordinate system (21) of a construction site (2), comprising the steps of: at a plurality of positions on the construction site (2), and optionally additionally with a plurality of orientations of the mobile device (1), at least one image (3a-3c) showing at least part of the construction site (2) is recorded (110) in each case using the mobile device (1); · for each reception (110), the position (12a-12c) and optionally additionally the orientation (13a-13c) of the mobile device (1) in the internal coordinate system (11) of the mobile device (1) are additionally detected by sensors (111, 121); equations (51-56) are set up (130) with unknowns (5a-5c) which characterize the sought transformation between the internal coordinate system (11) of the mobile device (1) and the absolute coordinate system (21) of the construction site (2), wherein these equations (51-56) are obtained by at least one geometric feature (22) with a known position (22a) in the absolute coordinate system (21) of the construction site (2) is detected (131) in an image (3a-3c) and / or in a three-dimensional actual model (23) of the construction site (2) determined from a plurality of images (3a-3c); the unknowns (5a-5c) are determined (140) as solutions of a system of equations (5) formed from the established equations (51-56).

2. Method (100) according to claim 1, wherein at least one two- or three-dimensional marking code applied to the construction site (2) is selected as the geometric feature (22).

3. Method (100) according to any one of claims 1 to 2, wherein at least one component and / or accessory is selected as the geometric feature (22).

4. Method (100) according to any one of claims 1 to 3, wherein the recognition (131) of the geometric feature (22) involves determining (131a) a spatial direction of the mobile device (1) to the geometric feature (22) at the time of reception from the distortion of the geometric feature (22) in the image (3a-3c), and / or determining (131b) a distance of the mobile device (1) from the geometric feature (22) at the time of reception from the size of the geometric feature (22) in the image (3a-3c).

5. Method (100) according to any one of claims 1 to 4, wherein different weights are assigned (133) to the equations (51-56) in the system of equations (5) depending on the reliability of the information contained in them.

6. Method (100) according to any one of claims 1 to 5, wherein the system of equations (5) is solved iteratively (141) starting from initial values obtained from a priori information about the construction site (2).

7. Method (100) according to any one of claims 1 to 6, wherein the detection (131) of the geometric feature (22) involves detecting (131c) at least one surface and / or at least one point in the image (3a-3c) and assigning (131d) at least one surface or at least one point, respectively, to the geometric feature (22).

8. Method (100) according to any one of claims 1 to 7, wherein at least one geometric feature (22) is selected whose position (22a) in the absolute coordinate system (21) of the construction site (2) is derived from a predetermined three-dimensional nominal model (24) of the construction site (2) and / or of a structure to be erected or modified.

9. Method (100) according to claim 8, wherein at least one assignment of a geometric feature (22) depicted in the image (3a-3c) to a geometric feature (22') contained in the three-dimensional nominal model (24) of the construction site (2) is requested (131e) by an operator.

10. Method (100) according to any one of claims 1 to 9, wherein a three-dimensional actual model (23) of the construction site (2) is generated (135) from a plurality of images (3a-3c) by recognizing (136) three-dimensional patterns and shapes and / or two-dimensional projections thereof in the images ( 3a-3c) are recognized (136), and / or the pixels of the images (3a-3c) are aggregated (137) to form a point cloud (23a) in three-dimensional space and the three-dimensional patterns or shapes in this point cloud (23a) are recognized (138).

11. Method (100) according to claim 10, wherein image pixels whose color information is similar are evaluated (136a, 137a, 138a) as belonging to the same three-dimensional pattern or the same three-dimensional shape and / or to the same point in three-dimensional space.

12. Method (100) according to any one of claims 1 to 11, wherein geometric features (22) which are recognized (131) in the three-dimensional actual model (23) are preselected (139) on the basis of the color information of the pixels in images (3a-3c).

13. Method (200) for computer-aided quality control of a predetermined formwork and / or a predetermined scaffold at a construction site (2) comprising the steps of: an internal coordinate system (11) of a mobile device (1) is calibrated (210) with the method (100) according to any one of claims 1 to 12 to an absolute coordinate system (21) of the construction site (2); a three-dimensional nominal model (24) of the formwork and / or scaffold constructed in the absolute coordinate system (21) of the construction site (2), and / or at least one spatially resolved state variable of the formwork and / or scaffold recorded in the absolute coordinate system (21) of the construction site (2) is transformed (220) into the internal coordinate system (11) of the mobile device (1), and the nominal model (24) and / or the state variable of the formwork or scaffold is overlaid (230) on a current view of the construction site (2) from the perspective of the mobile device (1), and / or on the basis of a comparison (240) between the nominal model (24) of the formwork or the scaffold, on the one hand, and at least one image and / or scan (3a-3c) taken by the mobile device (1) and / or at least one actual model (23) of the construction site (2) determined from a plurality of such images and / or / or scans (3a-3c) (23) of the construction site (2), on the other hand, it is checked (250) whether at least one component and / or accessory of the formwork or the scaffold has been correctly installed on the construction site (2).

14. Computer program containing machine-readable instructions that, when executed on at least one computer and / or on at least one mobile device, cause the computer or the mobile device, respectively, to execute the computer-implementable steps of a method (100, 200) according to any one of claims 1 to 13.

15. Machine-readable data carrier and / or download product with the computer program according to claim 14.