Construction site inspection recording method and system

Abstract

The invention discloses a construction site inspection recording method and system. The present invention relates to a method and system for building inspection recording in which an inspection device moves along a trajectory while building, while capturing a plurality of inspection data sets. Thus, a first trajectory shape and / or time-related attribute is automatically extracted by evaluating the shape and / or timeline of the trajectory, and the extracted first attribute is automatically associated with a corresponding second building attribute provided by a stored building plan or model, the trajectory is automatically aligned and preferably also scaled using an architectural plan or model as a location reference based on the related first and second attributes. This enables the respective inspection data sets to be automatically assigned to locations within the architectural plan or model based on the aligned and scaled trajectories.

Description

Technical Field

[0001] The present invention relates to a method and system for generating inspection documentation of a construction site. Background Art

[0002] Building inspection systems and instruments that provide building or inspection documentation of a construction site are used to assess the actual condition of a building that has deteriorated after an impact such as an earthquake or due to aging, or to verify the accuracy or timeliness of a building, or to document aspects of the building process for subsequent reference in large or spatial buildings. Exemplary large buildings are buildings for which completion data is obtained (a building should be understood as a broad term including houses, factory buildings, streets, bridges or tunnels, etc.), ships or aircraft, etc. or industrial facilities, and the completion data can be compared with target building data arranged, for example, in a floor plan or model of the building (such as a two-dimensional or three-dimensional floor plan or CAD drawing).

[0003] For example, at a construction site, various tasks are performed simultaneously on different parts of a construction project, and it is difficult to track the progress of each aspect of the construction project to determine whether the project is on schedule for timely completion. It is necessary to monitor the progress at the construction site by capturing inspection data sets (such as images (including videos) of the construction site that record the condition of the construction site), especially by cyclically monitoring as the building gradually takes shape. Also, the interior of the construction site is regularly inspected, for example, the interior of a building is inspected by designers such as architects, glaziers, roofers or staircase architects, for example, for completion data capture.

[0004] For inspection, an inspection device is moved by a user or vehicle in a building (such as around or inside), for example, as a so-called walk-through, while collecting inspection data (such as 2D or 3D images of the building) along the movement trajectory, and this data provides the inspection documentation. For example, a user walks through a construction site with a 360° camera to generate a "virtual" visual tour as the building record of the building.

[0005] To associate the corresponding building inspection data set with the corresponding planned building data, the location of the data set needs to be obtained. For example, a user takes a large number of building pictures, and in order to be able to find the relevant pictures later, it is necessary to store the pictures together with their reference locations at the building or construction site.

[0006] Associations in the building floor plan coordinate system can be manually completed by the user through position registration (log). For example, the user manually annotates each inspection image of their corresponding position within the construction site as an example of the dataset. This is time-consuming and cumbersome. Additionally, the accuracy of the inspection location depends on the accuracy of the user input.

[0007] As an alternative, position or motion sensors on board the inspection device can be used to determine the corresponding inspection location, i.e., for example, automatic position registration based on GNSS sensor data (however, this is not feasible everywhere, especially indoors), image-based positioning (e.g., using the Simultaneous Localization And Mapping (SLAM) algorithm), or dead reckoning (e.g., using an Inertial Measurement Unit (IMU)). Such a position and / or orientation determination device can record the 2D or 3D trajectory of the inspection in a manner synchronized with the capture time of the inspection data, and can use timestamps to assign instances or sets of inspection data within the trajectory. However, the derived inspection location refers to the internal reference frame of the instrument or sensor and needs to be transformed to the external coordinate system of the building floor plan. Such a reference frame or coordinate transformation is usually done manually, for example, by manually associating at least two trajectory points to two corresponding positions of the building floor plan or model, which is still time-consuming and cumbersome and may lack accuracy. Summary of the Invention

[0008] Therefore, an object of the present invention is to provide an improved method and system for generating inspection records of a building.

[0009] Another object of the present invention is to provide a method and system for automatic position - truth generation of building inspection records.

[0010] The present invention relates to a method for using a model or floor plan of a building to generate inspection records of the building.

[0011] The method includes the following steps: moving an inspection device at a building such that a movement trajectory develops or grows, while continuously or cyclically / occasionally capturing a plurality of inspection data sets, such as images of at least a part of the building at positions along the trajectory. The movement trajectory is recorded or determined based on the movement data and / or position data of the inspection device collected while moving and, for example, using timestamps to assign the correspondingly captured data sets to points or positions or points or positions along the trajectory (specifically by sensors on board the inspection device).

[0012] The method further includes: automatically extracting first trajectory shape and / or time-related attributes by evaluating the shape and / or timeline of the trajectory. Additionally, automatically correlating the extracted first trajectory attributes with corresponding second building floor plan or model attributes provided by the stored building floor plan or model.

[0013] Based on the relevant first and second attributes, there is a step of using the building floor plan or model as a position reference to automatically align and optionally also scale the trajectory, which enables the automatic assignment of the corresponding inspection data set to positions or points within the building floor plan or model based on the aligned and optionally also scaled trajectory.

[0014] Optionally, the trajectory is automatically corrected, particularly during the alignment and scaling process, by adding, cutting, and / or shifting trajectory points and / or trajectory segments based on constraints derived from the building model or floor plan and / or from the trajectory. Such constraints are optionally linked to attributes.

[0015] In a further development of the method, the first attribute includes sections of the trajectory classified according to shape and / or motion classes during the attribute extraction process, or in other words, the first attribute is extracted according to predetermined different classes of trajectory shape sections / parts and / or trajectory motion sections / parts.

[0016] Optionally, the second attribute includes different classes of building sections, such as facilities or devices, which at least include space or room classes (particularly differentiated according to size and / or function) and / or the location and / or dimension classes of barriers or obstacles (particularly walls and / or fences) or door opening classes of the building or construction site.

[0017] As another option, the first attribute and / or the second attribute includes patterns of shape and / or motion patterns, patterns of construction site sections, and / or graphical representations. Such patterns are optionally derived from the association with the corresponding classes mentioned above. As another option, there is a weighting depending on the constraints of the first attribute classes and / or second attribute classes mentioned above.

[0018] In another further development, there is a weighting of the first attribute and / or trajectory points and / or sections depending on the associated timestamp. This weighting is particularly applied to attribute extraction, trajectory alignment, and / or scaling and / or trajectory correction.

[0019] For the said automatic alignment and / or scaling, optionally the motion data and / or position data of the inspection device are considered. As another option, the previous inspection record trajectories of the building are considered during alignment and / or scaling. Alternatively or additionally, at least one previous image of the construction site is considered for alignment and / or scaling. For example, one (or more) older record trajectories and / or images are compared with the most recent trajectory or most recent image of the building taken along the most recent trajectory.

[0020] Optionally, a first machine learning algorithm is used to extract first attributes from the trajectory, especially for class-related extraction, for example, the algorithm includes an algorithm of a first neural network. As another option, a second machine learning algorithm (especially including a second (especially deep learning) neural network) is used to extract second attributes from the building floor plan or model. A preferred example of the neural network is a deep learning neural network. A neural network (especially a graph neural network) can also be used to correlate the first attribute and the second attribute.

[0021] Optionally, the extraction of the first attribute is based on a 2D planar representation of the trajectory. During the process of this method, the 2D data of the trajectory can be derived from the 3D trajectory data. As another option, the building floor plan or model is specifically implemented as a 2D or 3D building and / or building floor plan, such as a floor plan and / or a Building Information Model (BIM) derived therefrom.

[0022] As another option, the inspection device includes an optoelectronic 2D and / or 3D imaging device, and the data set includes 2D images and / or 3D images. Therefore, the motion data and / or position data can include at least a part of such 2D images for visual odometry / SLAM.

[0023] In some embodiments, there is human supervisor verification of the automatic alignment and scaling, where a graphical representation of the aligned and scaled trajectory is generated and displayed and / or automatically generated or displayed as an overlay of the graphical representation of the construction site model, especially using a signed distance field. The verification is preferably performed before assigning the inspection data set.

[0024] The present invention also relates to a building inspection system for generating a building inspection record. The system includes a movable inspection device having motion and / or position detection means for recording the movement trajectory of the device and inspection means for capturing at least a part of the building at positions along the trajectory as an inspection data set. In addition, the system has an evaluation unit configured to assign the captured corresponding data set to the position of the trajectory, especially the assignment is based on a timestamp.

[0025] The evaluation unit is configured to extract first trajectory shape and / or time-related attributes by evaluating the shape and / or timeline of a trajectory, correlate the extracted first construction site attributes with corresponding second construction site model attributes provided by the stored building floor plan or model, use the building floor plan or model as a location reference to align and optionally also scale the trajectory based on the correlated first and second attributes, and assign corresponding inspection data sets to locations within the building floor plan or model based on the aligned and optionally also scaled trajectory.

[0026] The invention also relates to a computer program product comprising program code stored on a machine-readable medium or embodied by an electromagnetic wave comprising program code segments and having computer-executable instructions for performing the claimed method, in particular when executed on a computing unit of the claimed system.

[0027] The invention provides the advantage of performing construction documentation without user input regarding the positioning of inspection data sets such as images in a building floor plan or model, but rather by leveraging the geometric and / or time-related characteristics of inspection trajectories and corresponding or associable characteristics of the building being inspected, thereby making the construction documentation process faster and more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The devices, methods, systems, arrangements, and computer programs according to the invention will be described or explained in more detail below only by way of example with reference to working examples schematically illustrated in the drawings.

[0029] Specifically,

[0030] Figure 1a and Figure 1b an exemplary embodiment of an inspection system for inspecting a building known in the art is schematically illustrated;

[0031] Figure 2a and Figure 2b an example of the evaluation of an inspection trajectory for providing positioning in a building floor plan or model in a 2D top view is schematically illustrated;

[0032] Figure 3 an example of trajectory alignment or scale correction is schematically depicted; and

[0033] Figure 4 another example of trajectory alignment or scale correction is schematically depicted. DETAILED DESCRIPTION

[0034] Figure 1a and Figure 1bAn exemplary embodiment of an inspection system for inspecting a building 10 known in the art is schematically shown. Figure 1a is a 3D scheme, while Figure 1b is basically a 2D top view. The exemplary depicted building 10 is a building under construction, or more specifically a building floor. The interior of the building includes different building elements, such as walls 11, doors or door openings 12 or windows 13, and rooms or halls / corridors 14.

[0035] The inspection system includes an electronic camera device 20 having a display 21 as an exemplary inspection device for capturing images 23 of the building (specifically the construction site 10) as an example of an inspection data set (whereby a single image or multiple images, for example as part of a video, are understood as a data set). The inspection device 20 may also include, as alternative examples of inspection means, a depth camera (such as a time-of-flight camera), a stereo camera, a distance image camera, a laser scan or profile analysis unit, a structured light 3D scanner, etc.; and according to the example, the inspection data set is one or more spatial or 3D points, for example in the form of a point cloud or a 3D / depth image. In addition to the optoelectronic inspection means, other sensors or detectors feasible for building inspection may also be used, such as eddy current sensors or other non-destructive sensing devices.

[0036] In this example, the user 100 holds the imaging device 20 and walks through the building 10 from a starting point P', which in this example is also the end point of a gradually formed movement trajectory T of the user 100 (specifically the imaging device 20). In this example, the user 100 walks through all the rooms 14 of the floor and simultaneously records inspection data. For example, the user 100 collects the inspection data set by capturing one or more video streams while walking and taking still images at different points P (for example, using a 360°, panoramic or depth camera) in order to cover more specifically certain relevant building sections, installations or features, such as windows 13 or some rooms 14.

[0037] The system includes means, such as a position or motion sensor 22, for recording the trajectory T and assigning the inspection data set to the position of the acquired data set along the trajectory T (and generally also for checking the orientation or observation / sensing / measurement direction of the inspection device 20 when acquiring inspection data). The assignment of the data set can be based, for example, on time stamps (specifically, the time synchronization of the inspection device and the position determination device 22). For example, based on time stamps, any camera image or any inspection data can be located on the trajectory T.

[0038] As is known in the art, such movement and / or position recording (specifically, the (continuous) determination of the position of the inspection device 20) can be derived from data of a navigation system (such as a navigation satellite system (GNSS)) and / or local navigation by base stations.

[0039] If no navigation system is available, for example, for indoor inspections or at narrow or blocked outdoor construction sites, or as an additional or auxiliary position / motion sensing device, the inspection device 20 includes motion and / or position sensors 22, such as accelerometers, gyroscopes, or solid-state compasses. A preferred example is an inertial measurement unit (IMU), which provides measurements of acceleration and angular rate, and these accelerations and angular rates are integrated to derive velocity in a first step and ultimately the position and orientation of the device 20.

[0040] The derivation of position information can in particular include sensor fusion of a plurality of such units, which can be configured to derive position or navigation information, for example, also to overcome the situation where one or more of the plurality of sensors 22 are blocked or have insufficient data.

[0041] As another example of a position or motion sensing device, and in particular in the case of the imaging inspection device 20 as illustrated, visual odometry, structure from motion, simultaneous localization and mapping (SLAM), LIDAR simultaneous localization and mapping, or dense matching algorithms can be used, for example, to visually record the trajectory T.

[0042] Spatial reference information can also be derived by visual SLAM methods. Visual SLAM (or VSLAM) is basically the repeated application of resection and forward intersection, which has an optional bundle adjustment at the very end, and visual SLAM allows the capture of optical inspection data of the environment while tracking the position and orientation of the camera unit within it. The spatial reference information allows the derivation of the position information of the field of view of the camera image. Visual SLAM is evaluating the images from the camera device 20.

[0043] In visual SLAM, feature tracking is to detect object features / point features or so-called landmarks in the image stream and track their positions from one image frame to the next. Now, when the camera moves in a building (construction site), the detected features (e.g., the corners of the wall 11 or the door 12 or the window 13) will then move in the image 23. For example, when the camera rotates from left to right, the features on the image will then move right and left. If there is a system with several cameras, the features can also move from the field of view of one camera to the field of view of another camera during rotation. Therefore, based on the movement of the object features between the frames of the image stream or the appearance of the features in different camera fields of view, the movement direction of the camera in 3D space can be derived.

[0044] In a continuous process, the algorithm calculates the 3D coordinates (mapping) of the tracked feature from two or more positions and uses these coordinates to determine subsequent positions (localization). The generated map of landmarks gradually changes as the operator 100 (specifically the device 20) moves along the route or trajectory T and serves as a reference for the entire localization algorithm. The VSLAM can be further improved by adding data from the IMU. Such Visual Inertial Systems (VIS) (also known as Visual Inertial SLAM (VI-SLAM) or Visual Inertial Localization) combine visual and inertial ranging by fusing camera and IMU data and are typically filter-based or optimization-based. Thus, for example, the IMU data can be fused only to estimate the orientation and possibly the position change, rather than the full pose. As an alternative, the states of the camera and IMU can be fused into the motion and observation equations, followed by state estimation.

[0045] Therefore, regardless of the position (specifically motion) determination device used, the trajectory T describing the movement of the inspection device 20 can be recorded and the position of the inspection data set can be determined.

[0046] However, this trajectory T is only available in the sensor space or the internal reference frame and does not refer to the building floor plan or model, and the localization on the trajectory T has not been provided in the building floor plan or model. Otherwise, the inspection records with the as-built data must be linked to the reference data in the form of the building floor plan or model with the target data in a position-true manner. For example, in order to be able to find the relevant picture 23 as the inspection data set later, the picture 23 needs to be stored together with its position within the building or construction site.

[0047] In addition, since the position measurement is affected by measurement errors, the recorded trajectory T is prone to position errors. In particular, in the case of applying the IMU (specifically dead reckoning), the integration results in a significant drift of the derived quantity. The longer the trajectory 3, the greater the deviation between the recorded data and the "true" data may become.

[0048] Figure 2a and Figure 2b An example of the evaluation of the inspection trajectory T for providing localization in the building floor plan or model 15 is schematically shown, both in a 2D top view schematically.

[0049] The inspection trajectory T generated as described above is automatically analyzed by an evaluation unit of the inspection system (e.g., the computer unit of the inspection device or a connected or back-end computer). The purpose of the evaluation is to extract the attributes 1, 2, 2a, 2’, 3, 4, 4a, 4’ of the trajectory T (represented as the first attributes), and to provide the attributes 11a, 12a, 12a’, 13a, 14a, 14a’ of the building model 15 (represented as the second attributes), i.e., the prominent features of both the trajectory T and the building floor plan 15 that can be related to each other. Then, based on the correlation, the trajectory T is aligned and scaled with the floor plan or model 15 used as a reference, so as to assign the inspection data set to the building model 15. In this example, the trajectory attributes (or the first attributes) are extracted based on the shape or a part of the shape (shape segment) of the trajectory T. Such shape patterns 4, 4’ are typical for the inspection of rooms (along the corridor).

[0050] Rooms are examples of the models or second attributes 14a, 14a’ provided by the model 15. Such attributes can be stored together with the model 15 and thus have been given. Otherwise, some or all of the second attributes 11a, 12a, 12a’, 13a, 14a, 14a’ are extracted from the given model 15 by an evaluation algorithm.

[0051] In any case, the rooms (such as those represented as rooms 14a, 14a’) are recognized as model features. Since the said trajectory shapes 4, 4’ are considered to be caused by the inspection of the rooms, the first attributes 4, 4’ are related to the second attributes 14a, 14a’. The correlation is indicated in the figure by the wide long double arrows. In other words, such loop-shaped trajectory shape parts 4, 4’ are regarded as corresponding to the rooms 14a, 14a’, especially the rooms 14a, 14a’ having only one or more main entrances.

[0052] Therefore, it is also possible to determine and consider dimensions such as the diameter or length of the attributes to distinguish different sizes of attributes of the same type or class, e.g., to distinguish rooms 14a, 14a’ of different sizes and the corresponding shapes 4, 4’.

[0053] Examples of the second attributes further depicted are the door openings 12a, 12a’. Such transition sections of the building usually result in inspection movements with narrow trajectory sections, as indicated by the arrows 2 or 2’ in the figure. Also, as shown by the arrow 2a, the distance between adjacent trajectory sections or points can be considered. Therefore, such trajectory attributes 2, 2’, 2a can be related to the door openings 12a or 12a’. In addition to the pure shape, time-related information can also be attributed here by considering the moving direction, i.e., the anti-parallel moving direction as shown by the arrow 2.

[0054] Attributes can also be combined or aggregated or formed from multiple aspects of the trajectory T, or can be related to more than one "corresponding" attribute (first to second attribute correlation). For example, the correlation with the second attribute (class) "room" 14a, 14a' can be accomplished by evaluating together the first attributes of the "loop" 4 and the neck 2, for example, in the form of a first input and a second input of a correlation algorithm (such as a neural network), or classifying such a shape section 4 + 2a as a protrusion.

[0055] As already mentioned, time-related trajectory features can also be used as attributes or attribute components. For example, the moving direction and / or speed (pattern) as indicated by the arrow 1 can be used as a trajectory-side attribute. Since the inspection movement at the trajectory section 1 is relatively straight, this straight path is related to a path obstacle (in this case, the wall 11a). Other such second attributes 11a that constrain the inspection path and thus result in the typical trajectory section 1 can be fences or natural obstacles (such as rivers or trenches) that are also typically imaged in an outdoor building (construction site) floor plan.

[0056] In other words, an indoor built environment such as a building is typically divided into different rooms by walls. Attributes that can be applied in this case are attributes such as the position of doors and the size of rooms. In an outdoor environment, natural obstacles (such as rivers) and man-made structures (such as roads or fences) divide the space and create patterns in the camera trajectory T that can be aligned with the corresponding modeled attributes of the environment.

[0057] Stationary points 4a, 3 can also be used as attributes. For example, the point 4a can indicate the rooms 14a, 14a' (e.g., in the case of an imaging inspection, where a 360° image of the room is taken here), or the point 3 can indicate building details such as a turning edge or a window or a door opening 13a, or a specific facility such as an electrical facility or a pipe joint that requires more or specific inspection time, for example, when using an inspection device with an eddy current sensor. Thus, the first attribute "stationary point" 3 is related to the second attribute "facility (point)" in the example.

[0058] To detect the attributes or unique patterns of the trajectory T, neural networks such as Convolutional Neural Networks (CNN), Long Short-Term Memory networks (LSTM), or other Recurrent Neural Networks (RNN) can be employed. Such patterns are (but not limited to) corridors, small / large rooms, or stationary (captured) positions. The trajectory T can also be used as a time series in the network to detect patterns such as stationary positions.

[0059] The neural network classifies the segments of the trajectory T into different classes. Additionally, a graphical representation of the trajectory T can be created, linking the different classes. Independently, a similar graphical representation of the corresponding distinctive patterns can be extracted from the construction site model 15 using a different network. Then, a correspondence between the two graphs can be established using, for example, a graph neural network.

[0060] Based on the correspondence between the two graphs, one or more potential alignments (transformations) can be estimated. This is schematically indicated in the figure because Figure 2a a trajectory T and a trajectory reference frame 5 are shown with different orientations and scales compared to the building model system 6 before the transformation, where the trajectory T has not been referenced to the floor plan 15.

[0061] Figure 2b The result is shown when the established correspondence of the first and second attributes has been used to align and scale the trajectory T with the building model M. Schematically, this is shown as the trajectory T’ being fitted into the floor plan 15 after rotation and scaling (indicated by the arrow 7), and the corrected trajectory reference frame 5r now being equal to the building model system 6 in terms of orientation and scale.

[0062] In summary, the inspection trajectory T is automatically aligned with the construction site model 15 based on the correspondence between the trajectory attributes and the model attributes. The algorithm can be, for example, a machine learning algorithm, specifically, a deep learning algorithm. The input is the trajectory T in the sensor space and the construction site model (or models) 15, and the output is the trajectory T’ scaled and positioned on the construction site model 15, such as a floor plan. The algorithm uses the trajectory shape and / or time series features 1, 2, 2a, 2’, 3, 4, 4a, 4’ of the building (model) and the attributes 11a, 12a, 12a’, 13a, 14a, 14a’, such as building corridor, door, and room dimensions, to perform the alignment 7.

[0063] Therefore, the evaluation can utilize the fact that, to have consistency between different construction sites and different users and to ensure useful recorded data, typically users will follow a set of rules on how to move through the construction site for inspection. Examples of such rules are entering the room on the left - hand side of the corridor before entering the room on the right - hand side of the corridor. Another example is taking photos of all the walls in the room from an appropriate distance or standing in the middle of any room with a 360° camera to obtain an overview shot. Such parameters can be considered for shaping or learning the attributes or defining / learning the attribute classes, or as constraints for the trajectory alignment or scaling or the correction of such alignment or scaling.

[0064] Other examples of additional parameters or constraints are geometric cues (distance, scale), especially if the inspection data set includes such distance information (e.g., when the inspection device includes a time-of-flight sensor, LiDAR, or IMU, etc.), then the geometric cues can be used for alignment 7. Another possibility is that in addition to using the trajectory shape, the recorded images can also be used to locate the trajectory T in the building model 15. The image correspondence between the current trajectory T and the previously recorded data can provide additional hints or constraints for (final) alignment or scaling or alignment / scaling refinement or correction.

[0065] Figure 3 Examples of trajectory alignment / scale correction or verification are shown.

[0066] On the left side of the figure, shown is a trajectory T' aligned and scaled with the building floor plan 15 based on attribute correlation as described in Figure 2a , Figure 2b . The resulting fit of the trajectory T' is verified with respect to one or more minimum test criteria that must be met or adhered to (e.g., given the constraints of the building model 15 reflecting impossible or unlikely trajectory positions).

[0067] In this example, it is tested whether the aligned and scaled trajectory T' avoids path obstacles. In other words, the trajectory T' should not leave streaks or touch any path obstacles, such as the wall 11a (or should have a minimum distance to such model attributes). As depicted, this is the case at the trajectory position 8 where the trajectory T' crosses or intrudes into the wall section 11a. Thus, obstacles can be provided as an attribute class (and thus the applied constraints are linked, for example, to the "wall" class).

[0068] Such verification can be applied because the aligned trajectory T' is placed in the building model 15 and controlled using, for example, a signed distance field, thereby avoiding, for example, the crossing of the trajectory segment and the wall 11a in the floor plan 15.

[0069] Then, in order to remedy defects such as obstacle crossing, a (further) shift, rotation, or scaling of the trajectory T' is performed, indicated by the arrow 7c in the figure, such that the result of the corrected trajectory T' is to have a minimum distance to all obstacle attributes 11a at each trajectory point or segment.

[0070] This kind of trajectory correction, verification of position criteria, or consideration of constraints does not need to be a separate or subsequent step after the first alignment and scaling, but can also be an integral part of the alignment and scaling process based on "all" attributes.

[0071] In particular, if multiple constraints of different types are considered, they can be weighted. For example, the weighting can depend on the association of the constraint with a class or property. For instance, a constraint referring to an attribute class is given more weight than another constraint connected to another attribute class. Also, a weighting depending on time or "ranking" in the trajectory sequence can be applied. For example, a constraint that is given less weight or has a smaller correction impact at an "earlier" trajectory segment compared to a "later" trajectory segment, as further illustrated below.

[0072] Figure 4 Another example of trajectory alignment / scaling correction or verification is shown.

[0073] In an example like Figure 3 the previous example, a part of the aligned trajectory T' hits or crosses a wall segment again at position 8, in this example, especially in the trajectory region towards the end of the trajectory T'. Additionally, the end point T of the trajectory P is far from the starting point P', and in this example, the starting point P' does not reflect the actual traveled path that is more or less a closed loop as a trajectory constraint.

[0074] Such trajectory defects may be caused by cumulative errors in position or motion sensing, which is the case, for example, when using an IMU (specifically, determining position based on the principle of dead reckoning).

[0075] To correct such time-related trajectory defects, for example, to compensate for the drift of the visually measured range or the trajectory T' based on SLM, the shape or route of the trajectory T' can also be adjusted during the alignment process.

[0076] In this example, not only the trajectory segment 8 that hits the wall is corrected, but also the end point T of the end of the trajectory T' P and the position of the connection point. For example, the end point T P along with additional trajectory segments therewith are shifted to overlap with the starting point P'.

[0077] Since such end points T P are considered less reliable compared to earlier trajectories or checkpoints, they are repositioned to a greater extent than the earlier ones. Thus, time is considered in the editing of the trajectory because time-related weighting is applied, where later determined positions are given less weight and are more easily shifted / corrected.

[0078] Also, for example, a graph-based optimization can be formulated, where the constraints from the trajectory shape are appropriately weighted relative to the trajectory pattern constraints.

[0079] Time-related weighting using, for example, timestamps can be applied not only to trajectory points and / or segments, and not only in the correction step, but also in attribute extraction. As another example, the first attribute can also be weighted, for example because an "after" attribute (an attribute associated with a later time point or a posterior part of the trajectory) is weighted less for alignment or scaling.

[0080] Those skilled in the art are aware of the fact that the details shown and explained here for different embodiments can also be combined with details from other embodiments and other arrangements within the meaning of the present invention.

Claims

1. A method for generating an inspection record of a building, the method comprising: moving an inspection device at the structure so that a trajectory is formed while capturing a plurality of inspection data sets of at least a portion of the structure along the trajectory, Automatically record the trajectory of the movement based on the motion data and / or position data of the inspection device collected onboard / by the sensors of the inspection device during the movement; as well as In particular, the corresponding data set captured is assigned to the position of the trajectory based on the timestamp, Features automatically extracting first trajectory shape and / or time-related properties by evaluating the shape and / or timeline of said trajectory, automatically correlating the extracted first attributes with corresponding second building attributes provided by a stored building plan or model, automatically aligning and in particular scaling the trajectory using the building plan or model as a position reference based on the correlated first and second attributes, This enables an automatic assignment of the corresponding inspection data sets to locations within the building plan or model based on the aligned and, in particular, scaled trajectories.

2. The method according to claim 1, Features The trajectory is automatically corrected by adding, cutting and / or shifting trajectory points and / or segments based on constraints derived from the trajectory and / or a building model or plan.

3. The method according to claim 2, Features Constraints are linked to attributes.

4. The method according to any one of the preceding claims, Features The first attribute is extracted based on predetermined different classes of trajectory shape segments and / or trajectory motion segments.

5. The method according to any one of the preceding claims, Features The second attribute includes different types of building segments, including at least one of the following: Space class, The location and / or size of the obstruction, and / or Doorway type.

6. The method according to any one of the preceding claims, Features Attributes include Patterns of shapes and / or movement, Pattern of construction site sections, and / or Graphical representation.

7. The method according to claim 4, claim 5 and claim 6, Features The pattern is derived from the association with the corresponding class.

8. The method according to claims 3 and 4 or claims 3 and 5, Features Constraints are weighted based on attribute classes.

9. The method according to any one of the preceding claims, Features The first attributes and / or trajectory points and / or segments are weighted depending on the associated time stamp, in particular for attribute extraction, trajectory alignment and / or scaling and / or trajectory correction.

10. The method according to any one of the preceding claims, Features For the automatic alignment and / or scaling, consider the following: Movement data and / or position data of the inspection device, and / or A record of previous inspections of the building in question, and / or At least one previous image of the building from a previous inspection record.

11. The method according to any one of the preceding claims, Features The first attribute is extracted using a first machine learning algorithm, in particular comprising a first neural network, and the second attribute is extracted from the building plan or model using a second machine learning algorithm, in particular comprising a second neural network.

12. The method according to any one of the preceding claims, Features A first attribute is extracted based on the 2D planar representation of the trajectory.

13. The method according to any one of the preceding claims, Features The examination device comprises an optoelectronic 2D and / or 3D imaging device and the data set comprises a 2D and / or 3D image, in particular whereby the movement data and / or position data comprise at least a part of the image.

14. A building inspection system for generating a building inspection record, the building inspection system comprising A movable inspection device having a motion and / or position detection device for recording the movement trajectory of the device, and an inspection device for capturing an inspection data set of at least a portion of the building along the trajectory, and an evaluation unit configured to assign the captured respective data set to a position of the trajectory, in particular based on a time stamp, Features The evaluation unit is configured to extracting first trajectory shape and / or time-related properties by evaluating the shape and / or timeline of said trajectory, associating the extracted first construction site attributes with corresponding second construction site model attributes provided by a stored building plan or model, aligning and in particular scaling the trajectory using the building plan or model as a position reference based on the associated first and second attributes, Based on the aligned and, in particular, scaled trajectories, corresponding inspection data sets are assigned to positions within the building plan or model.

15. A computer program product comprising a program code stored on a machine-readable medium or embodied by an electromagnetic wave comprising program code segments and having computer executable instructions for performing the method according to claim 1, in particular when executed on an evaluation unit of a system according to claim 14.

Images