Monitoring device for detecting objects within infrastructure

Abstract

The present invention relates to a monitoring device for detecting an object within an infrastructure, the monitoring device comprising: a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes and to generate light detection and ranging (LIDAR) data for generating a three-dimensional model of the infrastructure based on the scanning movement; a camera unit configured to capture image data along at least two different imaging directions relative to the monitoring device; and an object detector configured to detect an object within the monitored infrastructure based on at least one of the LIDAR data and the image data, the monitoring device configured to generate a three-dimensional model of the infrastructure by taking into account the LIDAR data, and the object detector configured to detect the object based on the three-dimensional model, in particular wherein the LIDAR data and the image data are different from each other. The object detector is configured to run a feature recognition algorithm for the three-dimensional model and to recognize semantic features and / or geometric features based on the feature recognition algorithm.

Description

[0001] This application is a divisional application of the invention patent application with the original application number 201980083014.5 (International application number: PCT / EP2019 / 065045, international application date; June 7, 2019, invention name: Reality capture using laser scanner and camera). Technical Field

[0002] The present invention relates to a reality capture device for generating a digital three-dimensional representation of an environment, in particular for surveying and / or detecting objects within an infrastructure. Background Art

[0003] For example, the measurement of buildings and the surrounding terrain is of interest to architects or craftsmen in order to quickly assess the actual condition of a room or the progress of construction at a building site, respectively, in order to plan the next work steps efficiently, for example. With the help of a digital visualization of the actual status (e.g. in the form of a point cloud or vector file model) or with the help of augmented reality functions, different options for further steps or expansion options can be checked and optionally presented to employees or customers in an easily accessible manner.

[0004] Another example is the use of 3D surveillance systems to monitor neural nodes within a city, such as train stations, airports, city parks or other busy public places, or to protect restricted or dangerous areas, such as industrial plants, construction sites or commercial complexes. Moreover, it can support the operation of facilities, such as supervising warehouses or parking lots.

[0005] The environment can be optically scanned and measured by means of a laser scanner emitting a laser measurement beam, for example using pulsed electromagnetic radiation, wherein echoes of backscattered surface points from the environment are received and the distance to the surface point is derived and associated with the angular emission direction of the associated laser measurement beam. In this way, a three-dimensional point cloud is generated. For example, the distance measurement can be based on the flight time, shape and / or phase of the pulse.

[0006] For additional information, the laser scanner data can be combined with the camera data, for example by means of an RGB camera or an infrared camera, in particular in order to provide high-resolution spectral information.

[0007] Typically, the distance measurement modules used when surveying laser scanners are able to detect intensity changes, but do not have color sensitivity, which is why 3D models (especially point clouds or vector file models) generated purely by laser scanners can only be displayed in grayscale. As a result, many details remain hidden to the human observer due to the lack of color effects and the lack of depth effects supported by color. Therefore, a "colorized" 3D point cloud is usually generated by referencing the "gray" 3D point cloud of the laser scanner with the color data from the camera, which makes the display easier for the human eye.

[0008] The referencing and fusion of different data types (eg, laser scanner data, camera data, and positioning data from global navigation satellite systems) is now increasingly standardized.

[0009] In particular, the reality capture device may be mobile and configured to provide measurement data and reference data simultaneously, e.g. wherein at least trajectory data (e.g. position and / or pose data) of the device are provided with detection data (e.g. laser scanner data and / or camera data) such that detection data of different positions of the reality capture device can be combined into a common coordinate system. Typically, the reality capture device is configured to autonomously create a 3D map of a new environment, e.g. by means of a simultaneous localization and mapping (SLAM) function.

[0010] The three-dimensional model data can then be analyzed by means of feature recognition algorithms for automatically identifying semantic features and / or geometric features captured by the probe data, for example by using shape information provided by the virtual object data from the CAD model. Such feature recognition (particularly for identifying geometric primitives) is nowadays widely used for analyzing 3D data.

[0011] In prior art surveillance systems, laser rangefinders and / or light barriers provide intrusion and / or motion detection, wherein a moving object is detected when it crosses an observation plane within the infrastructure. However, compared to camera imaging, the spatial coverage of laser scanners and / or light barriers is often limited, e.g., where different observation planes are placed only at entrances (e.g., at doors and windows), or where different observation planes are at least a few meters apart.

[0012] A particular problem with prior art monitoring devices relates to handling changing environmental conditions, such as changes in ambient light, and / or adapting to changes in the environment, wherein the arrangement of allowed objects has to be distinguished from the arrangement of unallowed objects to be monitored within the infrastructure.

[0013] A special task of monitoring involves detecting objects left behind in the surveillance area, such as packages containing explosives or other hazardous materials. However, in prior art survey systems, automatic object recognition and tracking is often slowed down in the case of crowded areas (i.e., areas with many different kinds of objects), where all of these objects may be moving in different directions, resulting in frequent intersections and obstructed lines of sight. Such a situation exists, for example, in train stations or subway stations. Summary of the invention

[0014] It is an object of the present invention to provide an improved reality capture device that is more compact and easier to use, thereby allowing faster and more reliable capture of various environments.

[0015] One aspect of the invention relates to a mobile reality capture device configured to be carried and moved by a mobile carrier, in particular a person or a robot or a vehicle, and to move during a measurement process for generating a digital representation of an environment.

[0016] The mobile reality capture device has a positioning unit, which in particular comprises an inertial measurement unit (IMU), wherein the positioning unit is configured to generate positioning data for determining a trajectory of the mobile reality capture device.

[0017] Moreover, the reality capture device has a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud.

[0018] In addition, the mobile reality capture device has: a base, which supports the laser scanner; and a cover, in particular a cover that is opaque to visible light, which is mounted on the base so that the cover and the base surround all moving parts of the laser scanner so that the moving parts are not accessible from the outside.

[0019] In one embodiment, the positioning unit has an inertial measurement unit (IMU) that generates inertial data of the mobile reality capture device, the IMU comprising two inertial sensors, wherein one of the inertial sensors is mounted on a portion of the laser scanner that rotates during the measurement process, and the other of the inertial sensors is mounted on a portion of the laser scanner that is stationary relative to the base during the measurement process. In particular, the positioning unit is configured to determine a drift of the inertial data of the mobile reality capture device by comparing the data of the two inertial sensors taking into account a rotation parameter describing a relative rotation between the two inertial sensors.

[0020] In another embodiment, the localization unit is configured such that the localization data is based on at least a portion of LIDAR data, in particular wherein the mobile reality capture device is configured to execute a LIDAR based localization and mapping algorithm.

[0021] In another embodiment, the mobile reality capture device comprises a panoramic camera unit, which is arranged on a side surface of the mobile reality capture device, the side surface defining a vertical axis of the mobile reality capture device, i.e., wherein the side surface is arranged circumferentially around the vertical axis. The panoramic camera unit is configured to provide image data covering a field of view of at least 120°, in particular at least 180°, more in particular 360° around the vertical axis.

[0022] In particular, the panoramic camera unit comprises a plurality of cameras which are arranged circumferentially on a side surface, and the mobile reality capture device is configured to generate a panoramic image based on the image data, i.e., wherein individual images of the plurality of cameras are stitched together to form an image having a wider field of view than the individual images.

[0023] In another embodiment, the mobile reality capture device comprises a positioning camera, which is used by the positioning unit, in particular wherein the positioning camera is part of a panoramic camera unit. The positioning unit is configured such that the positioning data is based on image data generated by the positioning camera, in particular wherein the mobile reality capture device is configured to perform visual positioning and mapping algorithms.

[0024] In particular, the mobile reality capture device includes a plurality of positioning cameras, which are used by the positioning unit, and in particular, the plurality of positioning cameras are configured and arranged for a nominal minimum operating range of the positioning unit, and each of the plurality of positioning cameras has a field of view that overlaps with at least another positioning camera of the plurality of positioning cameras.

[0025] In another embodiment, the mobile reality capture device includes a color camera, in particular as part of a panoramic camera unit, which is configured to capture color images, in particular wherein the mobile reality capture device is configured to provide point cloud data for generating a colorized three-dimensional point cloud based on LIDA data and the color image.

[0026] In another embodiment, the mobile reality capture device comprises a high dynamic range (HDR) camera, in particular as part of a panoramic camera unit, in particular a single exposure HDR camera.

[0027] In another embodiment, the mobile reality capture device comprises a multispectral camera, in particular a hyperspectral camera, in particular as part of a panoramic camera unit.

[0028] In another embodiment, the mobile reality capture device includes a Global Navigation Satellite System (GNSS) transceiver.

[0029] In another embodiment, the mobile reality capture device includes a direction determining unit (eg, a compass) for providing directions relative to geographic cardinal directions.

[0030] In another embodiment, the laser scanner is configured to rotate the two rotation axes faster than 0.1 Hz, in particular faster than 1 Hz, in order to generate LIDAR data, wherein the LIDAR data is generated with a point acquisition rate of at least 300,000 points per second, in particular at least 500,000 points per second.

[0031] In particular, the laser scanner may be configured to generate LIDAR data by adaptively setting at least two different rotation rates of at least one of the two rotation axes and / or by adaptively setting at least two different point acquisition rates.

[0032] In another embodiment, the cover provides a field of view of the laser scanner that is larger than half of a unit sphere centered at the laser scanner. For example, the cover has a hemispherical tip portion that merges into the cylindrical shell in the direction of the base, particularly wherein the laser scanner is configured to generate LIDAR data based on the following orientations: the orientation of the laser measurement beam passing through the hemispherical tip portion; and the orientation of the laser measurement beam passing through the cylindrical shell.

[0033] In another embodiment, the cover is made of a material comprising plastic, wherein the cover has an atomic layer deposition (ALD) coating on the outside and on the inside, in particular wherein the ALD coating on the outside and / or on the inside is covered by a hard coating.

[0034] In another embodiment, the cover has an anti-reflection (AR) coating on the inside and / or on the outside, in particular wherein the cover has areas on the inside and / or on the outside where no AR coating is present. For example, the AR coating can be applied on a circumferential band, in particular on the inside, which covers a limited height range.

[0035] In another embodiment, the cover has a hemispherical head portion, wherein the hemispherical head portion comprises a flat region having a flat surface both on the outside and on the inside, in particular wherein the flat region is arranged at the highest point.

[0036] In another embodiment, the flat area is specifically foreseen for mounting additional sensors, in particular a Global Navigation Satellite System (GNSS) transceiver, or wherein the flat area is specifically foreseen for providing a zenith LIDAR measurement of a laser scanner.

[0037] In another embodiment, the positioning unit is configured to determine the trajectory in six degrees of freedom, i.e., the position and orientation of the mobile reality capture device. In particular, the mobile reality capture device is configured to generate a three-dimensional map by performing simultaneous localization and mapping (SLAM) involving at least one of the following items: data of the IMU, image data of the camera unit for visual simultaneous localization and mapping (VSLAM); and LIDAR data for LIDAR-based simultaneous localization and mapping (LIDAR-SLAM).

[0038] In another embodiment, a laser scanner of a mobile reality capture device includes: a support portion, which is mounted on a base and can rotate relative to the base; and a rotating body, which deflects an output laser measurement beam and a return portion of the laser measurement beam, and the rotating body is mounted on the support portion and can rotate relative to the support portion. In this arrangement, the generation of LIDAR data includes: continuous rotation of the support portion relative to the base and continuous rotation of the rotating body relative to the support portion; and emission of the laser measurement beam via the continuously rotating rotating body, and detection of the portion of the laser measurement beam returned via the rotating body.

[0039] In particular, the laser scanner is configured such that the continuous rotation of the rotating body relative to the support is faster than the continuous rotation of the support relative to the base, for example, the continuous rotation of the support is at least 1 Hz and the continuous rotation of the rotating body is at least 50 Hz, in particular, two different rotation rates can be set for at least one of the rotation of the rotating body and the rotation of the support.

[0040] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier and to move during a measurement process for generating a digital representation of an environment. The mobile reality capture device has a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device. Moreover, the device has a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud.

[0041] According to this aspect, the laser scanner comprises a deflection component which deflects the return portion of the laser measurement beam, and the deflection component is configured to rotate about one of the two rotation axes and has a reflective curved surface, for example a parabolic shape, which serves as a collecting optical device for the return portion of the laser measurement beam.

[0042] In one embodiment, the deflection member is configured to deflect the output laser measurement beam and the return portion of the laser measurement beam, wherein the laser scanner includes a base and a support, the support is mounted on the base and can rotate relative to the base; and the deflection member is mounted on the support and can rotate relative to the support. Here, the generation of LIDAR data includes: continuous rotation of the support relative to the base; and continuous rotation of the deflection member relative to the support; and emission of the laser measurement beam via the continuously rotating deflection member; and detection of the portion of the laser measurement beam returned via the deflection member.

[0043] In another embodiment, the laser scanner has no beam forming optics in the receiving path between the deflection element and the receiver of the laser scanner.

[0044] In a further embodiment, the receiving path between the deflection element and the receiver of the laser scanner comprises a folding mirror, in particular arranged in the support, wherein the folding mirror is movable for adjusting the alignment of the optical axis of the receiving path to the receiver.

[0045] In another embodiment, the laser scanner has a receiver plate, in particular arranged in a support part, which receiver plate includes a photosensitive detection surface of the receiver, wherein the receiver plate is movable for adjusting the alignment of the detection surface along two orthogonal directions transverse to the optical axis of the receiving path.

[0046] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier and to move during a measurement process of generating a digital representation of an environment. The device has a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device. Moreover, the device includes a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud.

[0047] The laser scanner further comprises a rotating body configured to rotate about one of the two rotation axes for variably deflecting the output laser measurement beam and a return portion of the laser measurement beam, wherein the rotating body has a reflective surface that reflects the return portion of the laser measurement beam towards a detector of the laser scanner.

[0048] According to this aspect of the invention, the rotating body has a channel area, which is arranged in the reflecting surface and is configured to allow radiation from the interior of the rotating body to pass through, so that the output laser measurement beam from the interior of the rotating body (i.e. from the side facing away from the reflecting surface) can be sent into the environment through the channel area.

[0049] In one embodiment, the rotating body comprises a deflection assembly, in particular a prism or a reflector, which is configured to rotate together with the rotating body and deflect the output laser measurement beam from the interior of the rotating body. For example, the deflection assembly is embodied by a reflective prism, in particular a polygonal or circular cylindrical prism, more particularly a cuboid prism, that passes through the rotating body.

[0050] In another embodiment, the rotating body has a socket for the deflection assembly, wherein the socket is formed along the rotation axis of the rotating body, i.e. along the one of the two rotation axes about which the rotating body is foreseen to rotate for the deflection of the outgoing laser measurement beam and for the deflection of the return portion of the laser measurement beam. The socket is configured to provide a coaxial alignment of the rotation axis of the rotating body with the optical transmission axis inside the rotating body, for example wherein the socket has a mirror axis coaxial with the rotation axis of the rotating body.

[0051] In another embodiment, the socket has a first contact surface and a second contact surface, wherein a recess is formed between the first contact surface and the second contact surface, and the first contact surface and the second contact surface are located in a first stabilization plane having an extension along the rotation axis of the rotating body. Furthermore, the socket has a third contact surface, wherein the third contact surface is located in a second stabilization plane having an extension along the rotation axis of the rotating body. The first stabilization plane and the second stabilization plane intersect in particular at a right angle, wherein the first contact surface and the third contact surface adjoin each other and form a corner.

[0052] In another embodiment, the deflection assembly has a first side and a second side, in particular flat sides that are adjacent to each other, wherein the deflection assembly has a facet. The deflection assembly is further configured to allow the deflection assembly to be inserted into a socket so that the first side is in contact with a first contact surface and a second contact surface of the socket and the second side is in contact with a third contact surface of the socket, wherein the facet has no corners. When inserted, a force is applied to the deflection assembly such that the forces on the first contact surface, the second contact surface and the third contact surface are equal.

[0053] In another embodiment, the socket is configured to accommodate a cylindrical prism, for which purpose the socket comprises a circular pressing area and a reaction mechanism, in particular comprising two bolt holes, for rotating and fixing the accommodated prism.

[0054] In another embodiment, the laser scanner includes a base and a support, wherein the support is mounted on the base and can rotate relative to the base; and the rotating body is mounted on the support and can rotate relative to the support, wherein the generation of the LIDAR data includes: continuous rotation of the support relative to the base and continuous rotation of the rotating body relative to the support; and emission of a laser measurement beam through a channel area of ​​the rotating body, and detection of a portion of the laser measurement beam that returns through a reflective surface of the rotating body.

[0055] In particular, the support has two opposing supports along the rotation axis of the rotating body, i.e. along the one of the two rotation axes about which the rotating body is foreseen to rotate for a variable deflection of the outgoing laser measurement beam and of the return portion of the laser measurement beam. The rotating body is arranged in the support so that a first of the two supports is arranged on the side facing the reflection surface and a second of the two supports is arranged on the side facing away from the reflection surface. Moreover, the first support has an inlet embodied around the rotation axis of the rotating body, the inlet in particular comprising an optical window or a beam forming optic. The rotating body is mounted on the second support and is configured to deflect the outgoing laser measurement beam coming from the interior of the rotating body and passing through the passage area away from the rotation axis of the rotating body, in particular in a direction perpendicular to the rotation axis of the rotating body, and to deflect radiation of the laser measurement beam returning along the pointing direction of the outgoing laser measurement beam by the reflection surface towards the inlet of the first support.

[0056] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device, the mobile reality capture device being configured to be carried and moved by a mobile carrier and to move during a measurement process of generating a digital representation of an environment. The device has a positioning unit, the positioning unit particularly comprising an inertial measurement unit (IMU), the positioning unit being configured to generate positioning data for determining a trajectory of the mobile reality capture device.

[0057] Moreover, the mobile reality capture device has: a laser scanner configured to perform a scanning movement of a laser measurement beam during the movement of the mobile reality capture device, and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement; and a camera unit configured to capture image data along at least two different imaging directions relative to the mobile reality capture device.

[0058] According to this aspect of the invention, the mobile reality capture device has a plurality of light indicators, wherein the mobile reality capture device is configured to determine detection data based on at least one of positioning data, LIDAR data, and image data, and each of the light indicators is assigned to a scanning segment fixed relative to the mobile reality capture device, and each of the light indicators provides an indication of a quality parameter of the detection data related to the assigned scanning segment of the light indicator, in particular substantially in real time.

[0059] In one embodiment, the mobile reality capture device is configured to determine a spatial resolution parameter (in particular LIDAR point density or LIDAR point distribution) provided by the assigned detection data as a quality parameter for each scan segment.

[0060] In another embodiment, the mobile reality capture device is configured to run a feature recognition algorithm on the detection data, and to identify semantic features and / or geometric features in the detection data based on the feature recognition algorithm, wherein the indication of the quality parameter is based on the identified semantic features and / or geometric features.

[0061] In another embodiment, the plurality of light pointers are circumferentially arranged on a side surface of the mobile reality capture device such that an arrangement of each of the plurality of light pointers on the side surface corresponds to an assigned scanning segment of the light pointer.

[0062] For example, the plurality of light indicators are embodied by a single lamp, in particular an LED, or a light-guiding ring having a plurality of individually controllable segments.

[0063] In another embodiment, the mobile reality capture device is configured such that the indication of the quality parameter comprises at least an indication that the quality parameter meets a defined quality standard (particularly a threshold), and / or an indication that the quality parameter fails to meet the defined quality standard.

[0064] In another embodiment, the mobile reality capture device is configured such that the indication of the quality parameter is provided by color coding and / or flickering coding, which color coding and / or flickering coding particularly comprises a series of clearly separated single pulses and / or intensity pulses, i.e., a continuously decaying and enhancing pulse signal.

[0065] In another embodiment, the mobile reality capture device is configured to execute a visual positioning algorithm, and for this purpose, the mobile reality capture device includes a positioning camera configured to generate image data. Moreover, the mobile reality capture device is configured so that the activation of the plurality of light indicators is coordinated with the image capture rate of the positioning camera, in particular so that the plurality of light indicators are only activated when the shutter of the positioning camera is closed.

[0066] In another embodiment, the mobile reality capture device is configured such that the plurality of light indicators also provide color coding and / or flashing coding, which color coding and / or flashing coding particularly comprises a series of clearly separated single pulses and / or intensity pulsations, i.e., a continuously decaying and enhanced pulse signal.

[0067] For example, the color coding and / or flashing coding indicates at least one of the following operating conditions of the mobile reality capture device: device ready, positioning initialization in progress, positioning initialization completed, positioning reinitialization required, device moving too fast and / or too slow, images captured by the camera unit, low battery, and lost from the companion device.

[0068] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier and to move during a measurement process for generating a digital representation of an environment. The device has a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device. Moreover, the device has a detection unit, which includes at least a laser scanner configured to generate light detection and ranging (LIDAR) data and in particular a camera, wherein the detection unit is configured to generate detection data for determining a digital model of the environment during the movement of the mobile reality capture device.

[0069] In addition, the mobile reality capture device has a guidance unit configured to provide guidance from a current position of the mobile reality capture device towards an area of ​​the environment, and the mobile reality capture device is configured to determine detection data based on at least one of the positioning data and the LIDAR data.

[0070] According to this aspect of the invention, the mobile reality capture device is configured to generate guidance data based on analysis of the detection data and / or comparison with model data of the environment, the guidance data providing the location of an area of ​​the environment requiring additional detection data. The device is then configured to enable a guidance unit to provide guidance from the current location of the mobile reality capture device to the area of ​​the environment requiring additional detection data.

[0071] In one embodiment, areas of the environment requiring additional probe data are determined by analyzing the probe data relative to defined quality criteria for determining areas having insufficient data quality.

[0072] In another embodiment, the quality criterion is based on a spatial resolution parameter, in particular LDAR point density or LIDAR point distribution; and / or an intensity threshold parameter of the LIDAR data.

[0073] In another embodiment, the model data is based on previously acquired detection data and / or a predefined environment model, particularly a building information model (BIM) in the case where the environment is a building. For example, a comparison with the model data provides at least an indication of areas where detection data is missing.

[0074] In another embodiment, the mobile reality capture device is configured to run a feature recognition algorithm on the detection data, and identify semantic features and / or geometric features in the detection data based on the feature recognition algorithm, and generate guidance data based on the identified semantic features and / or geometric features.

[0075] In another embodiment, the detection unit is configured to include image data as part of the detection data, wherein the quality criterion provides an identification of areas having at least one of the following items: missing image data, image data with insufficient resolution, image data with insufficient image clarity, image data with smeared images, image data with insufficient intensity, and image data with too high intensity, in particular saturated images.

[0076] In another embodiment, the guidance unit is configured to provide an acoustic guidance signal that guides the operator with respect to the orientation of the mobile reality capture device.

[0077] In another embodiment, the guidance unit includes a plurality of light indicators configured to provide a visual guidance signal that guides the operator relative to the orientation of the mobile reality capture device. For example, the plurality of light indicators are arranged circumferentially on a side surface of the mobile reality capture device, such that the arrangement of each of the light indicators on the side surface corresponds to the direction of movement to be indicated.

[0078] In another embodiment, the mobile reality capture device is configured to provide visual encoding and / or acoustic encoding, for example, to provide at least one of: an expected turning speed, an indication of the amount of a corner, an indication of the remaining distance to an area in the environment for which insufficient detection data is available, and an expected speed for moving the mobile reality capture device.

[0079] In another embodiment, the mobile reality capture device is configured to initiate generation of guidance data based on user input.

[0080] In another embodiment, the mobile reality capture device is configured to automatically initialize the generation of guidance data based on positioning parameters indicating the position and / or movement direction of the mobile reality capture device, for example in combination with characteristic parameters derived from the detection data, wherein the characteristic parameters indicate the position of an exit (in particular a door) of the current measurement perimeter.

[0081] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device configured to be carried and moved by a mobile carrier (particularly a person or a robot or a vehicle) and to move during a measurement process to generate a digital representation of an environment.

[0082] The device has: a positioning unit, which particularly includes an inertial measurement unit (IMU), and is configured to generate positioning data for determining the trajectory of a mobile reality capture device; a laser scanner, which is configured to perform a scanning movement of a laser measurement beam during the movement of the mobile reality capture device, and generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement; and a status indicator, which is used to visually indicate the working status of the mobile reality capture device.

[0083] According to this aspect of the invention, the status indicator is circumferentially arranged on a side surface of the mobile reality capture device, which defines a vertical axis of the mobile reality capture device, i.e., wherein the side surface is circumferentially arranged around the vertical axis, and the status indicator is configured to appear the same for all viewing directions around the vertical axis, i.e., regardless of the rotational position of the mobile reality capture device around the vertical axis, the same information provided by the status indicator is visible when the mobile reality capture device is observed in a plane perpendicular to the vertical axis.

[0084] In one embodiment, the positioning unit is configured to execute a visual positioning algorithm, and for this purpose, the mobile reality capture device includes a positioning camera configured to generate image data, and the mobile reality capture device is configured so that the activation of the status indicator is coordinated with the image capture rate of the positioning camera, in particular so that the status indicator only illuminates when the shutter of the positioning camera is closed.

[0085] In another embodiment, the status indicator is embodied by a single light, in particular an LED, which is arranged circumferentially around the vertical axis, in particular wherein the single light is located in a same plane orthogonal to the vertical axis.

[0086] In another embodiment, the status indicator is embodied as a light-guiding ring that is circumferentially arranged around the vertical axis.

[0087] In another embodiment, the status indicator is implemented by means of an optical fiber ring having at least one optical coupling, in particular a plurality of couplings, more particularly six couplings, wherein the status indicator is configured such that the ratio of the radiation emitted along a radiation direction relative to the vertical axis to the radiation emitted along the optical fiber ring increases as the distance from the coupling position along the optical fiber ring increases.

[0088] In another embodiment, the status indicator is configured to provide the working status of the mobile reality capture device by means of color coding and / or flashing coding, wherein the color coding and / or flashing coding particularly includes a series of clearly separated single pulses and / or intensity pulsations, i.e., a continuously decaying and enhanced pulse signal, more particularly wherein the coding is provided by means of a rotating coding running around a vertical axis.

[0089] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a laser scanner configured to generate light detection and ranging (LIDAR) data based on a laser measurement beam, wherein the laser scanner has: a base; a support mounted on the base and rotatable relative to the base; and a rotating body that deflects an output laser measurement beam and a return portion of the laser measurement beam, the rotating body being mounted on the support and rotatable relative to the support. The generation of the LIDAR data includes: continuous rotation of the support relative to the base and continuous rotation of the rotating body relative to the support; and emission of the laser measurement beam via the rotating body and detection of the portion of the laser measurement beam that is returned via the rotating body.

[0090] According to this aspect of the invention, the laser scanner includes a reference component having a curved surface, wherein the reference component is configured and arranged so that the output laser measurement beam can generate a trajectory on the curved surface due to continuous rotation of the rotating body relative to the support portion. At different trajectory positions along the trajectory, the output laser measurement beam is generated with different incident angles of the curved surface, which causes different light portions to be scattered back along the incident direction of the laser measurement beam according to the trajectory position.

[0091] In one embodiment, the curved surface is arched outwardly (ie in the direction of the body of rotation).

[0092] In another embodiment, the top of the curved surface is configured to have uniform reflective properties along the track, in particular over the entire surface. Alternatively, the top of the curved surface is configured to have variable reflective properties along the track, in particular wherein the top is configured to have a transition from high reflectivity to low reflectivity at least along a portion of the track.

[0093] In another embodiment, the reference component is embedded in a recess of the support, in particular wherein the side walls of the recess are configured to act as a radiation swamp.

[0094] In another embodiment, the reference component is arranged relative to the rotating body in a direction toward the base (ie, the lowest point position).

[0095] In another embodiment, a motor is arranged in the support portion to rotate the support portion relative to the base portion, and the curved surface is formed by a motor cover of the motor.

[0096] In another embodiment, the reference component is arranged relative to the rotating body in a direction facing away from the base (ie, the highest point position).

[0097] In another embodiment, the laser scanner includes another reference component, which is configured and arranged so that due to the continuous rotation of the rotating body relative to the support part, the output laser measurement beam can generate a trajectory on the surface of the other reference component, and at different trajectory positions along the trajectory, different parts of the light are backscattered along the incident direction of the laser measurement beam according to the trajectory position.

[0098] For example, the other reference component is arranged relative to the rotating body in a direction toward the base (ie, the lowest point position), or the other reference component is arranged in a direction away from the base (ie, the highest point position).

[0099] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, and the mobile reality capture device has: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device.

[0100] According to this aspect of the invention, the mobile reality capture device comprises a laser scanner as described above, wherein the laser scanner is configured to generate LIDAR data during movement of the mobile reality capture device.

[0101] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device configured to be carried and moved by a mobile carrier (particularly a person or a robot or a vehicle) and to move during a measurement process to generate a digital representation of an environment.

[0102] Here, the device has a positioning unit, which includes a positioning camera, in particular a global shutter camera, and in particular an inertial measurement unit (IMU), which is configured to determine the trajectory of the mobile reality capture device by means of a positioning algorithm involving positioning image data generated by the positioning camera. Furthermore, the reality capture device has: a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement; and a detail camera, which has a higher resolution as a positioning camera, in particular wherein the detail camera is a color camera.

[0103] According to this aspect of the invention, the mobile reality capture device is configured to continuously generate positioning image data through a positioning camera and to generate detail image data through a detail camera, wherein the detail image data is generated at a low rate compared to the acquisition rate of the positioning camera that continuously generates the positioning image data.

[0104] In one embodiment, the positioning unit comprises three (in particular global shutter) positioning cameras, wherein the detail camera and the three positioning cameras are arranged circumferentially on a side surface of the mobile reality capture device, the side surface defining a vertical axis of the mobile reality capture device, i.e., wherein the side surface is arranged circumferentially around the vertical axis. Moreover, two of the three positioning cameras are located in separate planes orthogonal to the vertical axis and in particular have different angular orientations with respect to a rotation around the vertical axis.

[0105] In particular, two of the three positioning cameras and the detail camera are located in the same plane orthogonal to the vertical axis, wherein the one of the three positioning cameras located in a separate plane orthogonal to the vertical axis has the same angular orientation as the detail camera. Thus, the mobile reality capture device can be configured to run a stereo photogrammetry algorithm on the image data taking into account the separation of the orthogonal planes.

[0106] In another embodiment, the positioning unit includes multiple positioning cameras, in particular, wherein the multiple positioning cameras are configured and arranged for a nominal minimum operating range of the positioning unit, and each of the multiple positioning cameras has a field of view that overlaps with at least another positioning camera of the multiple positioning cameras.

[0107] In another embodiment, the mobile reality capture device is configured to generate a panoramic image based on the positioning image data and / or the detail image data, i.e., wherein separate images from different viewing directions are stitched together to form an image having a wider field of view than the separate images, for example, wherein the mobile reality capture device is configured to generate a 360° panoramic image.

[0108] In another embodiment, the mobile reality capture device is configured such that the generation of the detail image data and / or positioning image data is triggered by at least one of: a user command; a movement parameter that specifically indicates a pause of the mobile reality capture device; the position of the mobile reality capture device along the trajectory and / or the orientation of the mobile reality capture device relative to the trajectory; and a constant repetition rate.

[0109] In another embodiment, the field of view of the positioning camera or the field of view of each positioning camera in the plurality of positioning cameras is at least 90°×120°.

[0110] In another embodiment, the field of view of the detail camera is at least 80°×80°.

[0111] In another embodiment, the mobile reality capture device is configured to determine data for colorizing a three-dimensional point cloud represented by LIDAR data, wherein at least a portion of the positioning image data is processed for the purpose of including color information to the three-dimensional point cloud represented by the LIDAR data.

[0112] In another embodiment, the process of including color information into the three-dimensional point cloud is based on a selection algorithm that is configured to specifically determine and select image areas of individual images in the positioning image data to be used for colorization, i.e., wherein only sub-images that include image points to be matched with points in the three-dimensional point cloud are used.

[0113] In another embodiment, the mobile reality capture device is configured to generate positioning image data to be involved in a positioning algorithm at a first rate; and generate positioning image data for colorizing a three-dimensional point cloud represented by LIDAR data at a second rate, wherein the second rate is lower than the first rate.

[0114] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process of generating a digital representation of an environment, the mobile reality capture device having: a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement; a camera unit, which is configured to generate image data including a colorized image for colorizing the three-dimensional point cloud represented by the LIDAR data; and a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to determine the trajectory of the mobile reality capture device with the aid of a positioning algorithm involving the image data generated by the camera unit.

[0115] According to this aspect of the invention, the camera unit is configured to generate first image data and second image data by the same camera in the camera unit at different rates, the first image data being specifically foreseen for colorizing a three-dimensional point cloud represented by the LIDAR data, and the second image data being generated at a higher rate than the first image data being specifically foreseen for use by a localization algorithm.

[0116] In one embodiment, the camera unit is configured to generate the second image data as monochrome image data.

[0117] In another embodiment, the camera unit is configured to generate the second image data by means of a sub-sampling readout mode of the camera, the sub-sampling readout mode providing an increased frame rate compared to a high-resolution readout mode.

[0118] In another embodiment, the mobile reality capture device is configured to determine data for colorizing a three-dimensional point cloud represented by LIDAR data, wherein at least a portion of the first image data is processed for colorization, which includes specifically determining and selecting image areas of separate images in the first image data to be used for colorization, i.e., wherein only sub-images including image points to be matched with points in the three-dimensional point cloud are used for colorization.

[0119] In another embodiment, the mobile reality capture device is configured to generate third image data for generating a panoramic image by the same camera that is foreseen to generate the first image data and the second image data, in particular wherein the mobile reality capture device is configured to generate a panoramic image, i.e. wherein separate images generated by the camera unit from different viewing directions are stitched together to form an image having a wider field of view than the separate images.

[0120] In another embodiment, at least two of the first image data, the second image data, and the third image data originate from the same acquisition process of the camera, wherein a raw image is generated, and the camera unit is configured to generate image data with different resolutions for the same acquisition process by means of an image compression or subsampling algorithm that is executed on the raw image, in particular on a field programmable gate array (FPGA) or a graphics processing unit (GPU).

[0121] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process of generating a digital representation of an environment, the mobile reality capture device having: a camera unit, which is configured to generate image data of a panoramic image, in particular, wherein the camera unit is configured to generate a panoramic image, that is, wherein separate images of the camera unit from different viewing directions are stitched together to form an image with a wider field of view than the separate images; a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to determine the trajectory of the mobile reality capture device with the aid of a positioning algorithm involving the image data generated by the camera unit; and a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud.

[0122] According to this aspect of the invention, the camera unit is configured to generate first image data and second image data at different rates by the same camera in the camera unit, the first image data being specifically generated and stored for generating a panoramic image, wherein the camera adjusts a shutter speed for generating the first image data by allowing sudden corrections within a first tolerance range, the first tolerance range essentially spanning a nominal range of the camera, and second image data generated at a higher rate than the first image data is specifically foreseen to be used by a positioning algorithm, wherein the camera adjusts a shutter speed for generating the second image data by only allowing gradual corrections within a second tolerance range, the second tolerance range being smaller than the first tolerance range and being specifically tuned to the working range of the positioning unit.

[0123] In one embodiment, the camera generating the first image data and the second image data is a rolling shutter camera, and the mobile reality capture device includes: a motion determination unit, which in particular has an IMU, the motion driving unit is configured to determine motion parameters indicating the movement of the mobile reality capture device; and a compensation unit, which is configured to generate the second image data by correcting an initial image captured for generating the second image data, wherein the motion blur of the initial image caused by the rolling shutter is corrected by taking into account the motion parameters.

[0124] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device configured to be carried and moved by a mobile carrier (particularly a person or a robot or a vehicle) and moved during a measurement process of generating a digital representation of an environment. The mobile reality capture device also has a housing enclosing an interior space, wherein the interior space has: a first area without a rotating part; and a second area including a rotating part, the rotating part being configured to provide a scanning movement of a laser measurement beam for generating light detection and ranging (LIDAR) data.

[0125] According to this aspect of the invention, the device has a cooling system having a fan unit, an air inlet for letting outside air into a first area, and an air duct for transferring air (i.e., air entering across the air inlet) from the first area to a second area.

[0126] In one embodiment, the cooling system comprises a two-stage filtration system having: a first filter, in particular at least a rainproof filter, which is arranged at the air inlet and separates the interior space from the surrounding environment; and a second filter, which is arranged in the interior space and separates the interior space into a dirty interior area and a clean interior area, the dirty interior area being located upstream of the second filter between the first filter and the second filter, and the clean interior area being located downstream of the second filter between the second filter and the air outlet for releasing air into the surrounding environment.

[0127] In another embodiment, the first filter has a coarser filter fineness than the second filter.

[0128] In particular, the two-stage filter system has a modular design, which provides modular removal of the first filter and / or the second filter, for example wherein the first filter and / or the second filter are configured to be washable and / or vacuum cleanable.

[0129] In another embodiment, the mobile reality capture device has a filter monitoring system configured to monitor a contamination level of the two-stage filter system based specifically on determining an air resistance of air flowing through the two-stage filter system.

[0130] In another embodiment, the mobile reality capture device includes a fan control unit configured to control a fan for generating an airflow for bringing external air into the first area, wherein the fan control unit is configured to prevent the generation of airflow based on a determined state of the two-stage filtration system (in particular, based on a threshold of the contamination level of at least one of the first filter and the second filter), and / or based on detection of a missing filter in the two-stage filtration system.

[0131] In another embodiment, the first region and the second region respectively include a stator or a rotor, which is configured to provide an air channel between the first region and the second region, wherein both the stator and the rotor have annular segment slits between the radial struts, which open a ventilation opening between the first region and the second region in at least some relative rotational positions of the rotor and the stator.

[0132] In another embodiment, the cooling system comprises an overpressure-based air outlet (particularly a non-return valve) and / or the air outlet is configured to protect the interior space from contamination by allowing air from the outside to flow back with the aid of a filter unit, particularly wherein the filter unit comprises a filter of the same type as the first filter or the second filter.

[0133] In another embodiment, the mobile reality capture device includes a device for guiding air, which guides air from the first area separately to different areas of the interior space, such as guiding air separately to the second area and an area including cooling fins of a computing processor of the mobile reality capture device.

[0134] In another embodiment, the second area includes at least one of: a LIDAR sensor, a deflection optic to deflect the laser measurement beam, and a laser emitter to emit the laser measurement beam.

[0135] In another embodiment, the mobile reality capture device has a positioning unit, which in particular comprises an inertial measurement unit (IMU), and is configured to generate positioning data for determining a trajectory of the mobile reality capture device.

[0136] In another embodiment, the rotating part is associated with a laser scanner, which is configured to perform a scanning movement of the laser measurement beam as a scanning movement relative to two rotation axes during the movement of the mobile reality capture device, and generate LIDAR data for generating a three-dimensional point cloud based on the scanning movement.

[0137] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (particularly a person or a robot or a vehicle) and moved during a measurement process of generating a digital representation of an environment, the mobile reality capture device having: a shell that encloses an internal space, wherein the internal space includes a rotating component that is configured to rotate and, based on the rotation, provide a scanning movement of a laser measurement beam for generating light detection and ranging (LIDAR) data.

[0138] According to this aspect of the invention, the rotating assembly for providing the scanning movement of the laser measurement beam is further configured to provide generation of an air flow to drive an air cooling system of the mobile reality capture device, in particular wherein the rotating assembly is equipped with rotor blades.

[0139] In one embodiment, the rotating assembly drives internal air circulation that is isolated from the surrounding environment and includes a cooling section that cools the passing air.

[0140] In another embodiment, the mobile reality capture device has an air inlet, and the cooling system is configured such that rotation of the rotating assembly causes external air to flow in through the air inlet.

[0141] In another embodiment, the rotating component is equipped with rotor blades, the rotating component is arranged in the following area: the area has an air inlet for supplying air to the air channel in the area and an air outlet for removing air from the area, and the rotating blades are configured and arranged so that due to the rotation of the rotating component, the rotating blades pass through the inlet or the outlet respectively so as to generate a pressure gradient at the inlet or the outlet respectively, which pressure gradient drives the air from the inlet to the outlet.

[0142] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process of generating a digital representation of an environment, the mobile reality capture device having: a simultaneous localization and mapping (SLAM) unit, the SLAM unit in particular including an inertial measurement unit (IMU), the SLAM unit being configured to generate SLAM data and based on the SLAM data, a three-dimensional map of the environment and a trajectory of the mobile reality capture device in the three-dimensional map. Moreover, the reality capture device has a detection unit, which includes at least a laser scanner and in particular a camera, wherein the detection unit is configured to generate detection data of the environment during movement of the mobile reality capture device.

[0143] According to this aspect of the invention, the mobile reality capture device has a data evaluation unit, which is configured to perform an evaluation on the geometric relationship between the collection position of the mobile reality capture device in the three-dimensional map and the area in the three-dimensional map to be detected from the collection position, wherein the mobile reality capture device is configured to control the generation of detection data by considering the evaluation of the geometric relationship.

[0144] In one embodiment, the SLAM unit is configured to include visual simultaneous localization and mapping (VSLAM) for generating a three-dimensional map, and / or the SLAM unit is configured to include LIDAR-based simultaneous localization and mapping (LIDAR-VSLAM) for generating a three-dimensional map. For example, the mobile reality capture device is configured to select at least a portion of the SLAM data as detection data by considering an evaluation of geometric relationships, and more particularly, wherein the mobile reality capture device is configured to delete at least a portion of the SLAM data that is not selected as detection data.

[0145] In another embodiment, the data evaluation unit is further configured to perform a quality assessment of the detection data based on geometric relationships, and the quality assessment is based on at least one of the following items: an expected spatial resolution of the detection data of the area to be detected in the three-dimensional map, an expected signal strength of the detected detection signal used to generate the detection data of the area to be detected in the three-dimensional map, an expected signal-to-noise ratio of the detection data of the area to be detected in the three-dimensional map, an expected angle of incidence of the detection radiation on the surface of the area to be detected in the three-dimensional map for generating the detection data, and a comparison of the focal length and the expected distance between the mobile reality capture device and the area to be detected in the three-dimensional map.

[0146] In another embodiment, the data evaluation unit is configured to perform quality assessment on the SLAM data based on the three-dimensional map, wherein the mobile reality capture device is configured to control the generation of the detection data by considering a comparison between the quality assessment of the SLAM data for the same area in the three-dimensional map and the quality assessment of the detection data.

[0147] In another embodiment, the mobile reality capture device is configured to perform a comparison between the collected detection data associated with the multiple acquisitions, in particular in real time, by taking into account an evaluation of geometric relationships associated with the acquisition positions of the multiple acquisitions of the area to be detected of the three-dimensional map, and to delete at least part of the collected detection data based on the comparison.

[0148] In another embodiment, the mobile reality capture device is configured to select and / or adaptively control a data acquisition mode of the detection unit for generating detection data by taking into account the evaluation of the geometrical relationship.

[0149] In another embodiment, the detection unit includes a laser scanner, which is configured to perform a scanning movement of a laser measurement beam during the movement of the mobile reality capture device, and generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement, and the mobile reality capture device is configured to select and / or adaptively control at least one of the following items: a point repetition rate of the laser scanner, and movement parameters of the scanning movement, in particular a rotation speed of the laser measurement beam relative to one rotation axis and / or two rotation axes.

[0150] In another embodiment, the detection unit includes a detail camera, in particular wherein the SLAM unit is configured to include VSLAM and the detail camera has a higher resolution than the VSAL camera of the SLAM unit, and the mobile reality capture device is configured to select and / or adaptively control at least one of: a shutter speed of the detail camera, an exposure time of the detail camera, and an exposure rate of the detail camera.

[0151] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a simultaneous localization and mapping (SLAM) unit, the SLAM unit in particular including an inertial measurement unit (IMU), the SLAM unit being configured to generate SLAM data and, based on the SLAM data, generate a three-dimensional map of the environment and a trajectory of the mobile reality capture device in the three-dimensional map; a detection unit, the detection unit including at least a laser scanner and in particular a camera, wherein the detection unit is configured to generate detection data of the environment during movement of the mobile reality capture device.

[0152] According to this aspect of the invention, the mobile reality capture device also has a data evaluation unit, which is configured to perform quality assessment on the SLAM data based on the three-dimensional map, wherein the mobile reality capture device is configured to control the generation of detection data by considering the quality assessment of the SLAM data.

[0153] In one embodiment, the quality assessment of the SLAM data is based on at least one of the following items: the spatial resolution provided by the SLAM data of the area in the three-dimensional map, the intensity level provided by the SLAM data of the area in the three-dimensional map, the signal-to-noise ratio provided by the SLAM data of the area in the three-dimensional map, the geometric relationship between the acquisition position of the mobile reality capture device for generating SLAM data and the area in the three-dimensional map captured by the SLAM data, and a comparison of the focal length of a visual SLAM (VSLAM) camera and the distance between the mobile reality capture device and the area in the three-dimensional map captured by the VSLAM camera.

[0154] In another embodiment, the mobile reality capture device is configured to delete detection data captured by the detection unit of an area in the three-dimensional map based on a quality assessment of the SLAM data.

[0155] In another embodiment, the mobile reality capture device is configured to select and / or adaptively control a data acquisition mode of the detection unit for generating detection data by taking into account a quality assessment of the SLAM data.

[0156] In another embodiment, the mobile reality capture device is configured to enable generation of detection data through user input.

[0157] In another embodiment, the detection unit comprises a detail camera, in particular wherein the SLAM unit is configured to comprise a VSLAM, and the detail camera has a higher resolution than the VSLAM camera of the SLAM unit. The detection unit has a burst mode, which can be enabled by a user input, and in the burst mode, the detail camera continuously generates image data for the photogrammetry algorithm, for example wherein the duration of the burst mode can be controlled according to the user input.

[0158] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a detection unit, which includes at least a laser scanner and in particular a camera, wherein the detection unit is configured to generate detection data of the environment for generating a three-dimensional point cloud during the movement of the mobile reality capture device; and a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device.

[0159] According to this aspect of the invention, the reality capture device has: an edge computing function, which is configured to generate a three-dimensional vector file model of the environment, in particular a mesh, based on the detection data; and an in particular wireless data interface, which is configured to provide a data stream including the detection data and the three-dimensional vector file model to a separate receiving device, wherein the mobile reality capture device is configured to generate and provide the data stream based on a prioritization algorithm, which is configured to prioritize the streaming of the data of the three-dimensional vector file model over the streaming of the detection data.

[0160] In one embodiment, the mobile reality capture device is configured to delete at least a portion of the detection data based in particular on the identified semantic features and / or geometric features.

[0161] In another embodiment, the mobile reality capture device is configured to mark at least a portion of the detection data as redundant in the view of the three-dimensional vector file model, in particular the detection data associated with a surface in the three-dimensional vector file model.

[0162] In another embodiment, the mobile reality capture device is configured to omit detection data marked as redundant when generating the data stream.

[0163] In another embodiment, the mobile reality capture device is configured to perform a quality assessment of the collected detection data based on the three-dimensional vector file model, and to control the generation of the detection data by taking into account the quality assessment of the detection data.

[0164] In another embodiment, the mobile reality capture device is configured to control the generation of the detection data by performing an evaluation of a geometric relationship between a capture position of the mobile reality capture device within the three-dimensional vector file model and an area in the three-dimensional vector file model to be detected from the capture position. For example, the mobile reality capture device is further configured to perform a quality assessment on the detection data based on the geometric relationship.

[0165] In another embodiment, the quality assessment of the collected detection data according to the geometric relationship or the quality assessment of the detection data is based on at least one of the following items: the spatial resolution provided by the detection data, the signal strength of the detection signal used to generate the detection data, the signal-to-noise ratio provided by the detection data, the incident angle of the detection radiation on the surface represented by the surface of the three-dimensional vector file model, and the comparison of the focal length and the distance between the mobile reality capture device and the point within the three-dimensional vector file model.

[0166] In another embodiment, the three-dimensional vector file model is a grid with a fixed grid size, for example, wherein the mobile reality capture device is configured to set the grid size.

[0167] In another embodiment, the three-dimensional vector file model is a grid with a variable grid size, for example wherein the grid size is automatically adapted to the representation of the identified semantic and / or geometric features or based on a quality assessment of the acquired detection data.

[0168] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a simultaneous localization and mapping (SLAM) unit, the SLAM unit in particular including an inertial measurement unit (IMU), the SLAM unit being configured to generate SLAM data and based on the SLAM data, generate a three-dimensional map of the environment and a trajectory of the mobile reality capture device in the three-dimensional map, wherein the three-dimensional map is generated by identifying multiple features in the environment that allow the SLAM data to be linked to each other.

[0169] According to this aspect of the invention, the mobile reality capture device includes a feature tracker configured to determine position data of a subset of the plurality of features at different positions of the mobile reality capture device along the trajectory. For each of the different positions of the mobile reality capture device along the trajectory, the corresponding position data provides a relative positional relationship between the subset of features relative to the corresponding position of the mobile reality capture device along the trajectory. Moreover, the mobile reality capture device is configured to reinitialize the SLAM unit for continuing the generation of the three-dimensional map by calling at least a portion of the position data.

[0170] In one embodiment, the mobile reality capture device is configured to reinitialize the SLAM unit by calling: position data that has been determined for the last (particularly the most recent) position of the mobile reality capture device along the trajectory, or a series of position data corresponding to the most recent position of the mobile reality capture device along the trajectory.

[0171] In another embodiment, the mobile reality capture device has an edge computing function, which is configured to determine the current position of the mobile reality capture device by comparing a three-dimensional model based on current SLAM data with a three-dimensional model based on previous SLAM data, wherein the mobile reality capture device is configured to select position data based on the determined current position to reinitialize the SLAM unit. For example, the mobile reality capture device is configured to generate a three-dimensional model of the environment (particularly a vector file model) based on the SLAM data, and to run a feature recognition algorithm on the three-dimensional model, and to recognize semantic features and / or geometric features based on the feature recognition algorithm. In addition, the mobile reality capture device is configured to assign at least a portion of the recognized semantic features and / or geometric features to position data of different positions of the mobile reality capture device along the trajectory, and to determine the current position along the trajectory based on the recognized semantic features and / or geometric features.

[0172] In another embodiment, the mobile reality capture device has a guidance unit configured to provide guidance from a current position of the mobile reality capture device towards a desired position. Here, the mobile reality capture device is configured to: determine a current position within a three-dimensional map or a three-dimensional model generated by SLAM data, in particular based on the identified semantic features and / or geometric features, provide guidance from the current position to a target position of determined position data on the trajectory by means of the guidance unit, and reinitialize the SLAM unit based on the position data determined for the target position.

[0173] In another embodiment, the mobile reality capture device is configured to have a built-in position determination unit for generating positioning data or receiving positioning data from an external position determination unit, wherein the position determination unit is based on at least one of the following: triangulation by means of wireless signals (particularly wireless LAN signals), radio frequency positioning, and a global navigation satellite system (GNSS). Moreover, the mobile reality capture device is configured to select position data to reinitialize the SLAM unit based on the positioning data of the position determination unit, or to provide guidance from a current position provided by the positioning data to a target position of the determined position data on the trajectory by means of a guidance unit.

[0174] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a laser scanner configured to generate light detection and ranging (LIDAR) data based on a scanning movement of a laser measuring beam relative to two rotation axes. The laser scanner has a cover, in particular a cover that is opaque to visible light, which surrounds a moving part of the laser scanner so that during the scanning movement the laser measuring beam passes through the cover at different penetration points; an angle determination unit configured to provide angle determination data for determining the orientation of the emission axis of the laser measuring beam relative to the two rotation axes; and a data memory storing spline-based correction parameters that correct the angle determination data for offsets caused by the cover.

[0175] In one embodiment, the laser scanner is configured to correct the angle determination data, in particular in real time, according to an offset provided by the correction parameters.

[0176] In another embodiment, the correction parameters are stored as a lookup table or in the form of a correction matrix.

[0177] In another embodiment, the surface of the cover has a kink, wherein the laser measuring beam passes through the kink during the scanning movement.

[0178] In another embodiment, the cover has a hemispherical tip portion that is incorporated into a cylindrical shell, for example, wherein the laser scanner is configured to generate LIDAR data based on an orientation of the laser measurement beam through the hemispherical tip portion and an orientation of the laser measurement beam through the cylindrical shell.

[0179] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a system having a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moved during a measurement process of generating a digital representation of an environment. The mobile reality capture device has: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device; a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud; and a camera, which is configured to capture image data.

[0180] Furthermore, the system comprises a companion device, which is particularly embodied as a smartphone, a tablet or a personal computer, wherein the mobile reality capture device and the companion device are configured to perform server-client communication. The system is configured to encode access data for server-client communication, in particular a service set identifier (SSID) and a password of the server, into a matrix barcode, in particular a QR code. The mobile reality capture device and / or the companion device are configured to capture an image of the matrix barcode and decode the matrix barcode from the image to establish server-client communication.

[0181] In one embodiment, the mobile reality capture device is configured to act as a server for server-client communication, and the system includes a label displaying a matrix barcode, particularly wherein the label is attached to a component of the mobile reality capture device.

[0182] In another embodiment, the companion device is configured to display a matrix barcode on a display, wherein the mobile reality capture device is configured to, in particular, automatically identify the acquisition of an image including a matrix barcode, the image being captured by a camera, decode the matrix barcode from the image, and establish server-client communication when decoding the matrix barcode.

[0183] For example, the companion device is configured to act as a server for server-client communications, or to identify access data to an available communication network and encode the access data to the available communication network into a matrix barcode.

[0184] In another embodiment, the mobile reality capture device comprises a positioning camera, which is used by the positioning unit, and the positioning camera is configured to generate positioning data by including a camera image of the positioning camera, and in particular to generate map construction data for generating a three-dimensional map. The mobile reality capture device also includes a detail camera, which has a higher resolution than the positioning camera, and the mobile reality capture device is configured to capture an image of the matrix barcode with the help of the detail camera. For example, the mobile reality capture device can be configured to capture an image of the matrix barcode that can be manually triggered by a user.

[0185] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a computer program product comprising a program code stored on a machine-readable medium or embodied by electromagnetic waves comprising program code segments and having computer executable instructions, which, in particular when run on a companion device of a mobile reality capture device according to one of the preceding claims, at least perform the following steps:

[0186] reading input data from a database, the input data comprising: positioning data for determining a trajectory of the mobile reality capture device; Light Detection and Ranging (LIDAR) data of a laser scanner of the mobile reality capture device, wherein the LIDAR data have been generated during a movement of the mobile reality capture device along the trajectory and are based on a scanning movement of a laser measuring beam relative to two rotation axes; and image data of a camera unit arranged on a side surface of the mobile reality capture device, the side surface defining a vertical axis of the mobile reality capture device, i.e., wherein the side surface is arranged circumferentially around the vertical axis, wherein the image data have been generated during the movement of the mobile reality capture device along the trajectory, in particular wherein the camera unit provides image data covering a field of view of at least 120°, in particular at least 180°, more in particular 360° around the vertical axis,

[0187] Generate a 3D model of the environment based on input data, typically a 3D point cloud or 3D vector file model,

[0188] generating a link between the image data and the three-dimensional model such that individual images in the image data can be related to the capture locations of those individual images within the three-dimensional model,

[0189] providing a thumbnail representation, wherein at least a portion of an individual image in the image data is represented as a thumbnail,

[0190] receiving a selection of one of the thumbnails, and

[0191] • Providing a representation of the three-dimensional model, in particular a three-dimensional representation, with an indication of the capture location within the three-dimensional model of the individual image associated with the selected thumbnail.

[0192] In one embodiment, the representation of the three-dimensional model comprises at least a portion of the trajectory, in particular the portion of the trajectory up to the trajectory position associated with the selected thumbnail.

[0193] In another embodiment, the thumbnail representation is based on a group of thumbnails in a two-dimensional or three-dimensional grid.

[0194] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a monitoring device for detecting objects in infrastructure (particularly buildings or facilities, more particularly areas frequently exposed to passing people and / or vehicles).

[0195] The monitoring device comprises a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud of the infrastructure based on the scanning movement. Furthermore, the monitoring device has: a camera unit configured to capture image data along at least two different imaging directions relative to the monitoring device; and an object detector configured to detect an object within the monitored infrastructure based on at least one of the LIDAR data and the image data.

[0196] According to this aspect of the invention, the monitoring device is configured to automatically switch from a low-frequency capture mode to a high-frequency capture mode when an object is detected by the object detector, wherein, in the high-frequency capture mode, at least one of the following parameters is increased compared to the low-frequency capture mode: the point repetition rate of the laser scanner, the rotation speed of the laser measurement beam relative to at least one of the two rotation axes, and the exposure rate of the camera of the camera unit.

[0197] In one embodiment, the monitoring device comprises a base supporting a laser scanner and a cover, in particular a cover that is opaque to visible light, mounted on the base such that the cover and the base surround all moving parts of the laser scanner so that the moving parts are not accessible from the outside.

[0198] In another embodiment, the monitoring device is configured to generate a three-dimensional model of the infrastructure (in particular a three-dimensional point cloud and / or a three-dimensional vector file model) by taking into account the LIDAR data, and / or to generate a panoramic image based on the image data, i.e., wherein separate images from different viewing directions of the camera unit are stitched together to form an image with a wider field of view than the separate images, in particular wherein the mobile reality capture device is configured to generate a 360° panoramic image.

[0199] In another embodiment, the monitoring device is configured to generate a data stream and provide the data stream to a separate receiving device by means of a wireless and / or wired data interface, in particular. For example, the data stream includes at least one of LIDAR data, image data, model data of a three-dimensional model, and image data of a panoramic image.

[0200] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a monitoring device for detecting objects within an infrastructure (particularly a building or facility, more particularly an area frequently exposed to traffic of people and / or vehicles), the monitoring device comprising: a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes, and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud of the infrastructure based on the scanning movement; a camera unit configured to capture image data along at least two different imaging directions relative to the monitoring device; and an object detector configured to detect objects within the monitored infrastructure based on at least one of the LIDAR data and the image data.

[0201] According to this aspect of the invention, the monitoring device is configured to generate a data stream and provide the data stream comprising LIDAR data and image data to a separate receiving device by means of, in particular, a wireless and / or wired data interface, and automatically switch from a low data mode to a high data mode when an object is detected by the object detector, wherein in the high data mode the amount of data in the data stream is increased compared to the low data mode.

[0202] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a monitoring device for detecting objects within an infrastructure (particularly a building or facility, more particularly an area frequently exposed to traffic of people and / or vehicles), the monitoring device comprising: a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes, and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud of the infrastructure based on the scanning movement; a camera unit configured to capture image data along at least two different imaging directions relative to the monitoring device; and an object detector configured to detect objects within the monitored infrastructure based on at least one of the LIDAR data and the image data.

[0203] According to this aspect of the invention, the monitoring device is configured to generate a three-dimensional model of the infrastructure by taking into account the LIDAR data, in particular a three-dimensional point cloud and / or a three-dimensional vector file model. The object detector is configured to detect objects based on the three-dimensional model, in particular wherein the object detector is configured to run a feature recognition algorithm for the three-dimensional model and recognize semantic features and / or geometric features based on the feature recognition algorithm.

[0204] Typically, the camera unit of any of the above-mentioned monitoring devices may include at least one of a visual imaging camera and a thermal imaging camera.

[0205] In particular, the camera unit comprises two visual imaging cameras having different viewing directions from each other, wherein each of the two visual imaging cameras has a field of view of at least 180 degrees, and the two visual cameras are arranged circumferentially around the laser scanner. For example, the two visual imaging cameras are arranged on an outer ring around the laser scanner and defining a central annular axis, wherein the angular spacing of the two visual imaging cameras relative to the central annular axis is 150 degrees to 180 degrees.

[0206] The camera unit may also include four thermal imaging cameras having different viewing directions from each other, wherein each of the four thermal imaging cameras has a field of view of at least 80 degrees, and the four thermal imaging cameras are arranged circumferentially around the laser scanner. For example, the four thermal imaging cameras are arranged on an outer ring around the laser scanner and defining a central annular axis, wherein the four thermal imaging cameras are spaced apart from each other at uniform angular spacing relative to the rotation around the central annular axis.

[0207] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a monitoring device for detecting objects within an infrastructure (particularly a building or facility, more particularly an area frequently exposed to traffic of people and / or vehicles), the monitoring device comprising: a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes, and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud of the infrastructure based on the scanning movement; two visual imaging cameras having different viewing directions from each other, wherein each of the two visual imaging cameras has a field of view of at least 180 degrees, and the two visual cameras are arranged circumferentially around the laser scanner; four thermal imaging cameras having different viewing directions from each other, wherein each of the four thermal imaging cameras has a field of view of at least 80 degrees, and the four thermal imaging cameras are arranged circumferentially around the laser scanner; and a particularly wireless and / or wired data interface configured to provide a data stream comprising LIDAR data and image data of the visual imaging camera and the thermal imaging camera to a separate receiving device.

[0208] In one embodiment, the monitoring device comprises: a high dynamic range (HDR) camera, in particular a single exposure HDR camera; and / or a multispectral camera, in particular a hyperspectral camera; and / or a 3D camera.

[0209] The 3D camera is configured to perform range imaging, which is the name for a collection of techniques used to generate a 2D image showing the distance to points in a scene from a specific point, usually associated with some type of sensor device. The resulting image (the range image) has pixel values ​​corresponding to the distance. If the sensor used to generate the range image is properly calibrated, the pixel values ​​can be given directly in physical units (such as meters).

[0210] In particular, the laser scanner can be replaced by a 3D camera. In another embodiment, the two visual imaging cameras are arranged on an outer ring around the laser scanner and defining a central annular axis, wherein the angular spacing of the two visual imaging cameras relative to the central annular axis is 150 to 180 degrees.

[0211] In another embodiment, the four thermal imaging cameras are arranged in an outer ring around the laser scanner and defining a central annular axis, wherein the four thermal imaging cameras are spaced apart from each other at uniform angular spacing relative to rotation around the central annular axis.

[0212] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device; and a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement.

[0213] According to this aspect of the invention, the mobile reality capture device has a socket having a recessed portion for accommodating a pin of an additional component of the mobile reality capture device along a through-axis. Furthermore, the device has a locking mechanism, wherein the locking mechanism is configured to, in a locking position as a basic position of the locking mechanism, push a locking body radially inwardly toward the through-axis so as to engage the recess of the pin, and enable the locking body to be radially disengaged by pushing a contact member in a direction along the through-axis, which allows the pin to be released.

[0214] In one embodiment, the locking mechanism comprises at least three locking bodies.

[0215] For example, the locking device is configured as a body of revolution, in particular as a sphere or ellipsoid, a trapezoid, a pyramid, a trapezoid with rounded corners, or a pyramid with rounded corners.

[0216] In another embodiment, the recess has the form of a body of revolution formed about a through-axis, and the contact parts are arranged on a circumference about the through-axis.

[0217] In another embodiment, the locking mechanism includes a sliding component configured to be axially movable along the through-axis, wherein pushing or pulling the sliding component in a locking direction parallel to the through-axis minimizes the radial movement range of the locking body, so that the locking body is pushed radially inward toward the through-axis. Moreover, the locking mechanism is configured such that when the contact component is pushed in a direction along the through-axis, the sliding component moves in a release direction opposite to the locking direction, which releases space due to the displacement of the sliding component and thereby enables the locking body to radially disengage.

[0218] In another embodiment, the sliding component has a contact surface which contacts the locking body in the locking position, so that the locking body is pressed between the contact surface of the sliding component and the contact surface of the contact component, wherein the locking mechanism is configured such that when the contact component is pushed in a direction along the through-axis, the locking body and the contact component are pushed in the same direction until the locking body can be radially disengaged.

[0219] In another embodiment, the locking mechanism comprises a prestressing member, in particular a prestressing spring, which maintains a basic position, ie the position of the sliding member in the locking position.

[0220] Furthermore, the present invention relates to a connector for connecting an additional component to a mobile reality capture device with a locking mechanism as described above. The connector has a pin configured to be inserted into a socket, wherein the pin has a circumferentially continuous recess around the pin axis, the pin being foreseen to be parallel or coaxial with the through-axis in the locked position, and wherein the recess is specifically foreseen to accommodate a locking body of the locking mechanism. Furthermore, the connector has a release mechanism configured to push the contact part in the locked position in a direction along the through-axis (i.e., in the direction of the pin axis).

[0221] In one embodiment, the release mechanism has a pushing member for pushing the contact member along the through axis, wherein the pushing member is configured to be axially movable relative to the pin along the pin axis, wherein the axial movement range of the pushing member relative to the pin allows the contact member to be sufficiently displaced to radially disengage the locking body from the recess.

[0222] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data and determine the trajectory of the mobile reality capture device based on the positioning data; and a detection unit, which includes at least a laser scanner configured to generate light detection and ranging (LIDAR) data and in particular a camera, wherein the detection unit is configured to generate detection data for determining a digital model of the environment during the movement of the mobile reality capture device.

[0223] According to this aspect of the invention, the mobile reality capture device is configured to set a data collection mode for generating detection data according to movement parameters of the mobile reality capture device.

[0224] In one embodiment, the mobile reality capture device is configured such that setting of the data collection mode involves changing a data collection rate used to generate the detection data and / or changing a spatial resolution setting represented by the detection data.

[0225] In another embodiment, the mobile reality capture device is configured to increase a data collection rate for generating detection data with increasing speed of the mobile reality capture device along the trajectory.

[0226] In another embodiment, the mobile reality capture device is configured to set the data collection mode according to the change of the trajectory (ie, according to the gradient of the trajectory).

[0227] In another embodiment, the mobile reality capture device is configured to set the data collection mode according to a speed threshold of the mobile reality capture device along the trajectory. For example, the mobile reality capture device has a stop collection mode, which is used to provide maximum data quality at the expense of maximizing data volume, for example, wherein when the speed of the mobile reality capture device drops below the speed threshold, the stop collection mode is automatically enabled.

[0228] In another embodiment, the laser scanner is configured to perform a scanning movement of the laser measurement beam during the movement of the mobile reality capture device and to generate LIDAR data based on the scanning movement, and the mobile reality capture device is configured to set the data acquisition mode by selecting and / or adaptively controlling at least one of the following items: the point repetition rate of the laser scanner, and the movement parameters of the scanning movement, in particular the rotation speed of the laser measurement beam relative to one rotation axis and / or two rotation axes.

[0229] In another embodiment, the detection unit includes a camera, and the mobile reality capture device is configured to set the data acquisition mode by selecting and / or adaptively controlling at least one of the following items: a shutter speed of the camera, an exposure time of the camera, and an exposure rate of the camera.

[0230] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data and determine the trajectory of the mobile reality capture device based on the positioning data; a detection unit, which includes at least a laser scanner configured to generate light detection and ranging (LIDAR) data and in particular a camera, wherein the detection unit is configured to generate detection data for determining a digital model of the environment during the movement of the mobile reality capture device.

[0231] According to this aspect of the invention, the mobile reality capture device includes a radio signal module, in particular a wireless local area network (WLAN) module, which is configured to measure the signal strength of the radio signal, wherein the mobile reality capture device is configured to determine the signal strength of the radio signal (the radio signal can be obtained along the trajectory of the mobile reality capture device) and provide a data set including a series of determined signal strengths of the radio signal, wherein each signal strength is associated with the position of the mobile reality capture device along the trajectory.

[0232] In one embodiment, the mobile reality capture device is configured to analyze a data set and, based on the analysis, generate a signal reception map of the environment, classifying the environment into different radio signal reception areas, wherein a signal strength value (particularly a mean or median) is associated with each of the reception areas.

[0233] In another embodiment, the mobile reality capture device is configured to analyze the data set by means of a triangulation algorithm and to generate a map including the locations of radio signal transmitters within the environment based on the analysis.

[0234] Another aspect of the invention, alone or in combination with other aspects of the invention, relates to a computer program product comprising a program code stored on a machine-readable medium or embodied by electromagnetic waves comprising program code segments and having computer executable instructions, which, in particular when run on a companion device of a mobile reality capture device according to one of the preceding claims, perform at least the following steps:

[0235] reading input data from a database, the input data comprising: positioning data for determining a trajectory of the mobile reality capture device, light detection and ranging (LIDAR) data of a laser scanner of the mobile reality capture device, wherein the LIDAR data have been generated during a movement of the mobile reality capture device along the trajectory and are based on a scanning movement of a laser measuring beam relative to two rotation axes; and image data of a camera unit arranged on a side surface of the mobile reality capture device, the side surface defining a vertical axis of the mobile reality capture device, i.e., wherein the side surface is arranged circumferentially around the vertical axis, wherein the image data have been generated during the movement of the mobile reality capture device along the trajectory, in particular wherein the camera unit provides image data covering a field of view of at least 120°, in particular at least 180°, more in particular 360° around the vertical axis,

[0236] generating panoramic image data providing a panoramic image along the trajectory such that there is a continuous panoramic image along the trajectory, the panoramic image being an image in which individual images of different viewing directions of the camera unit are stitched together to form an image having a wider field of view than the individual images, in particular wherein the panoramic image provides a 360° field of view around the vertical axis, and

[0237] Providing a variable display of panoramic image data, wherein when moving along a trajectory, the panoramic image changes continuously during the movement.

[0238] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device; a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud; and a camera unit, which is arranged on a side surface of the mobile reality capture device, which side surface defines a vertical axis of the mobile reality capture device, that is, wherein the side surface is arranged circumferentially around the vertical axis, wherein the camera unit is configured to provide image data covering a field of view of more than 180°, in particular 360° around the vertical axis.

[0239] In one embodiment, the camera unit includes a plurality of cameras that are arranged circumferentially on a side surface, and the mobile reality capture device is configured to generate a panoramic image based on the image data, i.e., wherein individual images from the plurality of cameras are stitched together to form an image having a wider field of view than the individual images.

[0240] In another embodiment, the mobile reality capture device is configured to generate a three-dimensional map of the environment by performing simultaneous localization and mapping (SLAM) involving at least one of the following items: data from the IMU (IMU-SLAM); image data from the camera unit for visual simultaneous localization and mapping (VSLAM); and LIDAR data for LIDAR-based simultaneous localization and mapping (LIDAR-SLAM).

[0241] In another embodiment, the mobile reality capture device is configured to generate a colorized three-dimensional point cloud based on the LIDA data and the image data of the camera unit.

[0242] In another embodiment, the mobile reality capture device is configured to generate a three-dimensional vector file model of the environment, in particular a mesh.

[0243] In another embodiment, the mobile reality capture device is configured to adaptively set at least one of the following items: generate LIDAR data by adaptively setting at least two different rotation rates of at least one of the two rotation axes and / or by adaptively setting at least two different point acquisition rates; generate image data to be used for panoramic images, in particular image data provided by a camera of a camera unit; generate image data to be used to determine the trajectory of the mobile reality capture device; and generate image data to be used to colorize a three-dimensional point cloud, in particular image data provided by a camera of a camera unit.

[0244] In another embodiment, the mobile reality capture device comprises an attachment unit for attaching an accessory device to the mobile reality capture device. The attachment unit has: a fixing unit having a socket, the socket being configured to receive a counterpart of the socket, and in particular to fix the counterpart in the socket in a manner to maintain a particularly predetermined orientation of the counterpart relative to the socket; and a wireless data bus configured to provide unidirectional or bidirectional data transmission between the accessory device and the mobile reality capture device.

[0245] For example, the fixing unit includes at least one of: a magnet, a part of a hook and loop fastener, a female part or a male part of a plug-in connection, and a clamp.

[0246] In another embodiment, the attachment unit has an inductive power exchange unit configured to supply power from the mobile reality capture device to the accessory device secured by the securing unit and / or from the secured accessory device to the mobile reality capture device.

[0247] In another embodiment, the mobile reality capture device includes a sensing unit configured to: detect an accessory device within the range (reach) of wireless data transmission by the wireless data bus, and upon detecting the accessory device within the range, enable the wireless data bus to start data transmission, and / or detect the accessory device being fixed by the fixing unit, and upon detecting the fixed accessory device, enable the inductive power exchange unit to start power exchange.

[0248] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a detection unit, the detection unit comprising at least a laser scanner configured to generate light detection and ranging (LIDAR) data and in particular a camera, wherein the detection unit is configured to generate detection data for determining a three-dimensional model of the environment during the movement of the mobile reality capture device; a simultaneous localization and mapping (SLAM) unit, the SLAM unit in particular comprising an inertial measurement unit (IMU), the SLAM unit being configured to access the detection data and generate a three-dimensional map of the environment and a trajectory of the mobile reality capture device in the three-dimensional map based on the detection data; and a guidance unit, the guidance unit being configured to provide guidance from the current position of the mobile reality capture device toward an area of ​​the environment.

[0249] According to this aspect of the invention, the mobile reality capture device is configured to generate guidance data that provides guidance from a current position of the mobile reality capture device to a target position, wherein the guidance data is generated based on a comparison of a three-dimensional map with a known model of the environment, and is configured to enable a guidance unit to provide guidance from the current position of the mobile reality capture device to the target position.

[0250] In one embodiment, the known model of the environment is based on detection data from previous measurement activities performed by the mobile reality capture device.

[0251] In another embodiment, the mobile reality capture device comprises a positioning unit, which is in particular based on a Global Navigation Satellite System (GNSS) and / or based on radio positioning and / or based on a compass, wherein the guidance data are generated based on positioning data from the positioning unit.

[0252] Another aspect of the present invention, alone or in combination with other aspects of the present invention, relates to a mobile reality capture device, which is configured to be carried and moved by a mobile carrier (in particular a person or a robot or a vehicle) and moves during a measurement process for generating a digital representation of an environment, the mobile reality capture device having: a positioning unit, which in particular includes an inertial measurement unit (IMU), which is configured to generate positioning data for determining the trajectory of the mobile reality capture device; and a laser scanner, which is configured to perform a scanning movement of a laser measurement beam relative to two rotation axes during the movement of the mobile reality capture device, and based on the scanning movement, generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud.

[0253] According to this aspect of the invention, the mobile reality capture device has an axial flux motor, which includes a rotor and a stator, which are configured to magnetically interact to drive a rotating component that provides a scanning movement about one of the two rotational axes.

[0254] In one embodiment, the axial flux motor has: a plurality of coils fixedly arranged in a ring around said one of the two rotation axes, wherein the coils have a winding axis parallel to said one of the two rotation axes; a plurality of magnets arranged axially opposite to the coils, wherein adjacent magnets have alternating polarities; and an actuating device configured to actuate the plurality of coils so that interaction between the coils and the plurality of magnets provides for applying torque to a rotating component.

[0255] In a further embodiment, the coil is arranged on one side of a printed circuit board, for example wherein the flow deflection element is arranged on the other side of the printed circuit board, or wherein the flow deflection element and the printed circuit board are components of a composite plate.

[0256] In another embodiment, the magnets are arranged on a rotor disk which is connected in a rotationally fixed manner to a shaft which is connected in a rotationally fixed manner to a rotating assembly.

[0257] In another embodiment, the mobile reality capture device is configured such that a bearing preload of a bearing of the shaft is specifically affected by an axial force generated by selectively actuating the coil.

[0258] In a further embodiment, the rotor disk is axially surrounded on both sides by a unit which in each case has fixed coils arranged in the form of a ring, in particular wherein one of the units is a circuit board and wherein the magnet is positioned opposite the coil.

[0259] In another embodiment, the mobile reality capture device is configured to have an operating mode for energy regeneration during braking of said one of said two rotation axes of the rotor.

[0260] In another embodiment, the mobile reality capture device is configured to apply torque on the rotating component by energizing the coil via the actuation device so that the resulting rotating electromagnetic field applies a circumferential magnetic force to a rotor disk provided with magnets.

[0261] Another aspect of the present invention, either alone or in combination with other aspects of the present invention, relates to a monitoring device for detecting objects in infrastructure (particularly buildings or facilities, more particularly areas frequently exposed to people and / or vehicles). The monitoring device comprises: a 3D camera for generating a three-dimensional point cloud of the infrastructure; a visual imaging camera, particularly two visual imaging cameras with different viewing directions, wherein each of the two visual imaging cameras has a field of view of at least 180 degrees, and the two visual cameras are arranged around the circumference of a laser scanner; a thermal imaging camera, particularly four thermal imaging cameras with different viewing directions, wherein each of the four thermal imaging cameras has a field of view of at least 80 degrees, and the four thermal imaging cameras are arranged around the circumference of the laser scanner; and a particularly wireless and / or wired data interface, which is configured to provide a data stream including data from the 3D camera and image data of the visual imaging camera and the thermal imaging camera to a separate receiving device.

[0262] A 3D camera is a device configured to perform range imaging (which is the name for a collection of technologies used to generate 2D images showing the distances to points in a scene from a specific point, usually associated with some type of sensor device). The resulting image (the range image) has pixel values ​​that correspond to distances. If the sensor used to generate the range image is properly calibrated, the pixel values ​​can be given directly in physical units (such as meters).

[0263] For example, the monitoring device includes: a high dynamic range (HDR) camera, in particular a single exposure HDR camera; and / or a multi-spectral camera, in particular a hyperspectral camera.

[0264] In another embodiment, the 3D camera is embodied as one of: a stereoscopic imaging arrangement, a structured light sensor, and a time-of-flight camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0265] Below, various aspects of the invention are described or explained in more detail, purely by way of example, with reference to the working examples schematically shown in the accompanying drawings. The same reference numerals are used to identify the same elements in the drawings. The embodiments are generally not shown to scale, and these embodiments should not be interpreted as limiting the invention. Specifically,

[0266] Figure 1 It is an exemplary application of mobile reality capture device in building measurement;

[0267] Figure 2 is an exemplary embodiment of a mobile reality capture device according to the present invention;

[0268] Figure 3 is to Figure 2 An exemplary embodiment of a laser scanner for use within a mobile reality capture device;

[0269] Figure 4 is an exemplary embodiment of a rotating body configured to send an output laser measurement beam from an interior of the rotating body through a passage region into an environment;

[0270] Figure 5 yes Figure 4 A cross section of a rotating body without prisms;

[0271] Figure 6 yes Figure 4 A cross section of a rotating body with a prism inserted therein;

[0272] Figure 7 is a cross-section of a rotating body configured to hold a cylindrical prism;

[0273] Figure 8 yes Figure 3 An exemplary shape of a cover of a laser scanner;

[0274] Fig. 9 is an exemplary embodiment of a plurality of light indicators, wherein each of the light indicators is assigned to a scanning segment that is fixed relative to a mobile reality capture device;

[0275] Fig.10 is the area where additional data is needed for detection, e.g. Fig. 9 The depicted light indicator provides guidance to the area being inspected;

[0276] Fig.11 is an exemplary embodiment of a laser scanner including a reference component having a curved surface;

[0277] Fig.12is an exemplary embodiment of a cooling system having a first area without rotating parts and a second area including rotating parts for the scanning movement of the laser measuring beam, wherein the cooling system has an air inlet for letting outside air into the first area;

[0278] Fig.13 is controlling the data storage by considering an evaluation of the geometric relationship between the acquisition location and the area to be probed from the acquisition location;

[0279] Fig.14 The SLAM unit is reinitialized by calling the relative position relationship between the SLAM features and the position of the mobile reality capture device along the previous trajectory;

[0280] Fig.15 is a system including a mobile reality capture device and a companion device configured to establish server-client communication;

[0281] Fig.16 It is an exemplary application of the reality capture device as a monitoring device, here it is the monitoring of a subway station;

[0282] Fig.17 is an exemplary embodiment of a monitoring device according to the present invention;

[0283] Fig.18 is an exemplary embodiment of a socket for attaching a mobile reality capture device to an additional component;

[0284] Fig.19 The invention relates to determining a signal strength of a radio signal obtainable along a trajectory of a mobile reality capture device, the radio signal being used to determine a heat map indicating classification of an environment into different radio signal reception areas. DETAILED DESCRIPTION

[0285] Figure 1 An exemplary application of the mobile reality capture device 1 in the field of architectural construction or real estate is shown, for example, where an architect or potential home buyer wants to have a 3D model of a room or an entire building to provide improved visualization of details or potential expansion plans.

[0286] The mobile reality capture device 1 comprises: a localization unit, a laser scanner and a camera unit, for example for the purpose of providing a simultaneous localization and mapping functionality, wherein the mobile reality capture device is configured to be carried by a user through a room. During the movement of the mobile reality capture device, the room is surveyed, wherein data from the laser scanner and the camera unit are referenced to each other by means of the localization unit, for example within the scope of a SLAM functionality.

[0287] Due to the user's movements, objects and spatial areas may be measured from different angles, and as a result, shadows and / or blind spots may be avoided.

[0288] The laser scanner is configured to scan a room with the aid of a laser measurement beam 2, which moves relative to two rotation axes, such as a vertical rotation axis 3 (commonly referred to as a "slow" axis or azimuth axis) and a rotation axis 4 perpendicular to the vertical rotation axis 3 (also commonly referred to as a "fast" axis or elevation axis).

[0289] For example, the desired point-to-point resolution may be adjusted by adjusting the pulse rate of the laser measurement beam 2 and / or by adjusting the rotation speeds about both rotation axes, wherein the rotation about the elevation axis 4 is typically set higher than the rotation about the azimuth axis 3 .

[0290] The camera of the camera unit (see below) is arranged on a side surface of the mobile reality capture device, which side surface defines a vertical axis 5 of the mobile reality capture device, wherein the side surface is arranged circumferentially around the vertical axis. For example, the camera unit is configured to provide image data that substantially instantaneously covers a field of view of more than 180°, in particular 360°, around the vertical axis 5.

[0291] For example, the camera unit includes: one or more cameras foreseen to be used in visual SLAM functionality, such as one or more color cameras for colorizing three-dimensional point clouds, one or more high-resolution cameras for providing high-resolution detail images, one or more high dynamic range (HDR) cameras (e.g., single exposure HDR cameras), one or more multispectral (particularly hyperspectral) cameras for identifying surface features or distinguishing different types of surfaces, and one or more thermal imaging cameras for providing temperature information, for example.

[0292] The mobile reality capture device 1 may also include other sensors or have additional auxiliary device interfaces, such as an interface for attaching a GNSS rover or a display.

[0293] In particular, the mobile reality capture device 1 is configured to communicate with an external processing unit of a companion device (e.g., a computer, a tablet computer, or a smartphone), which is configured to process at least part of the measurement data of the reality capture device 1, such as for referencing camera data with laser scanner data or for providing extended display functionality.

[0294] In particular, the reality capture device 1 is configured to transmit the measurement data to the external processing unit by means of a data stream transmission that starts simultaneously with or at least close in time to the measurement processing, e.g. via a WLAN or Bluetooth connection, so that the processing of the measurement data on the external processing unit can be performed substantially in parallel with the data acquisition. For example, in this way, the measurement data can be continuously displayed to the user as a continuously growing colorized 3D point cloud, e.g. by means of a display connected to the mobile reality capture device 1.

[0295] For example, the positioning unit is configured to determine the trajectory of the mobile reality capture device 1 with six degrees of freedom, i.e., including the position and orientation (pose) of the mobile reality capture device. In particular, the mobile reality capture device 1 can be configured to perform simultaneous localization and mapping (SLAM) to generate a three-dimensional map by involving at least one of the following items: data from an inertial measurement unit (IMU-SLAM), image data from a camera unit for visual SLAM (VSLAM), and light detection and ranging (LIDAR) data from a laser scanner for LIDAR-based SLAM mapping (LIDAR-SLAM).

[0296] In addition to the positioning unit, the reality capture device 1 may be additionally provided with a positioning unit, such as a global satellite navigation system transceiver or a compass, for example for referencing data of the mobile reality capture device with a global coordinate system.

[0297] Figure 2 An exemplary embodiment of a mobile reality capture device comprising a laser scanner 6 and a camera unit with a plurality of cameras 7 is shown.

[0298] The laser scanner 6 has a cover 8 which is opaque to visible light but optically transmissive to the laser measurement beam. The cover 8 together with the base of the laser scanner forms a closed housing which is fixed relative to the mobile reality capture device, wherein all moving parts of the laser scanner 6 are enclosed by the housing.

[0299] For example, the mobile reality capture device is configured to require only a minimum number of controls integrated into the device. For example, the device has only a single integrated control component 9, which has an enabled state and an inactive state and can be switched to present an enabled state or an inactive state via an external action.

[0300] For example, individual measurement procedures and / or actions of the reality capture device may be triggered by at least one of: a change in the state of the control component 9 from an inactive state to an active state, a change in the state of the control component 9 from an active state to an inactive state, switching of the control component 9 by means of a persistent external effect during a defined period (e.g., continuous pressing of a control button), a coded sequence of state changes of the control component 9 between an active state and an inactive state, and a coded sequence of temporary persistent external effects on the control component 9 during a defined period. Such a measurement procedure or action may include at least one of: enabling / disabling the laser scanner 6, starting a defined measurement process, or interrupting / canceling and restarting a measurement process.

[0301] The mobile reality capture device can also be configured such that defined measurement procedures and actions are stored on the device and / or new measurement procedures and actions can be defined by a user, for example via corresponding input functions for assigning commands to states and / or state changes of the control component 9.

[0302] For example, the mobile reality capture device further comprises a light indicator 10, which is used, for example, to indicate the device status in such a way that the status indication appears consistent in all azimuth directions around the vertical axis of the reality capture device. Moreover, the light indicator 10 can be configured to provide guidance instructions (see below).

[0303] Figure 3 Shows Figure 2 1 is a close-up of a laser scanner 6 comprising a base 11 and a support 12, the support 12 being rotatably mounted on the base 11 about a support rotation axis 3. Typically, the rotation of the support 12 about the support rotation axis 3 is also referred to as an azimuth rotation, regardless of whether the laser scanner or the support rotation axis 3 is perfectly vertically aligned.

[0304] The core of the laser scanner 6 is an optical distance measuring unit 13 which is arranged in the support 12 and is configured to perform distance measurements by emitting transmitted radiation 2, for example pulsed laser radiation, and by detecting the returned part of the transmitted radiation by means of a receiving unit comprising a photosensitive sensor. Thus, pulse echoes are received from backscattered surface points of the environment, wherein the distance to said surface points can be derived based on the flight time, shape and / or phase of the emitted pulses.

[0305] In the embodiment shown, the scanning movement of the laser measuring beam about the two rotation axes 3, 4 is achieved by rotating the support 12 relative to the base 11 about the support rotation axis 3 and with the help of a rotating body 14, which is rotatably mounted on the support 12 and rotates about the beam rotation axis 4.

[0306] By way of example, both the transmitted radiation 2 and the returned portion of the transmitted radiation are deflected by means of a reflecting surface 15 which is integral with the rotating body 14 or applied to the rotating body 14 .

[0307] Alternatively, one aspect of the invention relates to the fact that the transmitted radiation comes from the side facing away from the reflecting surface 15, ie from the interior of the rotating body 14 and is emitted into the environment via a passage region within the reflecting surface (see below).

[0308] In order to determine the emission direction of the distance measuring light beam 2, many different angle determination units are known in the prior art. For example, the emission direction can be detected by means of an angle encoder, which is configured to collect angle data for detecting the absolute angular position and / or the relative angular change of the support 12 around the support rotation axis 3 or the absolute angular position and / or the relative angular change of the rotating body 14 around the beam rotation axis 4, respectively. Another possibility is to determine the angular position of the support 12 or the rotating body, respectively, by detecting only full revolutions and using knowledge of the set rotation frequency.

[0309] The visualization of the data may be based on known data processing steps and / or display options, for example wherein the acquired data is presented in the form of a 3D point cloud or wherein a 3D vector file model is generated.

[0310] Figure 4 An exemplary embodiment of a rotating body 14 is shown, which is attached to the support 12 and is configured to deflect the transmitted radiation 2 or a portion of the transmitted radiation returned from the environment about a beam rotation axis.

[0311] The rotating body 14 has a passage area 16 arranged in the reflecting surface for the transmitted radiation 2. The transmitted radiation 2 comes from the side facing away from the reflecting surface 15, ie from the interior of the rotating body 14.

[0312] A portion 17 of the transmitted radiation returning from the environment is deflected by the reflective surface 15 towards the optical window and forwarded, for example, via an additional fixed deflection device 18 to a receiver of the optical measuring unit.

[0313] This so-called “back-injection of transmitted radiation” has the advantage, for example, that a compact design of the laser scanner can be achieved.

[0314] For example, the deflection of the transmitted radiation 2 is achieved by means of a prism 19 arranged in the rotating body 14 and rotating together with the rotating body, wherein the prism deflects the transmitted radiation away from the beam rotation axis, in particular (as in the embodiment shown) to a direction perpendicular to the beam rotation axis.

[0315] The rotating body 14 may be configured in particular to emit the transmitted radiation 2 by the laser source 20 into a free beam section 21 within the rotating body 14 , for example directly or by means of an optical fiber, wherein in particular further additional optics 22 may be arranged within the rotating body 14 .

[0316] Figure 5 and Figure 6 Shows Figure 4 A portion of a cross section of a rotating body 14 includes an exemplary embodiment of a socket 23 formed along the beam rotation axis 4 , wherein the socket 23 is configured to accommodate a prism 19 as a deflection component for the transmitted radiation 2 . Figure 5 The socket 23 is shown without the prism 19, and Figure 6 A socket 23 with a built-in prism 19 is shown.

[0317] By way of example, the socket 23 has a generally cuboid shape for receiving a cuboid prism 19. The edges of the cuboid define a first stabilization plane 24 and a second stabilization plane 25. In the illustrated embodiment, the first stabilization plane 24 has a first contact surface 26 and a second contact surface 27, and the second stabilization plane 25 has a third contact surface 28 and a redundant surface 29 (not used for stabilization of the prism), wherein recesses 30 are formed between the first contact surface 26 and the second contact surface 27 and between the third contact surface 28 and the redundant surface 29. In this case, the first contact surface 26 abuts the third contact surface 28, thereby forming a corner 31, here a 90 degree corner.

[0318] In particular, the prism 19 may have facets releasing the corners 31 , wherein the prism is pressed by means of the fixing assembly 32 in the direction of the corners 31 , usually via an intermediate flip assembly 33 , so that the forces on the three contact surfaces 26 , 27 , 28 are substantially equal.

[0319] A particular aspect of the invention relates to the manufacture of the rotating body 14 or the socket 23, respectively, in particular with regard to a compact design of the rotating body 14 and of the laser scanner in general.

[0320] For example, the rotating body 14 is formed as a rotating body by means of a lathe, wherein the recess 30 of the socket 23 is subsequently produced, for example, by means of drilling, planing or milling. In particular, the rotating body 14 formed in this way can have a socket 23 for a deflection component (for example, a prism 19), wherein the socket 23 (as shown in the image) simultaneously has a reflector axis arranged coaxially with the beam rotation axis 4.

[0321] Alternatively, the rotating body 14 and the socket 23 may be formed by means of an injection molding process, particularly when mass production is required.

[0322] Figure 7 Shows the use of Figure 5 and Figure 6 An alternative to the cuboid prisms shown is a cylindrical prism 19' used as a deflection element for the transmitted radiation within the rotating body.

[0323] Here, the socket has a circular pressing area 31 ′ and a reaction mechanism (eg, two screw holes 34 ) for rotating and fixing the received prism 19 ′ again by means of a fixing component 32 and an intermediate flip component 33 .

[0324] Figure 8 A method for Figure 3 The cover is rotationally symmetrical with respect to the support rotation axis 3 and can be mounted on a substructure (not shown), wherein the cover and the substructure together form the support 12 and the rotating body 14 ( Figure 3 , Figure 4 ) of a housing that is stationary relative to a base.

[0325] The cover has a substantially hemispherical end portion 35 which merges into a cylindrical shell 36 in the direction of the substructure.

[0326] The cover and the substructure are configured to ensure that the total field of view of the measurement operation of the laser scanner 6 is 360 degrees in the azimuth direction defined by the rotation of the support 12 around the support rotation axis 3, and at least 120 degrees in the declination direction defined by the rotation of the rotating body 14 around the beam rotation axis 4. In other words, the cover and the substructure are configured so that regardless of the azimuth of the support 12 around the support rotation axis 3, the transmitted radiation 2 can cover the vertical field of view 38 extending in the declination direction with an extension angle of at least 120 degrees.

[0327] For example, the total field of view generally refers to a central reference point 39 of the laser scanner 6 defined by the intersection of the support rotation axis 3 and the beam rotation axis 4 .

[0328] In the context of the measurement process, the distance measurement is therefore performed based on the transmitted radiation 2 through the tip portion 35 and based on the transmitted radiation 2 through the cylindrical shell 36 .

[0329] The cross section of the boundary of the cover starting at the reference point 39 (in particular the thickness of the boundary as a function of the angle at which the transmitted radiation 2 passes through the cover) affects the beam shape of the transmitted radiation. Therefore, the curvature of the cover may lead to a defocusing of the transmitted beam 2 and thus to an increase in the focal length of the laser scanner 6. Therefore, the boundary is optimized so that spline-based correction parameters can be stored on the laser scanner to correct the beam deflection caused by the cover.

[0330] For example, the border in the head portion 35 has a substantially constant thickness which decreases in the direction of the substructure within the cylindrical shell 35 .

[0331] Furthermore, the cover may, for example, have special optical coatings, in particular an antireflection coating and / or an anti-scratch coating applied by atomic layer deposition (ALD).

[0332] Fig. 9 An exemplary embodiment of a mobile reality capture device 1 ′ with a plurality of light pointers 40 is shown, wherein each of the light pointers is assigned to a scanning segment that is fixed relative to the mobile reality capture device 1 ′.

[0333] The left part of the image shows a side view of the mobile reality capture device 1 ′, while the right part shows a top view of the mobile reality capture device, wherein only the plurality of light indicators 40 are shown in the top view.

[0334] For example, the plurality of light indicators 40 comprises six light indicators 40A, 40B, 40C, 40D, 40E, 40F, such that the arrangement of each of these light indicators on the side surface corresponds to the assigned scanning section of the light indicator: the “forward” direction 41A (opposite to the control part 9, Figure 2 ), the “backward” direction 41B, the “left front” direction 41C and the “right front” direction 41D, and the “left rear” direction 41E and the “right rear” direction 41F.

[0335] For example, the light pointer may provide (e.g., in real time) an indication of a quality parameter of the detection data collected within a corresponding scanning section, or the light pointer may provide guidance from the current position of the mobile reality capture device 1' towards an area in the environment (e.g., an area where additional data needs to be collected) (see Fig.10 ).

[0336] Fig.10 Schematically shows the use of Fig. 9 A scanning workflow of a mobile reality capture device 1', wherein the mobile reality capture device is configured to perform a data check, for example wherein the device is configured to automatically identify the end of a measurement activity based on the identification of a movement towards an exit 42 of a room, automatically triggering a data check at the exit 42.

[0337] The figure shows a top view of a room to be measured, wherein the trajectory 43 of the mobile reality capture device is indicated.

[0338] The device may have access to model data of the room, such as a three-dimensional map generated by a SLAM algorithm of the device or a predefined room model, such as from a Building Information Model (BIM).

[0339] In a first area 44, the reality capture device identifies that the distance to the wall of the room to be measured is outside the nominal distance range for providing the best point resolution of the laser scanner. Therefore, the device is configured to generate guidance data to guide the user towards the identified area 44 where the data quality is insufficient.

[0340] In the second area 45 , the mobile reality capture device identifies a room that was missed during the measurement, for example by taking into account a predefined room model. Also in this case, the device generates guidance data to guide the user to this missed room 45 .

[0341] For example, the plurality of light indicators 40A to 40F ( Fig. 9 ) to provide guidance.

[0342] Fig.11 An exemplary embodiment of a laser scanner including a reference component 46 having a curved surface is shown.

[0343] In the embodiment shown, the reference component 46 has a curved surface that arches outwards (i.e. in the direction of the rotating body 14), wherein the reference component 46 is embedded in a recess in the lowest point area of ​​the support 12. The side wall 4 of the recess, which intersects the scanning plane through the rotating laser measurement beam 2, is configured to act as a radiation swamp.

[0344] Therefore, the reference component 46 has the effect of generating a track of the output laser measurement beam 2 on the curved surface, wherein, depending on the track position, the output laser measurement beam 2 is generated with different incident angles of the curved surface, and different parts of the light are backscattered along the incident direction of the laser measurement beam 2 (see the enlarged view on the right side of the figure, wherein the enlarged view shows details from a side angle relative to the full image).

[0345] Fig.12 An exemplary embodiment of a cooling system for a mobile reality capture device is shown, wherein the device has a first region 48 without rotating parts and a second region 49 comprising a rotating part of a laser scanner to provide a scanning movement of a laser measurement beam.

[0346] The mobile reality capture device has a pumping device (not shown) for driving an air flow including external air, an air inlet 50 for allowing external air 51 to enter the first area 48, and an air channel 52 for transferring air (i.e., air entering across the air inlet 50) from the first area to the second area 49. In the illustrated embodiment, the reality capture device is also configured to transfer air from the first area to an area including cooling fins 53 separately.

[0347] Furthermore, the cooling system includes a two-stage filtering system having: a first filter 54 as at least a rainproof filter, and a second filter 55 having a finer filter fineness than the first filter 54. The first filter 54 is arranged on the air inlet 50 and separates the inner space of the cooling system from the surrounding environment, wherein the second filter 55 is arranged in the inner space and separates the inner space into a dirty inner area 56 and a clean inner area 57, the dirty inner area being located upstream of the second filter 55 between the first filter 54 and the second filter 55, and the clean inner area being located downstream of the second filter between the second filter and an air outlet 58 for releasing air into the surrounding environment. For example, the air outlet 58 includes a filter of the same kind as the first filter to protect the inner space from contamination by return air from the outside.

[0348] Fig.13 A scanning workflow is schematically shown, wherein redundant data is removed by taking into account an evaluation of the geometric relationship between the acquisition location and the area to be probed from the acquisition location.

[0349] The figure shows a top view of the room to be measured, wherein the trajectory 43 of the mobile reality capture device is indicated.

[0350] Here, the mobile reality capture device has a data evaluation unit that is configured to perform an evaluation of the geometric relationship between the acquisition position of the mobile reality capture device and the area to be detected. For example, the mobile reality capture device can have a SLAM unit (e.g., a visual SLAM unit) that provides a three-dimensional map of the environment. This allows the evaluation unit to determine the current position 59 of the mobile reality capture device within the three-dimensional map and derive a geometric relationship for this point relative to the environment surrounding this point.

[0351] For example, the evaluation unit notices that, compared to a previous position 60 of the mobile reality capture device on trajectory 43, a current distance 61 from a previously captured area 62 is now better suited to match the focal length of the camera unit of the mobile reality capture device than a distance 63 corresponding to the previous position 60.

[0352] Thus, image data corresponding to a previous acquisition can be automatically deleted without the need for extensive on-device data analysis. Thus, redundant data close to the sensor can be deleted, which has benefits, for example, in terms of the amount of storage required and the data transfer rate to the associated device.

[0353] Fig.14 A scanning workflow is schematically illustrated, wherein the mobile reality capture device is configured to reinitialize the SLAM unit by invoking the relative position relationship between the SLAM features and the position of the mobile reality capture device along a previous trajectory.

[0354] The figure shows a top view of the room to be measured, wherein the trajectory 43 of the mobile reality capture device is indicated.

[0355] The mobile reality capture device has a SLAM unit configured to generate a 3D map, wherein the 3D map is generated by identifying different salient features within the environment (e.g., corners or features with different reflective properties). For example, the SLAM unit can be based on a visual SLAM algorithm, wherein the SLAM unit matches salient features present in successively generated images to spatially link scenes represented by these images to each other.

[0356] According to this aspect of the invention, the reality capture device also has a feature tracker. For different points along the trajectory, the feature tracker determines and stores the relative position of the identified features 65, i.e., in a local coordinate system 64 associated with the current position. Then, in the event of an interruption in the measurement (e.g., due to a power outage) or in the event of continuation of work on a previous measurement activity, the user must reestablish positioning near the last position along the trajectory, wherein the mobile reality capture device is configured to reinitialize the SLAM unit to continue the generation of the 3D map by calling a series of relative feature positions corresponding to the latest position of the mobile reality capture device along the trajectory 43.

[0357] Fig.15 An exemplary system is shown that includes a mobile reality capture device 1 and a companion device 66 (e.g., a tablet or smartphone).

[0358] In the illustrated embodiment, the companion device 66 is configured to act as a server of a server-client communication protocol, wherein the mobile reality capture device 1 is configured to act as a client. Access data for server-client communication (e.g., a service set identifier (SSID) and a password of the server) is encoded into a matrix barcode 67 (e.g., a QR code) displayed on the companion device 66. The mobile reality capture device has a camera 7 that can take an image of the matrix barcode 67 when manually triggered by a user (e.g., by pressing a control component 9). The mobile reality capture device 1 is then configured to automatically recognize the matrix barcode 67 in the image, automatically decode the matrix barcode, and automatically establish server-client communication when the matrix barcode 67 is decoded.

[0359] Fig.16 An exemplary application of a reality capture device as a monitoring device 100 in a crowded area is shown, here for monitoring a subway station. Typically, a monitoring system comprises a plurality of such monitoring devices 100, which are distributed within the infrastructure to provide full coverage with minimal blind spots.

[0360] A large number of moving objects 101 (e.g., pedestrians, trains, subway passengers, and marketing personnel) are moving in the area, wherein a monitoring system utilizing such monitoring devices 100 may be specifically configured to automatically track the moving objects 101 and automatically detect left-behind objects 102, such as suitcases that may potentially include hazardous materials or explosives.

[0361] Fig.17 An exemplary embodiment of a reality capture device embodied as a monitoring device 100 is shown. The top of the figure shows a side view of the monitoring device 100, and the bottom of the figure shows a top view of the monitoring device 100.

[0362] The monitoring device 100 comprises a laser scanner 6 which is configured to perform a scanning movement of a laser measuring beam relative to two rotation axes and to generate light detection and ranging (LIDAR) data for generating a three-dimensional point cloud based on the scanning movement.

[0363] The base 11 supporting the laser scanner is configured as a common sensor platform that also supports the cameras of the camera unit. In the illustrated embodiment, the camera unit includes: two visual imaging cameras 103, each visual camera 103 has a field of view of at least 180 degrees; and four thermal imaging cameras 104, each of which has a field of view of at least 80 degrees.

[0364] The two vision cameras 103 are placed at an angular interval of 180 degrees on the circumference of the laser scanner 6 , and the four thermal imaging cameras 104 are placed at an angular interval of 90 degrees relative to each other on the circumference of the laser scanner 6 .

[0365] Fig.18 An exemplary embodiment of a socket 68 for attaching a mobile reality capture device to an add-on component is shown.

[0366] The socket 68 has a recess for receiving the pin 70 of the connector 71 of the additional component of the mobile reality capture device along the through-axis 69, and a locking mechanism. The locking mechanism comprises a locking body 72 (e.g., a ball) which, in the locked position, is pushed radially inwardly toward the through-axis 69 so as to engage in a recess 73 of the pin 70. The locking mechanism is configured such that the locked position is its basic position. The locking mechanism can be set to a release position by pushing the contact member 74 in the direction along the through-axis 69, which enables the locking body 72 to radially disengage and thereby release the pin 70.

[0367] In the embodiment shown, the locking mechanism comprises a sliding member 75 which is arranged to be axially movable along the through-axis 69 , wherein a prestressed spring pushes the sliding member 75 into a basic position which pushes the locking body 72 radially inwards.

[0368] The connector 71 comprises a pin 70 with a circumferentially continuous recess 73 , wherein the connector has a release mechanism 76 configured to push a contact member 74 of the socket 68 in a direction along the through-axis 69 in the locked position.

[0369] The top of the figure shows the connector 71 currently moved into the socket 68. The middle of the figure shows the connector fixed in the socket in the locked position. The bottom of the figure shows the connector 71 located in the socket 68, wherein the release mechanism is enabled and the socket is in its release position.

[0370] Fig.19 The workflow using a mobile reality capture device with a radio signal module (eg, WLAN mode) is exemplarily shown for determining the signal strength of a radio signal obtainable along a trajectory 43 of the mobile reality capture device.

[0371] At the top, the figure shows a top view of the room to be measured, wherein the trajectory 43 of the mobile reality capture device is indicated.

[0372] A large number of WLAN transmitters 77 are distributed in the room, wherein for each WLAN transmitter, the propagation of the WLAN signal is indicated. According to this aspect of the invention, the mobile reality capture device is configured to provide a data set including a series of determined signal strengths of the WLAN signals, wherein each signal strength is associated with a position of the mobile reality capture device along the trajectory 43.

[0373] For example, as shown at the bottom of the image, such a data set can then be used to generate a heat map 78 indicating the classification of the room into different radio signal reception areas, such as where no reception areas 79, very strong reception areas 80, strong reception areas 81, medium reception areas 82, and low reception areas 83 are identified.

[0374] Furthermore, such data may be used for WLAN-based positioning, for example to be used by smartphones.

[0375] Although the invention has been illustrated above in part with reference to some preferred embodiments, it must be understood that many modifications of these embodiments and combinations of different features can be made. These modifications all fall within the scope of the appended claims.

Claims

1. A monitoring device for detecting objects in infrastructure, in particular buildings or facilities, more particularly in an area frequently exposed to people and / or vehicles, the monitoring device comprising: a laser scanner configured to perform a scanning movement of a laser measurement beam relative to two rotation axes and, based on the scanning movement, to generate light detection and ranging LIDAR data for generating a three-dimensional model of the infrastructure, a camera unit configured to capture image data along at least two different imaging directions relative to the monitoring device, and an object detector configured to detect an object within the monitored infrastructure based on at least one of the LIDAR data and the image data, in, The monitoring device is configured to generate a three-dimensional model of the infrastructure by taking into account the LIDAR data, in particular a three-dimensional point cloud and / or a three-dimensional vector file model, and The object detector is configured to detect the object based on the three-dimensional model, in particular wherein the object detector is configured to run a feature recognition algorithm on the three-dimensional model and recognize semantic features and / or geometric features based on the feature recognition algorithm.

2. The monitoring device according to claim 1, characterized in that: The camera unit includes at least one of a visual imaging camera and a thermal imaging camera.

3. The monitoring device according to claim 2, characterized in that: The camera unit includes two visual imaging cameras having different viewing directions from each other, wherein each of the two visual imaging cameras has a field of view of at least 180 degrees, and the two visual cameras are arranged circumferentially around the laser scanner, In particular, the two visual imaging cameras are arranged on an outer ring surrounding the laser scanner and defining a central annular axis, wherein the angular spacing of the two visual imaging cameras about the central annular axis is 150 degrees to 180 degrees.

4. The monitoring device according to claim 2 or 3, characterized in that: The camera unit comprises four thermal imaging cameras having different viewing directions from each other, wherein each of the four thermal imaging cameras has a field of view of at least 80 degrees, and the four thermal imaging cameras are arranged circumferentially around the laser scanner, In particular, the four thermal imaging cameras are arranged on an outer ring surrounding the laser scanner and defining a central annular axis, wherein the four thermal imaging cameras are spaced apart from each other at a uniform angular spacing with respect to a rotation about the central annular axis.

5. The monitoring device according to any one of claims 1 to 4, characterized in that: The monitoring device is configured to automatically switch from a low-frequency capture mode to a high-frequency capture mode when an object is detected by the object detector, wherein, in the high-frequency capture mode, at least one of the following parameters is increased compared to the low-frequency capture mode: The point repetition rate of the laser scanner, a rotational speed of the laser measurement beam relative to at least one of the two rotational axes, and The exposure rate of the camera of the camera unit.

6. The monitoring device according to any one of claims 1 to 5, characterized in that: The monitoring device is configured to generate a panoramic image based on the image data, i.e., wherein the individual images of the camera unit in different viewing directions are stitched together to form an image having a wider field of view than the individual images, in particular wherein the mobile reality capture device is configured to generate a 360° panoramic image.

7. The monitoring device according to any one of claims 1 to 6, characterized in that: The monitoring device is configured to generate a data stream and provide the data stream to a separate receiving device by means of a data interface, in particular a wireless and / or wired data interface, in particular wherein the data stream comprises at least one of LIDAR data, image data, model data of the three-dimensional model and image data of a panoramic image.

8. The monitoring device according to claim 7, characterized in that: The monitoring device is configured to automatically switch from a low data mode to a high data mode when an object is detected by the object detector, wherein an amount of data in the data stream is increased in the high data mode compared to the low data mode.

9. The monitoring device according to any one of claims 1 to 8, characterized in that: The monitoring device comprises: a base supporting the laser scanner, and A cover, in particular a cover that is opaque to visible light, is mounted on the base such that the cover and the base surround all moving parts of the laser scanner such that the moving parts are not accessible from the outside.

10. The monitoring device according to claim 9, characterized in that: The cover provides a field of view for the laser scanner, the cover being larger than half of a unit sphere centered at the laser scanner, In particular, wherein the cover has a hemispherical end portion, which is incorporated into the cylindrical shell in the direction of the base, and in particular wherein the laser scanner is configured to generate the LIDAR data based on the following orientations: the orientation of the laser measurement beam passing through the hemispherical end portion; and the orientation of the laser measurement beam passing through the cylindrical shell.

11. The monitoring device according to any one of claims 9 to 10, characterized in that: The cover is made of a material including plastic, wherein the cover has an atomic layer deposition ALD coating on the outside and on the inside, in particular wherein the ALD coating on the outside and / or on the inside is covered by a hard coating.

12. The monitoring device according to any one of claims 9 to 11, characterized in that: The cover has an anti-reflective AR coating on the inside and / or on the outside, and wherein the cover has areas on the inside and / or on the outside in which the AR coating is absent, in particular wherein the AR coating is applied on an inner circumferential band covering a limited height range.

13. The monitoring device according to any one of claims 9 to 12, characterized in that: The laser scanner is configured to rotate the two rotation axes faster than 0.1 Hz, in particular faster than 1 Hz, in order to generate the LIDAR data, wherein the LIDAR data are generated with a point acquisition rate of at least 300,000 points per second, in particular at least 500,000 points per second.

14. The monitoring device according to any one of claims 9 to 13, wherein: The laser scanner comprises: a support portion mounted on the base portion and rotatable relative to the base portion; and a rotating body that deflects an outgoing laser measurement beam and a return portion of the laser measurement beam, the rotating body being mounted on the support and rotatable relative to the support, Wherein, the generation of the LIDAR data includes: continuous rotation of the support portion relative to the base portion and continuous rotation of the rotating body relative to the support portion, and The emission of the laser measurement beam via the rotating body which is continuously rotating, and the detection of a portion of the laser measurement beam which is returned via the rotating body.

15. The monitoring device according to claim 14, characterized in that: The laser scanner is configured such that the continuous rotation of the rotating body relative to the support portion is faster than the continuous rotation of the support portion relative to the base portion. In particular, The continuous rotation of the support is at least 0.1 Hz or 1 Hz, and the continuous rotation of the rotating body is at least 50 Hz.

Images